Judging teachers: Much of what you thought you knew is wrong

A new way of crunching test scores is turning conventional ideas in education on their head. The approach, called value added, has gained momentum in recent months as it has been embraced by the Obama administration and policymakers around the country, though it has generated strong opposition from teachers unions.

This weekend, The Times will examine how a bruising value-added debate played out in San Diego, offering a preview of a controversy likely to sweep the nation. Read the details here: Educator sees the value in 'value-added' approach to evaluating teachers

For years, schools and students have been judged on raw standardized test scores. Experts say this approach is flawed because they tend to reflect socioeconomic levels more than learning.

The "value-added" approach attempts to level the playing field by focusing on growth rather than achievement. Using a complex statistical analysis of test scores, it tracks an individual student's improvement year to year, and uses that progress to estimate the effectiveness of individual teachers, principals and schools.

Academics also have used the approach to test many assumptions about what matters in schools. While the scholars are still puzzling over what makes a great teacher or school, their results challenge many of the things once assumed important:

All teachers are equal. For decades, schools have treated teachers like interchangeable parts. Value-added results suggest that there are dramatic differences in the effectiveness of teachers.

More learning happens in wealthy schools. The highest growth among students is often in poor schools with low achievement scores, according to results at districts and states that have adopted the value-added approach. Students at affluent schools at times have high proficiency scores but make little new progress year to year.

Teachers can't overcome a student's background. Recent research shows that with several effective teachers in a row, students can overcome the disadvantages they bring to the classroom. Some studies suggest that minority and poor students make as much progress as other students when placed with the same effective teachers.

Class size is key. Research suggests that modest changes in class size, such as decreasing it by four or five students, has been shown to have little to no effect on student learning.

Bad teachers tend to teach in poor schools. Several studies suggest that there is more variation among teachers within a school that across schools. Effective instructors often are distributed across rich and poor schools, and they tend to stay in challenging schools longer than ineffective ones.

Teacher experience matters. Although teachers are generally paid more for years of experience, research suggests that instructors show dramatic improvement in their first few years and then level off. Teachers with 20 years of experience are often no more effective than peers with five years.

Teacher education matters. Schools routinely pay teachers higher salaries for obtaining master's degrees. But several studies have found that educators with advanced degrees do no better than those without (with the possible exception of high school math teachers).

Teacher credentials matter. Most public schools pay teachers more for certifications and advanced credentials. But several studies have shown that non-traditionally prepared instructors — such as those in Teach for America — have similar or slightly better outcomes to certified ones.

—Jason Song and Jason Felch

Comments

[LA Teacher]

Advanced degrees DO matter! You gain a wealth of knowledge and resources you can bring to your students. I have a colleague who gained a Master's in his subject area and he said it made him a much better teacher. Education "studies" are fraught with problems. Education "studies" gave us Open Court at the expense of good children's literature and fluency at the expense of comprehension. Who cares how much a kid understands as long as he reads fast.

I am obtaining my Master's and an additional credential and I can definitely say it has paid off for my students. I have gained resources from other teachers in my class and opportunities for my students as well. My university, Cal State LA has a Literacy Lab with children's literature and lesson plans. I know use literature across the curriculum. Advanced degrees are helpful if they are practical and if the professors teach not just theory but practice.

[Debbie]

I will have to wait to read Sunday's paper but it appears this information comes from one group. A group that is about charter schools. I don't have a problem with that but I hope to see firm statistics and facts not just a manipulation of data from one point of view.

[Joanne]

When I was in the 4th grade, there were 41 students. We obeyed the teacher, listened to her, did the projects she set out for us, and learned. Our books were obsolete and the school was being torn down around us, so it could be rebuilt. Earthquake damage. We had maps and a chalk board. That was 1954. Teachers make all the difference in the world.

[Jon]

The start if this piece is already showing its bias. Yes, I can understand the idea, however they need to look at the child as well. I know of a teacher, who a few students did not like. On purpose, these children failed their quarterly district mandated assessment. If there was this whole approach, based on the students previous results, the teacher would have been fired. LAUSD does not have the answers nor does all of UTLA. Let the true people (parents and the concerned teachers as well) talk this problem through. Forget Cortines, forget Monica Garcia and her huge maw of a mouth they are both puppet tools of Villaraigosa and his pathetic attempts to take over the schools.

[Sue]

Irregardless of the method of determining teacher effectiveness, what we are forgetting is exactly what the students are actually learning. As it stands now, our testing measures a very narrow group of skills, and because of the constant pressure to produce higher and higher scores, teachers are forced to drill their students over and over with practice tests after practice tests. On top of that, many states have been caught lowering the bar for proficiency. I believe one of those states is Illinois. So, whatever improvement Mr. Duncan points to in Chicago, it is tainted by those revelations. I don't think the average person realizes that each state writes its own test and sets its own standards. As soon as we include merit pay of any kind, I can predict that this will lead to further narrowing of the curriculum. I wish someone would do a study that follows students after they graduate. How many need remedial classes in college and how many drop out before they receive their degree?

[Amy]

Obviously, an increase in class size of only four to five students may have little effect on student learning. When you increase them by ten or more, which has been the case this year, the effect is keenly felt by both teachers and students. Did that educational study happen to examine those numbers?

[Dan Basalone]

Pre and post testing of students on basic competency skills that are actually taught in the classroom certainly are an indicator of teacher performance. However, individual student needs can hamper the development of other students. For instance, one severe behavior problem student may take up an inordinate amount of teacher time. Also, reducing class size by a few students may not make a difference, but large scale class size reduction does make a difference; that is why high priced private schools advertise their low pupil teacher ratios. This article is a good start for a true discussion of teacher effectiveness. Great teachers come in all ranges of experience...one of the absolute best teachers that I ever supervised was a first year teacher from Loyola Marymount University, and the absolute best kindergarten teacher that I supervised had over thirty years of experience which brought her much credibility with parents.

[Bruce Kuznicki]

Advanced degrees will usually matter if the teacher really wants to become a better teacher. When I was doing my teacher credentialing, the majority of students were quite open about their belief that the courses were a waste of their time and that they took them only because of the legal requirement that they do so. Their objectives were to do only the minimum necessary to complete their coursework. Not surprisingly, these teachers reported not getting much out of their classes and their self reports seemed true. I found most of my coursework relevant and interesting and I believe it helped me a lot.

[lisa davis]

As soon as I see "crunching test scores" and teacher effectiveness in the same article, I know the rest will be a waste of time! I hope anyone reading this knows that this is meaningless. Once more, the L.A. Times shows how much it hates teachers. Should we expect the Spanish Inquisition next? Actually, that might be a fairer way to judge teachers and reporters!

[Los Angeles educator]

This is all about deprofessionalizing teaching as a profession and a craft. After all, if a young colllege graduate can come in, teach for two years and then leave for a more lucrative profession, why we can pay them 30,000 a year with no benefits. The information in this article is bogus. It takes years to become competent. And I made most of my progress as a teacher between years five and six. Explain that.
In addition, the benefits of education are long- term as well as short- term. You can't measure that in our "data driven" society.

[Paula Oglesby]

While the concept is admirable, it is flawed in so many ways. Students behavior is not consistent, especially on testing. Student population turn over at almost 40 percent in some classes is not mentioned, the core that is left is not representational of the whole. Until special needs children are tested with accomodations that are specific to them, their scores are useless,
; yet they are also part of mainstream classes. Again testing becomes the master not the master teacher who does not mind being observed, reviewed, and credited for achievement.
And kudos to the point that principals be scored along side teachers using this format. Not much of article pointed to administrators' union killing that! This same group killed a move to have all principals return to classrooms after a set period in administration. Did this prove that administrators know the classroom is where the real work is done?

[factis]

Unionized public education in the United States has become a corrupt enterprise second only to unionized departments of transportation.

[Dave]

The LA Times serves once again as Jason Song's bully pulpit which he has consistently used to spread vitriol against public school teachers everywhere.

He makes claims about "results" of a study supposedly done in San Diego on teacher effectiveness. Among the claims he makes are that 5th year teachers are just as effective as 20th year teachers. Never mind that no information is given about the source of the data, nor about whether it has been peer reviewed or whether the methods used for gathering data were scientifically acceptable.

Given the source of this information, this is most likely one more attempt by Jason Song to vilify public school teachers, using any source he can grab, no matter how legitimate.

[Evette]

After reading this and other information on value-added teacher evaluations, I remain uncertain how such a system could be used to evaluate my teaching as I see my students for 180 days for five hours each week, after which, they have a different teacher the following year. Does the value-added system mean a change in that I will have the same students over three or more years in order to evaluate my performance? If so, I will welcome the opportunity to work with my students for an extended period of time. Yet this teacher evaluative approach seems to have not considered a number of factors about the reality of inner-city students including the numbers of students who have under-educated parents, the nature of transient populations, attendance and the lack thereof, working students, children in foster and group homes, large class sizes, and a host of other social issues that absolutely have an impact on student achievement in the classroom and on State exams. The idea seems to be that teachers in general, and inner-city teachers specifically, are incompetent, lazy, and choose not to do their best at teaching; however, promising more money as a result of improved test scores will prompt teachers to work harder - insulting to say the least.

I currently teach in the inner-city, and of my 10th grade students, the majority of whom are second language learners; slightly more than 72% of these students read between grade levels 2.9 and 6.7 according to a recently administered Gates McGinte. Will these students be at 10th grade level in time for CSTs this spring? I doubt it. Will I do everything in my power to ensure their reading and writing skills improve between now and spring? Absolutely and with every ounce of dedication I possess. Will I continue to teach inner-city if my pay, thus my ability to support my own family, is dependent upon the my students' exam performance? No way!

I agree that the current evaluation system is flawed, and there is no question that some teachers are better than others, just as the State testing system is flawed and some students are better than others. I also agree that teachers' unions, necessary though they may be, often hamper advancements in teacher evaluations and, in fact, protect incompetent teachers. A new system of teacher preparation and evaluation is necessary, but I do not see that value-added is the answer.

[MVB]

Why exactly is the Times crusading against teachers? It seems to have some stake in attacking the teacher unions and teachers in general. The recent reporting on education has been horribly biased and filled with errors.

[Bob Jones]

Why the hell do we allow this kind of garbage to go on? Every auto mechanic, cashier, newspaper writer, and videogame programmer is judged based on how well they perform at their jobs, meaning that if they're terrible, they're tossed out on their butts. If these completely meaningless jobs are treated this way, why are teachers, who perform some of the most important functions in our society, not judged at least as harshly? But no, because the teachers union bribes the idiots in the legislature, our fifty billion dollar a year education system is nothing more than a jobs program for teachers, no matter how good or bad they are, and California kids are the victims.

[Trying to Teach]

How can this measure a teacher's performance if a teacher is only with a student for a year at best. In high school its only a semester. The article states that the value added method measures students progress over three years and projects where the student will be after that time. So then which teacher of the at least three and upwards of 21 the student has had in that time frame will get credit for the students improvement or lack thereof. And yes, one can say that only the math teacher will receive credit for the student's progress in math, however, its not out of the question for the student's history or english teacher, or a coach, or a tutor to be responsible for the student's progress. How does the model account for that? As much as I'm for improving teachers and their effectiveness in the classrom, the real solution is holding parents and society at large accountable for pushing our students to greater success.

[Chinchu]

We can dance around the issue of poor education in our schools for years to come. No complicated formulas are needed - just open the dorky, slumbering education system to competition. Let the good teachers and the good students gravitate to good private schools and let the dorky, illeterate teachers continue to teach their bad habits to uninterested students in our current public school system. This way everyone gets a choice and everybody wins.

[Andrew Ghiglia]

After reading this article and many others on the subject of how to get "better teachers," I have found myself unsettled and slightly angered by the lack of discussion of the students themselves, and their parents. I teach in a low socioeconomic high school and I am surrounded by excellent teachers. They are not only excellent teachers who I admire, but they care so very much about their student's achievement. Unfortunately, our school is facing closure if it does not improve the test scores. However, there is one huge disconnect at our school with what the teachers are doing and what the students and their parents are doing to drive success.

It states in the article that the most effective teachers in San Diego teach in the wealthiest districts. WAKE UP! It has much less to do with the teachers, as it has to do with the students and their parents who nurture their kids in school oriented environment. Sure there are teachers out there that are tenured and do not have the vigor they once possessed, but so what. The students should still be held accountable for doing their work and learning the subject matter. It seems that we as educators at my school are pushed to be entertainers and motivational speakers to get students to want to learn, and honestly, that is their parents job. We should provide them with the guidance and knowledge to succeed on any and all tests, but we cannot force anything on them.

It is not as easy as it sounds teaching in certain areas in this state and it is most certainly not the teachers fault if students fail to meet state standards.

[Adam Yoon]

As a teacher, I can say that whoever wrote this article do not have all the facts. I always wonder why do these reporters (who have never taught in a classroom) write article like this as a legitimate article? Who are these "some scholars" in this article? Where are the data and stat? Why is the article being so vague as to the source of the article? I read this article and just shake my head (shaking my head now side to side) thinking how does this kind of article end up in a paper.

[Rich Emery]

I first encountered Terry Grier back in 1990, early in his career, when he was superintendent in Akron OH. His top-down management style, inability to achieve real collaboration, and refusal to work with parents in the district was a HUGE reason I found another job. We moved away from Akron primarily to get away from its school system, whose many problems (while not CAUSED by him) were only made worse by Grier.

Interesting to see how school boards across the country are literally such "slow learners", repeatedly hiring this man as superintendent -- in apparently complete ignorance of his track record in previous jobs.

[Seth]

Judging teachers based on student test scores is tantamount to judging a football coach on how fast his players run and how much they lift (and only during one 45 minute period throughout the whole football season). Does being strong and fast during this one training session suggest that you might play good football? Sure, sometimes - just as scoring well on standardized tests might indicate a successful student. But in football the point is not to run a fast 40 meters and benchpress 300 pounds - it is to win football games. Similarly, the point of schooling is not to absorb information and choose the best of four options, it is to prepare the future generations to be successful members of society. Until measures exist that can assess the myriad factors that can predict life success - such as creative thinking and ingenuity, which are totally ignored on standardized tests - the emphasis on these relatively arbitrary measures should be as minimal as their applicability to real life.

The other factor that is ignored in this article is that merit-based pay has been tried before. As was chronicled in the book "Freakonomics," when teachers know that their pay is based on student test scores they will simply start to cheat to get high scores. Whether it be altering the tests themselves or some well-timed eye contact during the testing period, this is not difficult to do. Even honest teachers would simple start teaching to the test to get results that would pay them. This would be instead of teaching units that encourage students to love their education, which is something that might actually have a positive long term effect.

[kay]

As far as the teacher credential part, I agree it shouldn't matter in some subjects, A perfect example: When I was in High School I was involved with Play Production, Stage, and Dance. It was a wonderful Theatre Company with dedicated Teachers in all corners (choir, Acting, Dance, Band, Stage Crew, Film Production etc.) I was a student of an amazing Dance teacher. Not only was she a great choreographer, but was one of the most honest, real and down to earth people I've had the opportunity to know in life. She was directly involved with choreographing our Musicals. She had years of experience and was even in Broadway for years, she was a great asset to what was then a very tight family (which is what it really becomes after being together for 4 years) But then the school would not let her teach dance because she did not have her credentials. She was also an English teacher which is what she was certified to do. They brought in the regular P.E. dance teacher who had the credentials but lacked all the experience and dedication and love for the theater arts.

Why was it such a big deal in this case? It was just a dance? I would understand more if she were teaching Math when she's got dance credentials but that was not the case..

And why is this matter so important for me to share with you all? Because it was being involved in this extra curricular activity at school that made me work harder in school and be a good student. Being a part of the theater company meant keeping AT LEAST a 2.0 grade point average, But when you look at all those students report cards, you see mostly A's and B's our grade point averages were always higher than the standard requirement, we always worked hard to get the highest grades we could possibly can, these kinds of programs WORK. So taking away from them does NOT benefit the students, or the school, it hurts them. I just think that it's something that should be talked about more when talking about things that make a school great, improve student progress, and builds units where teachers work together to deliver on student needs. Any kind of activity that nourishes the students creativity is a bonus for all.

[Kurfco]

I have said for a long time that K-12 education in California is essentially unmanaged. "Management" means setting direction, establishing policies and procedures, and, most importantly, appraising, hiring, firing and paying staff. Most of these functions have long ago been ceded to the teacher unions. For all practical purposes, they decide who is hired and fired, the basis on which teachers are paid, basic policies and procedures -- all in the interests of protecting teachers.

However, there is a ray of sunshine. President Obama, liberal Democrat that he is, and Arne Duncan have seen and understand how destructive untrammeled union power can be. If even the New York labor leaders have awakened and are beginning to smell the coffee, the UTLA better rediscover their sense of smell.

[Kurfco]

When will teachers wake up to a fact that is obvious to everyone: There is not a parent or teacher reading this who doesn't know at least one totally incompetent teacher! If teaching is to be taken seriously as a respected profession, it must be possible to cull teachers that parents and the teachers themselves know are incompetent. I want to get back to respecting teachers and holding them in high regard. When will you allow meaningful appraisals and discharge of poor performers to restore some integrity to your profession? Do you like being held in the same regard as General Motors auto workers?

[Dan Basalone]

I am enjoying the comments based on this article because the evaluation of all personnel (teachers, administrators and classified)as well as the evaluation of students is more of an art than a science. In the cycle of education where you start with planning and implementation and end with assessment and evaluation, educators have given precious little time to evaluation as a necessary component for improvement. How many teachers truly evaluate their students' work and then replan accordingly? How many school employees use their evaluations as an indicator of needed improvement? How many teachers refine their lesson plans based on individual student needs? The answer is that that lesson planning has been downgraded by UTLA under the misguided concept that teacher planning does not need to be monitored by school administrators because it is unprofessional to require adequate plans. Observe in any classroom and look for evidence of adequate planning that reflects student needs, the organization of the classroom for learning and evidence of learning in terms of authentic assessments and you will know who are the "good" teachers. Same is true of schools as a whole in evaluating administrators....schools with an adequate plan, good organization and activities that reflect student learning indicage "good" administrators.

[Wahine]

$ based on growth??!! What about teachers who already have students performing at 97+ percentile??!! Does that make us less effective teachers??

[Martha Infante]

As an educator, neither competition, threats, coercion, or merit pay will improve my teaching ability. Money doesn't keep me up at night wondering how I can sneak food to my homeless student so that his friends don't see. A bonus doesn't compel me to spend endless hours preparing lesson plans to make class informative and meaningful for my students. There is no sum that can buy the feeling I experience when a student finally "gets it" and it changes his/her life. For many educators, no one can hold us to a higher standard than that to which we hold ourselves. This motivation, this calling, is something to which most teachers gravitate, certainly not for the salary and the demeaning treatment we receive in society today. And for the record, I would never have dared to compare myself as a new teacher, to the many wise veterans who guided me and molded me in my early years of teaching. Experience matters in teaching, as it does in medicine, science, and countless other professions.

[Dr. Dennis O'Connor]

So as I read this I wonder if the so called "wealthy" schools will be penalized because their students will be making 'little progress' under your new metric - ignoring the fact that the 8th graders at these schools can out read the majority of adults...

My mother was a teacher, I am a teacher, my daughter is a teacher, I see these new methods come around every 35 or 40 years - as was said by Yogi, Deja Vu all over again...
Yes, the teacher in the class room can make all the difference...
No, we don't get class rooms filled with excellent teachers through the power of collective bargaining...
Yes, the home environment will over power anything done at school (for most kids)...

My grandson is a high achiever in a private school... His mother limits TV to one hour a day during the week... There is a reason that more learning occurs in the wealthy schools - it is because the parents demand it...

dr. o

[Jeff Kilgore]

There are very unintelligent notions posted here. The second idea is just ridiculous:

"Students at affluent schools at times have high proficiency scores but make little new progress year to year."

By this line of reasoning, getting a 16 year-old to learn his or her multiplication tables trumps another student such as my 16 year-old step-daughter who is taking AP Trigonometry. So who is making real progress? The first student can now make change at McDonalds. The second student may really someday perform brain surgery or rocket science. These "facts" are not facts at all, but simply using numbers to get the results that they wanted to see in the first place.

Does it really make sense to believe that better teachers teach in better schools where they can make a living wage?

I'm a liberal democrat, and while I applaud new approaches to education and believe that real change is needed in public education, these facts fly in the face of reality--and in 10 or 20 years, another study will come along and contradict them.

[Joan Sias]

While it is true that AFT president Randi Wiengarten is promoting innovation awards, the article fails to mention that the awards are paid for by the Broad Foundation. By chosing to accept and disburse these Broad Foundation reform award monies, the AFT leadership is doing its membership a great diservice. Will not the AFT now be less enthusiastic and aggressive as in its work to oppose pro-reform lobbying efforts at the state and national levels?

[nikto]

The pitiful Truth is that it is likely JASON SONG is just trying to impress the
Corporate Neocon drivers behind the larcenous "reform" movement.

Maybe he can get a good-paying job out of it after the Times closes down.

Or maybe MR. SONG is just trying to impress Eli Broad for a little personal gain down-the-road, in case Broad buys The Times, which is a possibility.

Corporate Tools are always well-rewarded.

Mr. Song is doing his Neocon duty, and he will do fine.