Judging teachers: Much of what you thought you knew is wrong

A new way of crunching test scores is turning conventional ideas in education on their head. The approach, called value added, has gained momentum in recent months as it has been embraced by the Obama administration and policymakers around the country, though it has generated strong opposition from teachers unions.

This weekend, The Times will examine how a bruising value-added debate played out in San Diego, offering a preview of a controversy likely to sweep the nation. Read the details here: Educator sees the value in 'value-added' approach to evaluating teachers

For years, schools and students have been judged on raw standardized test scores. Experts say this approach is flawed because they tend to reflect socioeconomic levels more than learning.

The "value-added" approach attempts to level the playing field by focusing on growth rather than achievement. Using a complex statistical analysis of test scores, it tracks an individual student's improvement year to year, and uses that progress to estimate the effectiveness of individual teachers, principals and schools.

Academics also have used the approach to test many assumptions about what matters in schools. While the scholars are still puzzling over what makes a great teacher or school, their results challenge many of the things once assumed important:

All teachers are equal. For decades, schools have treated teachers like interchangeable parts. Value-added results suggest that there are dramatic differences in the effectiveness of teachers.

More learning happens in wealthy schools. The highest growth among students is often in poor schools with low achievement scores, according to results at districts and states that have adopted the value-added approach. Students at affluent schools at times have high proficiency scores but make little new progress year to year.

Teachers can't overcome a student's background. Recent research shows that with several effective teachers in a row, students can overcome the disadvantages they bring to the classroom. Some studies suggest that minority and poor students make as much progress as other students when placed with the same effective teachers.

Class size is key. Research suggests that modest changes in class size, such as decreasing it by four or five students, has been shown to have little to no effect on student learning.

Bad teachers tend to teach in poor schools. Several studies suggest that there is more variation among teachers within a school that across schools. Effective instructors often are distributed across rich and poor schools, and they tend to stay in challenging schools longer than ineffective ones.

Teacher experience matters. Although teachers are generally paid more for years of experience, research suggests that instructors show dramatic improvement in their first few years and then level off. Teachers with 20 years of experience are often no more effective than peers with five years.

Teacher education matters. Schools routinely pay teachers higher salaries for obtaining master's degrees. But several studies have found that educators with advanced degrees do no better than those without (with the possible exception of high school math teachers).

Teacher credentials matter. Most public schools pay teachers more for certifications and advanced credentials. But several studies have shown that non-traditionally prepared instructors — such as those in Teach for America — have similar or slightly better outcomes to certified ones.

—Jason Song and Jason Felch

Comments

[Dan Basalone]

I am enjoying the comments based on this article because the evaluation of all personnel (teachers, administrators and classified)as well as the evaluation of students is more of an art than a science. In the cycle of education where you start with planning and implementation and end with assessment and evaluation, educators have given precious little time to evaluation as a necessary component for improvement. How many teachers truly evaluate their students' work and then replan accordingly? How many school employees use their evaluations as an indicator of needed improvement? How many teachers refine their lesson plans based on individual student needs? The answer is that that lesson planning has been downgraded by UTLA under the misguided concept that teacher planning does not need to be monitored by school administrators because it is unprofessional to require adequate plans. Observe in any classroom and look for evidence of adequate planning that reflects student needs, the organization of the classroom for learning and evidence of learning in terms of authentic assessments and you will know who are the "good" teachers. Same is true of schools as a whole in evaluating administrators....schools with an adequate plan, good organization and activities that reflect student learning indicage "good" administrators.

[Wahine]

$ based on growth??!! What about teachers who already have students performing at 97+ percentile??!! Does that make us less effective teachers??

[Martha Infante]

As an educator, neither competition, threats, coercion, or merit pay will improve my teaching ability. Money doesn't keep me up at night wondering how I can sneak food to my homeless student so that his friends don't see. A bonus doesn't compel me to spend endless hours preparing lesson plans to make class informative and meaningful for my students. There is no sum that can buy the feeling I experience when a student finally "gets it" and it changes his/her life. For many educators, no one can hold us to a higher standard than that to which we hold ourselves. This motivation, this calling, is something to which most teachers gravitate, certainly not for the salary and the demeaning treatment we receive in society today. And for the record, I would never have dared to compare myself as a new teacher, to the many wise veterans who guided me and molded me in my early years of teaching. Experience matters in teaching, as it does in medicine, science, and countless other professions.

[Dr. Dennis O'Connor]

So as I read this I wonder if the so called "wealthy" schools will be penalized because their students will be making 'little progress' under your new metric - ignoring the fact that the 8th graders at these schools can out read the majority of adults...

My mother was a teacher, I am a teacher, my daughter is a teacher, I see these new methods come around every 35 or 40 years - as was said by Yogi, Deja Vu all over again...
Yes, the teacher in the class room can make all the difference...
No, we don't get class rooms filled with excellent teachers through the power of collective bargaining...
Yes, the home environment will over power anything done at school (for most kids)...

My grandson is a high achiever in a private school... His mother limits TV to one hour a day during the week... There is a reason that more learning occurs in the wealthy schools - it is because the parents demand it...

dr. o

[Jeff Kilgore]

There are very unintelligent notions posted here. The second idea is just ridiculous:

"Students at affluent schools at times have high proficiency scores but make little new progress year to year."

By this line of reasoning, getting a 16 year-old to learn his or her multiplication tables trumps another student such as my 16 year-old step-daughter who is taking AP Trigonometry. So who is making real progress? The first student can now make change at McDonalds. The second student may really someday perform brain surgery or rocket science. These "facts" are not facts at all, but simply using numbers to get the results that they wanted to see in the first place.

Does it really make sense to believe that better teachers teach in better schools where they can make a living wage?

I'm a liberal democrat, and while I applaud new approaches to education and believe that real change is needed in public education, these facts fly in the face of reality--and in 10 or 20 years, another study will come along and contradict them.

[Joan Sias]

While it is true that AFT president Randi Wiengarten is promoting innovation awards, the article fails to mention that the awards are paid for by the Broad Foundation. By chosing to accept and disburse these Broad Foundation reform award monies, the AFT leadership is doing its membership a great diservice. Will not the AFT now be less enthusiastic and aggressive as in its work to oppose pro-reform lobbying efforts at the state and national levels?

[nikto]

The pitiful Truth is that it is likely JASON SONG is just trying to impress the
Corporate Neocon drivers behind the larcenous "reform" movement.

Maybe he can get a good-paying job out of it after the Times closes down.

Or maybe MR. SONG is just trying to impress Eli Broad for a little personal gain down-the-road, in case Broad buys The Times, which is a possibility.

Corporate Tools are always well-rewarded.

Mr. Song is doing his Neocon duty, and he will do fine.

[John M. Wood jr.]

Hey, Bob, who's your mechanic? We need to talk