A University of Colorado study released Monday took issue with the Los Angeles Times’ controversial teacher rankings, finding that they were unable to replicate the results that led the Times to rate the effectiveness of elementary school teachers in the Los Angeles Unified School District. 

Published by the National Education Policy Center, the University of Colorado study found only 46 percent of the English and 60 percent of the math classifications were the same for both studies. 

In response to a Tuesday Times article that said the new study confirmed the “broad conclusions” of their August 2010 analysis, Colorado’s lead researcher Derek Briggs said: 

 “I don’t see how one can claim as a lead that our study “confirmed the broad conclusions” – the only thing we confirmed is that when you use a value-added model to estimate teacher effects, there is significant variability in these effects. That’s the one point of agreement. But where we raised major concerns was with both the validity (“accuracy”) and reliability (“precision”), and our bigger focus was on the former rather than the latter. The research underlying the Times’ reporting was not sufficiently accurate to allow for the ratings.” 

Teachers were previously evaluated by the Times by analyzing students’ standardized test scores using the “value-added” model. A database of about 6,000 teachers was released last year, rating teacher effectiveness on a five-level scale in raising scores on standardized math and English tests. 

The report entitled “Due Diligence and the Evaluation of Teachers” by Briggs and co-author and doctoral student Ben Domingue said the Times’ analysis could not be replicated. The Times analysis was conducted by Richard Buddin, a senior economist at RAND Corporation. 

Over 8 percent of teachers that the Times had labeled as ineffective were found to be effective, while 12.6 percent graded effective were found ineffective by Briggs. 

The discrepancy may be in part because Briggs found that Buddin didn’t account for variables including the influence of peers, history of test performance, and school demographics. 

Briggs said he managed to quantify those variables and account for them in his regression equation. He and Domingue also looked at whether students came from high performing or low performing schools. 

The conflicting results bring up an issue of whether or not any model can accurately measure teacher effectiveness. Since the University of Colorado study countered Buddin’s claim that experience and training don’t matter, it also questions what is important for teacher qualifications.  

Reach reporter Jenny Chen here.

Sign up for our weekly e-mail newsletter.