New Policy Brief Released: "What Gets Measured Gets Done: Improving Performance Measures in the Next Generation of Accountability under ESSA"
May 14, 2018
A new study from the Education Research Alliance for New Orleans at Tulane University examines how school ratings would change if states evaluated school performance differently.
As the national debate on how states should measure school performance under the new federal ESSA law continues, the Education Research Alliance for New Orleans released a new study that explores how school ratings would change if states evaluated school performance differently.
The study examines data from Louisiana from 2006 to 2014 and finds that 28.6% of Louisiana high schools would receive different performance ratings (e.g., moving form a letter grade of F to D) if school performance measures evaluated not only test scores and graduation rates but also college entry levels. Lead author Douglas N. Harris said, “If the goal is to hold schools accountable for factors most associated with students long-term life success, then policymakers are probably still too focused on test scores.”
Harris and co-author Lihan Liu also analyze how school ratings would change if states evaluated schools on student growth, or value-added, measures in addition to outcome levels. The study finds that if school performance measures were based on a 50 – 50 mix of achievement levels and achievement value-added, instead of levels alone, then 24.2% of elementary schools and 32.8% of high schools in Louisiana would receive different performance ratings.
“Growth measures are better indicators of what schools actually contribute to student learning because they take into account where students start when they first walk into class,” says Liu. “By focusing so much on outcome levels, policymakers end up punishing schools just because their students started further behind.”
The study estimates the practical impact of using both value-added and levels to measure school performance by simulating a state policy of closing low-performing schools for four consecutive years, similar to the policy used in New Orleans from 2009 to 2014. The authors find that including both levels and value-added measures for test scores and graduation rates when determining school performance ratings would increase annual student achievement levels for the bottom fifth of schools statewide by about 0.4 percentiles and increase the statewide high school graduation rate by 0.4 percentage points.
Their analysis also suggests that switching from levels-only to a mix of levels and value-added for performance ratings based on test scores, high school graduation, and college entry rates would increase the statewide college entry rate by 0.4 percentage points for high schools.
“It appears that accountability policies are unnecessarily sacrificing student outcomes,” says Harris. The authors also noted that their estimates only capture part of the problem. For example, parents increasingly can choose the schools their children attend, and they also pay attention to the performance ratings.
The authors conclude that if state policymakers seek to hold schools accountable for what they can control, and for those outcomes that are most predictive of students’ long-term success, then most states’ ESSA plans are still placing too little emphasis on value-added measures and measures like college entry.