What Gets Measured Gets Done: Improving Performance Measures in the Next Generation of Accountability under ESSA

Over the past three decades, state gov­ern­ments have increas­ing­ly held schools account­able for their per­for­mance, espe­cial­ly stu­dent achieve­ment lev­els. Many states, as a result of the new fed­er­al ESSA law, are also revis­ing the ways they mea­sure and use school per­for­mance to assign school per­for­mance rat­ings, such as A‑F let­ter grades. Most of this brief focus­es on the ques­tion: How would school rat­ings change if states mea­sured school per­for­mance dif­fer­ent­ly? In par­tic­u­lar, how much would school per­for­mance rat­ings change if we added mea­sures, like col­lege entry, that are strong pre­dic­tors of stu­dents’ long-term life suc­cess? Also, how much would school per­for­mance rat­ings change if we focused not on out­come lev­els but on schools’ con­tri­bu­tions to those out­comes, some­times called ​“val­ue-added”? We address these ques­tions using data from Louisiana and find: If pol­i­cy­mak­ers mea­sured high school per­for­mance not only with test scores and grad­u­a­tion lev­els but also with col­lege entry lev­els, then our analy­sis sug­gests that 28.6% of high schools in Louisiana would receive dif­fer­ent per­for­mance rat­ings (e.g., mov­ing from a let­ter grade of F to D). If school per­for­mance mea­sures were based on a 50 – 50 mix of achieve­ment lev­els and achieve­ment val­ue-added, instead of lev­els alone, then 24.2% of ele­men­tary schools and 32.9% of high schools in Louisiana would change per­for­mance cat­e­gories. Val­ue-added can also be used to eval­u­ate school per­for­mance on out­comes oth­er than achieve­ment. If high school per­for­mance mea­sures were based only on grad­u­a­tion, 22.1% of Louisiana high schools would change per­for­mance cat­e­gories if per­for­mance was mea­sured by a mix of grad­u­a­tion lev­els and val­ue-added, instead of grad­u­a­tion lev­els alone. If high school per­for­mance was eval­u­at­ed sole­ly on col­lege entry, 30.7% of Louisiana high schools would change cat­e­gories if per­for­mance was mea­sured by a mix of col­lege entry lev­els and val­ue-added, instead of col­lege entry lev­els alone. For the above analy­ses, the results in New Orleans’ ele­men­tary schools most­ly mir­ror the results across Louisiana. How­ev­er, New Orleans’ high schools are more like­ly to change per­for­mance cat­e­gories than Louisiana high schools when using val­ue-added to high school grad­u­a­tion and col­lege entry. We esti­mate the prac­ti­cal impact of shift­ing toward val­ue-added by sim­u­lat­ing a state pol­i­cy of clos­ing low-per­form­ing schools for four con­sec­u­tive years, sim­i­lar to the pol­i­cy used in New Orleans dur­ing 2009 – 2014. Switch­ing from test score and high school grad­u­a­tion rate lev­els-only to equal weight on lev­els and val­ue-added when choos­ing which schools to close would increase annu­al stu­dent achieve­ment lev­els for the bot­tom fifth of all schools statewide by about 0.4 per­centiles and increase the statewide high school grad­u­a­tion rate by 0.4 per­cent­age points. When we include col­lege entry along­side test scores and high school grad­u­a­tion, switch­ing from lev­els-only to a mix of lev­els and val­ue-added would increase the statewide col­lege entry rate by 0.4 per­cent­age points. Since the choice of per­for­mance mea­sures is impor­tant, we also ask an addi­tion­al ques­tion: As part of their ESSA plans, how many states are plan­ning to add col­lege out­comes and val­ue-added mea­sures to their per­for­mance met­rics? Only 24 states are plan­ning to use val­ue-added or a sim­i­lar mea­sure accord­ing to state ESSA plans, and only 8 of these states are plan­ning to give val­ue-added a weight of 40% or high­er in their over­all per­for­mance mea­sures. While 18 states are plan­ning to include col­lege ​“readi­ness,” only 3 states plan to use actu­al post-sec­ondary out­comes as school per­for­mance mea­sures. If states seek to hold schools account­able for what they can con­trol, and for those out­comes that are most pre­dic­tive of stu­dents’ longterm suc­cess, then most states’ ESSA plans are still plac­ing too lit­tle empha­sis on val­ue-added mea­sures and out­comes like col­lege entry. What gets mea­sured gets done. This is evi­dent in our sim­u­la­tions of school clo­sure and takeover, but oth­er research also clear­ly shows that the mea­sures mat­ter in more sub­tle and indi­rect ways, such as when par­ents col­lect infor­ma­tion to choose schools. If we can improve school per­for­mance mea­sures, then our analy­sis shows that we can improve actu­al stu­dent outcomes.

Related Publications

How Do Schools Respond to State Policies on Teacher Evaluation?

Apr 27, 2017 | by Julie A. Marsh, Susan Bush-Mecenas, Katharine O. Strunk, Jane Arnold Lincove, Alice Huguet

Extreme Measures: When and How School Closures and Charter Takeovers Benefit Students

Oct 17, 2016 | by Whitney Bross, Douglas N. Harris, Lihan Liu

The Ultimate Choice: How Charter Authorizers Approve and Renew Schools in Post-Katrina New Orleans

Sep 12, 2016 | by Whitney Bross, Douglas N. Harris