The reanalysis of tutoring program impacts shows how researchers can compare the relative benefits of tutoring programs to help educators select the most cost-effective providers
Nashville, Tenn.—Today the national nonprofit Accelerate introduced an innovative approach to determining the efficiency and cost-effectiveness of tutoring programs in its new publication, “Contextualizing the Impact of Tutoring on Student Learning: Efficiency, Cost Effectiveness, and the Known Unknowns.” The report proposes two new metrics for evaluating the time efficiency and cost-effectiveness of high-dosage tutoring programs. These new metrics make it possible for future research to assess the return on investment of various tutoring programs, with the ultimate goal of helping schools make evidence-informed decisions when selecting tutoring providers.
“Every school and district leader wishes they had more of two things: time and money, and budgets are even more constrained now that Covid-relief aid is ending,” said Nakia Towns, COO at Accelerate. “At the same time, we know that schools are going to continue investing in learning recovery interventions. So the question researchers now need to answer is: How do we get the biggest bang for our buck? The clearer we can make it for districts to see which interventions are the best use of their precious time and money, the more likely it is that they will choose programs that will really make a difference for kids.”
To construct this new approach, Accelerate first reanalyzed tutoring program impact data from 14 high-quality randomized controlled trials (RCTs) of 12 tutoring providers that meet the evidence bar researchers, educators, and policymakers should demand all providers meet. These RCTs were first analyzed in a widely-cited 2023 study by Andre Nickow, et al. In their reanalysis, Accelerate authors Matthew Steinberg and Luke Kohlmoos propose and describe measures of the efficiency (defined as hours of tutoring necessary to improve student learning by one month) and cost-effectiveness (defined as additional months of student learning produced at a cost of $1,000 per pupil) of 12 tutoring programs to create a novel methodology for future researchers to compare the expected return on investment for any given tutoring provider.
In addition to establishing a new methodology for comparing programs, the reanalysis finds:
- Tutoring remains the most effective academic intervention, outperforming reduced class sizes, summer school, and extended school years.
- Tutoring efficiency varies widely not just across programs, but also across subject area.
Across the board, high-dosage math tutoring appears more efficient than high-dosage literacy tutoring, with less variability among providers. - For future researchers to adequately measure cost-effectiveness or the return on investment, tutoring providers must supply more transparent data about the cost of their programs.
“This research is far from the last word on efficiency and cost-effectiveness in the tutoring space — it’s closer to the start of a conversation,” said Matthew Steinberg, Managing Director of Research and Evaluation at Accelerate. “We have to start talking about tutoring in a more empirically-informed way than ever before. Up to this point, research has been focused on discovering how effective high-dosage tutoring is relative to other interventions like extended learning time and reduced class size, and we now know that tutoring is the most effective strategy for raising student achievement among a host of educational interventions. This analysis aims to orient researchers toward a new question: What program models offer the best use of schools’ limited resources — time and money?”
Accelerate makes the following recommendations in the report:
- Researchers should focus on large-sample evaluations to produce rigorous evidence on tutoring effectiveness so that education leaders can make the most cost-effective decisions when selecting tutoring providers
- Tutoring providers should gather data on all students they serve, not just those who receive the full dosage, and should provide more transparency around the cost of their programs
- Federal agencies should lead high-quality research efforts and enact policies that incentivize tutoring providers to meet rigorous standards
- States should identify and curate a list of vetted, evidence-based high-dosage tutoring programs that meet a standard of efficiency and mandate cost reporting for vetted tutoring providers that make it onto state lists
- School districts should build tutoring into their budgets for the long term and rely on evidence to select only the most effective and efficient programs
“Selecting a tutoring vendor is not the same thing as buying No. 2 pencils,” said Dan Goldhaber, director of the CALDER Center at the American Institute for Research. “There’s so much variation in these programs — how they work, what they offer — and it’s really important that policymakers move away from thinking about tutoring as a generic commodity. There are important nuances in tutoring programs that likely explain what makes some programs so much more effective than others. This report is an important step toward helping districts and schools make good decisions about the features of tutoring and better understand the outcomes they should expect for their students given the money they are investing in tutoring.”
To read a summary or to download the full report, click below.
