NKY by the numbers: What we can learn from Boone County Schools

It’s been five years since the start of the COVID-19 pandemic, and Kentucky student test scores have not returned to the levels they were prior to the onset of the lockdowns.

That’s according to data from the National Assessment for Educational Progress, or NAEP, which measures national trends in learning progress.

A newer analysis of state and national test scores through something called The Education Recovery Scorecard shows the average student remains half a grade level behind pre-pandemic achievement in both reading and math. In reading, especially, students are even further behind than they were in 2022, the analysis suggests.

Districts around the country have been experimenting with different measures to curb academic backsliding, and one local district, Boone County Schools, stood out in the Education Recovery Scorecard’s analysis as demonstrating some of the highest gains among the 500 districts analyzed throughout the nation.

It had some of the biggest gains in reading among large districts. It also had some of the biggest gains in math, including ranking third for math gains among Black students.

So, what’s the district doing right? And what can data from the project tell us about what any district can do to aid in student academic recovery?

The Education Recovery Scorecard

The project did a massive data analysis of national test score data (from NAEP) and state-level test score data and standardized it to track trends over time. This allowed them to make comparisons between states and districts in a predictable way, which is sometimes difficult because states will often change their testing metrics from year to year. Standards also vary from state to state.

In this way, the analysis allows for a kind of apples-to-apples comparison between states, districts and demographics.

To make it simple, the research project converted progress into grade levels. Thus, a district with a reading score of 1.0 saw an increase in reading recovery by one grade level from a previous data set. Conversely, a district with a reading score of -1.0 saw a decline in one reading grade level.

NAEP collected its most recent data set in spring 2024. By then, the worst of the pandemic was long past, but schools were still dealing with a mental health crisis and high rates of absenteeism, not to mention students who’d had crucial learning disrupted.   

“The losses are not just due to what happened during the 2020 to 2021 school year, but the aftershocks that have hit schools in the years since the pandemic,” said Tom Kane, a Harvard education economist who worked on the scorecard.

Stanford Education Sociologist Sean Reardon, who also worked on the scorecard, gave an overview of the national trends that emerged from the research.

“There’s enormous amount of variation across districts in how test scores have changed over the last five years, but a couple patterns stand out,” Reardon said. “One is that the highest poverty districts have fallen behind more than the lowest poverty districts. So, the gap between those high and low poverty districts was about 3.7 grade levels in 2019, and now it’s 4.1 grade levels. It’s about four tenths of a grade level larger, or about 11% larger. That’s a sizable growth in the disparity between poor and non-poor districts.”

Similar gap increases were observed between white and Black students, as well as between white and Hispanic students, Reardon said. Girls have also been falling behind boys in math assessments since 2019, Reardon added, in contrast to data from before 2019, when math assessment scores between boys and girls were more similar.

It’s important to note that the research scores are relative rather than absolute. In other words, they measure changes over time – increases and decreases in comparison to preceding data sets – instead of raw performance numbers.

Although Boone County Schools, for instance, scored high in the relative metrics from 2019 to 2024, its overall performance metrics are still lower than some of the higher-achieving districts in the region, according to state data. Likewise, some of the local districts that have high raw performance numbers saw declines in relative scores. The only other local district besides Boone County that saw across-the-board improvement from 2019 to 2024 was Walton-Verona Independent.

The case of Boone County

Although the district still suffers from a rate of chronic absenteeism similar to the state average for districts of its size, Boone County Schools saw overall increases in both math and reading achievement compared to other Kentucky districts of comparable size.

Additionally, every demographic group in grades 3 through 8 saw achievement improvements.

Why?

Administrators and academic specialists within the district attributed the improvements to several factors. The first was the district’s insistence that teachers teach to established standards rather than getting bogged down in trying to repair standards students may have missed during the lockdowns.

“Coming out of the pandemic, I think it’s very much human nature to want to try to catch everybody up,” said Chief Academic Officer Jim Detwiler. “We were super clear with the teachers: When we’re back to normal, whatever that is, we’re not going to spend time going back and trying to teach what we missed. We’re going to stay the course.”

Detwiler said the term the district uses for this philosophy is “accelerated learning.”

Kelly Stidham and Tina Withorn, the district’s math and reading consultants, respectively, described the mechanical underpinnings of this problem.

“If we’re teaching below grade level because it seems like that’s what kids need, then each year kids are carrying over part of a grade level that they haven’t learned,” said Withorn.”

Educational specialists call this the iceberg problem. Compounded year over year, it can lead to “exponential learning loss if teachers are starting below grade level,” Withorn said. This phenomenon is especially problematic for mathematics.

In an accelerated learning model, Stidham said, “You say, ‘I’m going to keep my high-level expectations so students are working on the most important things, and I’m going to fill any uninterrupted learning as we go.’ Versus a remediation model, which is where I just work on that filled gap without ever really reaching rich, meaningful learning. I just focus on all of the deficits.”

To augment the core teaching responsibilities of the teachers, the district employs interventionists who provide more intensive, individualized instruction for students who need it. In short, the schools fill in the gaps as they go, rather than languishing in lower-level material.

The main part of Withorn and Stidham’s job, however, is ensuring that teachers have the tools they need to keep the kids on track. They do this by providing ongoing teacher training to ensure their methods are up to snuff.

“We manage curriculum, really working on creating systems for coherence and consistency,” Withorn said. “We know that those two things are key for students to have equitable learning across the district, across the grade levels. We create and train [and] facilitate professional learning around grade level expectations in our disciplines.”

“We spent a great deal of [COVID emergency] funding for strong instructional resources and investing in professional learning time for teachers,” Stidham said.

Ensuring the students are challenged with high expectations is one point they harped on in their conversation with LINK nky. They pointed to 2018 study from The New Teacher Project, a research organization started by Wendy Kopp, the founder of Teach for America, called “The Opportunity Myth.”

The study examined lesson plans, assignments, and student behavior across five diverse school systems to investigate why more students seemingly left school without developing the prerequisites necessary to succeed in their life goals.

“Millions of students across the country are working hard to get through school, only to find themselves ill-prepared to live the lives they hope for,” the study reads. “They’re planning their futures on the belief that doing well in school creates opportunities—that showing up, doing the work and meeting their teachers’ expectations will prepare them for what’s next. They believe that for good reason: We’ve been telling them so. Unfortunately, that’s a myth.”

Students don’t learn by osmosis, the study argued. Showing up and going through the motions isn’t enough. How one teaches is just as important as what one teaches. The study came away with four primary recommendations for teachers who want their students to succeed long-term:

• Consistently employ assignments appropriate for the students’ grade levels
• Use lessons where the students are doing most of the thinking themselves (as opposed to, say, only lecturing them)
• Challenge students to engage deeply with the topics taught
• Cultivate teaching environments where students are held to high expectations, where students believe they’re capable of performing at grade level

When it came to deep engagement with math, for instance, Stidham emphasized that lessons need to impart an understanding of the underlying mechanics behind a particular operation and demonstrate how it plays out in the real world rather than simply manipulating the symbols on the page.

“Math should just make sense because it’s how the universe works,” Stidham said. “But our kids don’t always feel that. A lot of times, they feel like math is something arbitrary that’s just given to them.”

“Applying knowledge in new situations, I think, is a weakness and something that we can help teachers really do in the classroom,” Withorn said, “Not just regurgitate knowledge, but how to take what you’ve learned and apply it to a new situation.”

Limitations and future considerations

Looking at the numbers alone helps track progress, but doesn’t provide a complete picture. Even though the scorecard established trends based on demographic and income levels, its analysis couldn’t consider the underlying conditions that might lead a particular district to have a higher poverty rate, for instance. As such, considerations for teaching practices are only one factor a district needs to consider when addressing learning loss.

Additionally, even in districts such as Boone County that demonstrated higher recovery levels, many students in Kentucky and throughout the nation still struggle to perform at grade level. Less than half of Boone County 10th graders, for instance, performed at proficient and distinguished (the two highest) levels on the most recent Kentucky summative math assessments, according to the Kentucky Department of Education.

“I would say six out of 10 of our students are reading on grade level, where 4 out of 10 are not,” Detwiler admitted. “So, we still have a lot of work to do. [The scorecard] was reassuring, but at the same time, it’s not something we can rest on for sure.”

The scorecard’s analysis came away with a list of recommendations for districts to consider when it came to addressing learning loss brought about by the lockdowns:

• “States and districts should double down on academic catch-up efforts previously funded by federal relief. In the absence of federal pandemic relief, states will need to redirect their own dollars and the federal Title I dollars they administer for interventions which have been shown effective, such as tutoring and summer learning.
• “Mayors, employers and other community leaders should join schools in tackling student absenteeism. Rather than place the responsibility for academic recovery entirely on school leaders’ shoulders, reducing absenteeism is one burden that others can help schools carry. Such help could include public information campaigns, extracurricular activities to draw students to school and solving transportation problems.
• “Teachers must inform parents when their child is not at grade level. Since early in the recovery, the overwhelming majority of parents have been under the false impression that their children were unaffected. Parental perceptions are central to many of the challenges districts face. If they are to help lower absenteeism, sign up for summer learning and increase reading at home, parents need to know when their child is behind. And teachers are often the most credible source to tell them.
• “We must learn what’s working (and what is not) in the recent reforms. In the last few years, 40 states have implemented ‘science of reading’ reforms. But each state has taken a different approach, placing different emphasis on curriculum, teacher training, coaching and retaining students who do not demonstrate reading proficiency. In addition, many have implemented cell phone bans. Such policy innovation can be a strength of our federal system—but only if we learn which of those efforts are working (and which are not) and spread the most effective solutions.”

You can read a summary of the scorecard’s data analysis and reporting methodology here: “Pivoting from Pandemic Recovery to Long-term Reform: A District-Level Analysis.”

Check out fact sheets summarizing data analysis for each of the region’s districts included in the project’s analysis at the links below. Math data for Bellevue Independent was not available, and the project did not analyze Southgate Independent at all.

• Beechwood Independent
• Bellevue Independent
• Boone County
• Campbell County
• Covington Independent
• Dayton Independent
• Erlanger-Elsmere Independent
• Fort Thomas Independent
• Kenton County
• Ludlow Independent
• Newport Independent
• Walton-Verona Independent

Annie Ma and Jocelyn Gecker of the Associated Press contributed reporting to this story.