DPI analysis: All NC students have fallen behind academically during the pandemic

Lost classroom time during the COVID-19 pandemic prevented North Carolina students from performing as well as normal on nearly every state exam, new data shows.

2022-03-02

The state Department of Public Instruction released its first report Wednesday estimating the learning loss attributed to the COVID-19 pandemic.

“Most students continued to progress through the pandemic but at a pace slower than they would have done otherwise,” said Michael Maher, director of DPI’s Office of Learning Recovery and Acceleration, told the State Board of Education during his presentation of the report. The results were especially bad for middle grant math and biology.

“I almost felt like crying,” Board Member Jill Camnitz said, after seeing the report. “It sort of confirms what you knew was coming but seeing the reality is very painful.”

The board must get past that and focus on education, Camnitz said.

Another board member cited anecdotes from educators that they’ve been seeing students make gains this year, indicating some reason to be optimistic.

The report illustrates the challenges educators face in getting students back to grade level expectations, as schools also struggle to hire employees to help them do that.

DPI officials say it also shows the need to close test score gaps between students groups — gaps that mostly appeared to grow wider during the 2020-21 school year.

Department officials and contracted researchers compared students' predicted test scores to their actual tests scores for both last year and four years ago, to see if the trends were far worse during the pandemic than prior to it. By controlling for educator effectiveness scores, the report is intended to capture the impact of the pandemic only and not reflect the impact of teachers.

They found students' learning fell much farther short of predictions during the pandemic and hovered around predicted levels prior to it.

During the 2020-21 school year, reality versus expectation was worse across races, genders, disability status, economic status and academically gifted status.

Growth plummeted even more for students of color, students who lacked Internet access and economically disadvantaged students.

Surprisingly, growth was closer to expectations — but still worse than expected — for students with disabilities than those without identified disabilities and for English-language learners than those who were not English-language learners. It was not clear why.

In most cases, students who spent more days learning in person performed closer to expectations than those who spent more days in a virtual classroom.

It’s the first of several reports planned by the department, which also hopes to continue this same analysis annually going forward. By December, the department will estimate the impact of lost instructional time in terms of “months“ of learning students have fallen behind by.

Lawmakers gave DPI $1 million to contract with a third party to do the research. DPI is using SAS Institute, which works with the department already on education evaluation data.

Most states don’t have statewide data systems; Maher said the fact that North Carolina does made this report possible.

The department will work on analyzing the data on an even more granular level throughout the rest of 2022. Officials are also seeking recommendations from the State Board of Education for areas of focus for the coming months, such as content areas that might need extra attention or students who might need extra help.

Office of Learning Recovery and Acceleration officials determined four priorities of their own: Improving Internet access, ensuring consistent in-person instruction, targeting students disproportionately impacted by the pandemic with extra intervention, and focusing on content areas that had the worst changes in predicted performance, such as reading and middle-grade math.

Jeni Corn, research and evaluation director in the learning recovery office, also noted female students fell farther short of predicted performance than male students did (though male students also fell short of predicted performance).

“Don’t forget about your girls,” Corn said. “This is a place where we can be targeting interventions.”

DPI has some recommendations for catching kids up and plans to track how successful those approaches that are implanted are. Schools are adopting their own approaches as well.

But learning recovery has been limited so far because schools are struggling to hire tutors or other professionals to help them ramp up learning interventions, Maher said.

That’s despite millions of dollars in federal funds designated for those hires.

Maher told WRAL News he and Corn have traveled to school districts in recent months about found the biggest struggle to address learning recovery is staffing, both during the normal school day and for extracurricular programs.

“We know some districts are really trying to do things, like high-dosage tutoring,” Maher said. But they’re struggling to identify nonprofits and other partners to help them amid hiring challenges.