Pandemic Schooling Mode and Student Test Scores: Evidence from US States
- Date Posted:
- Is Database:
- Database
US school districts that reduced in-person classes during the pandemic saw significant learning losses, with math pass rates declining by 14.2% points. In contrast, districts with full in-person learning saw a smaller decline of 10.1% points.
"…We estimate the impact of district-level schooling mode (in-person versus hybrid or distance learning) on test scores. We combine Spring 2021 state standardized test score data with comprehensive data on schooling in the 2020-21 school year across 12 states. We find that pass rates declined compared to prior years and that these declines were larger in districts with less in-person instruction. Passing rates in math declined by 14.2 percentage points on average; we estimate this decline was 10.1 percentage points smaller for districts fully in-person. Changes in English language arts scores were smaller, but were significantly larger in districts with larger populations of students who are Black, Hispanic or eligible for free and reduced price lunch…”
“…Panel A of Figure 2 shows the year-on-year changes in math test scores by group for prepandemic years (left figure) and for the pandemic year (right figure). In most cases, the test score declines in the pandemic year are larger in the districts with less in-person instruction. This difference is largest in Virginia. There is no evidence that this pattern aligns with the pre-pandemic years. Prior year-on-year changes do not seem to systematically vary with 2020-21 schooling mode. Panel B shows the results for ELA. While the overall changes are smaller in ELA as compared to math, the pattern is similar….”
“…Figure 1 illustrates the changes in pass rates across states during the pandemic period (the 2020-21 school year) and in several previous periods (the 2015-16 school year through 2018-19). Panel A shows the scores in math and Panel B presents the scores in ELA. The solid triangles represent the change in pass rates between the 2018-19 school year (the last pre-pandemic year) and the 2020-21 school year. The open circles show prior year-on-year changes. Our analysis finds a decline in pass rates from the 2018-19 school year to the 2020-21 school year in all states. The decline ranges from -31.9 percentage points (Virginia, math) to -2.3 percentage points (Wyoming, ELA). Based on the prior year changes, typical year-on-year variation is very small. The pandemic year is a significant outlier. The changes in pass rates for math (Panel A) are considerably larger than the changes in scores for ELA (Panel B). This is broadly consistent with a larger literature which shows math scores are more responsive to schooling differences(Betts and Tang, 2011; Angrist et al., 2013)….”
The Evidence
Core findings, “…In this paper, we use data on district schooling modes alongside Spring 2021 test score data from 12 states to analyze the relationship between schooling mode and test scores. We estimate the relationship between the share of the year that in-person learning was offered (as opposed to hybrid or virtual learning) and student standardized test scores, both overall and interacted with district demographics. Our main regression specification uses a standard panel data approach, with data at the district-year level, to estimate the impact of the time spent in-person on 2021 test scores. These regressions include district-specific trends and a full set of demographics and year interactions. We demonstrate first that, across all 12 states in our sample, there were large reductions in test scores between 2020-21 and previous years. The average decline in math is 14.2 percentage points, versus 6.3 percentage points in English language Arts (ELA). These changes are well outside typical year-on-year variation. Following this, we show our main finding: the decline in students' 2021 test scores as compared to prior years was significantly larger in districts which offered less access to in-person schooling. For example, we estimate that offering full in-person instruction rather than fully hybrid or virtual instruction reduces test score losses in math by 10.1 percentage points (on the base of 14.2 percentage points). In ELA, the loss is reduced in fully in-person settings by 3.2 percentage points.These interactions between test score losses and schooling mode are highly significant. We demonstrate, as well, that access to in-person learning varied by demographics and baseline test scores. Districts with lower baseline test scores and a higher share of Black students offered less in-person learning. Within states, districts with a higher share of Hispanic students also offered less in-person learning (although across states this effect is reversed). Finally, we observe that the impact of access to in person learning on 2021 test scores also varies by subject and demographics. Specifically, among districts with a larger share of Black and Hispanic students, districts with less in-person schooling saw a greater decline in ELA test scores than those with more in-person schooling. Although the impact of schooling mode on ELA is fairly small for districts which are majority white, it is large for those districts with a majority of students of color. Meanwhile, the impact of access to in-person learning had a similar effect on math scores for all districts, regardless of their racial composition….”
Clare Halloran, Rebecca Jack, James Okun and Emily Oster, “Pandemic Schooling Mode and Student Test Scores: Evidence from US States,” National Bureau Of Economic Research, November 2021, https://www.nber.org/papers/w29497



