Macro Roundup

Summery, “…Intelligence test scores and educational duration are positively correlated. This correlation could be interpreted in two ways: Students with greater propensity for intelligence go on to complete more education, or a longer education increases intelligence. We meta-analyzed three categories of quasi-experimental studies of educational effects on intelligence: those estimating education-intelligence associations after controlling for earlier intelligence, those using compulsory schooling policy changes as instrumental variables, and those using regression-discontinuity designs on school-entry age cutoffs. Across 142 effect sizes from 42 data sets involving over 600,000 participants,we found consistent evidence for beneficial effects of education on cognitive abilities of approximately 1 to 5 IQ points for an additional year of education.Moderator analyses indicated that the effects persisted across the life span and were present on all broad categories of cognitive ability studied. Education appears to be the most consistent, robust, and durable method yet to be identified for raising intelligence….”

Stuart Ritchie and Elliot Tucker-Drob, "How Much Does Education Improve Intelligence? A Meta-Analysis,"Psychological Science, 2018, https://labs.la.utexas.edu/tucker-drob/files/2019/08/Ritchie-Tucker-Drob-2018-Psych-Science-How-Much-Does-Education-Improve-Intelligence.pdf

“….Table 2 shows the results from each study design after the inclusion of multiple moderators. We included as many moderators as possible for each design, though we chose to include the “achievement” categorization of the outcome test for the school-age-cutoff design (instead of the alternative categorization involving composite, fluid, and crystallized tests, which we used for the other designs) because it had such a substantial effect in the single-moderator model. Including multiple moderators reduced the τ statistic—to a larger degree for the control-prior-intelligence design than for the others— though significant heterogeneity remained in all cases. The moderators that were individually significant (e.g., age in the control-prior-intelligence design or achievement tests in the school-age-cutoff design) were also significant in the multiple-moderator model, indicating that their effects were incremental of the other moderators that we included….”

“…We tested whether studies that included only male participants showed a differential educational effect. This was not the case for control prior intelligence (effect for the 2 male-only estimates: 2.261 IQ points, SE = 0.897, p =.012; effect for 24 mixed-sex estimates: 1.027 IQ points, SE = 0.110, p = 6.79×10−21; difference SE = 0.905, p =.173), or for policy change (effect for the 8 male-only estimates: 1.683 IQ points, SE = 0.507, p =.001; effect for 22 mixed-sex estimates: 2.215 IQ points, SE = 0.788, p =.005; difference SE = 0.941, p =.572). There were no male-only school-age-cutoff studies….”

Multiple moderators.

“….Splitting the outcome cognitive tests into three broad categories—composite, fluid, and crystallized tests—we tested whether the category moderated effect sizes. For the control-prior-intelligence design, there were stronger educational impacts on composite tests (1.876 IQ points, SE = 0.467, p = 5.94×10−05) than on fluid tests (0.836 points, SE = 0.097, p =.152), and the difference between composite and fluid was significant (1.039 points, SE = 0.496, p =.036).There was only one crystallized outcome test for the control-prior- intelligence design (1.893 points, SE = 0.348, p = 5.34×10−08), so we did not include it in the moderator comparison here. For the policy-change design, there were significant effects for both composite (2.314 IQ points, SE = 0.869, p =.008) and fluid (2.272 points, SE = 0.765, p =.003) but not crystallized (1.012 points, SE = 1.125, p =.368) tests; however, the effects on the three different categories were not significantly different from one another (all difference p values >.35). Finally, for the school-age-cutoff design, there were significant effects of a year of education on composite (6.534 points, SE = 2.433, p =.007), fluid (5.104 points, SE = 0.621, p = 2.05×10−16), and crystallized (5.428 points, SE = 0.170, p = 1.04×10−223) tests; there were, however, no significant differences between effect sizes across outcome types (difference p values >.5). We then split the outcome tests into “achievement” tests versus “other” tests. There was only one achievement test in the control-prior-intelligence design, so we did not run this analysis. For policy change, there was no significant difference in the educational effect on the 7 achievement tests (2.760 IQ points, SE = 0.968, p =.004) versus the 23 other tests (1.784 points, SE = 0.553, p =.001; difference SE = 1.011, p =.334). However, for school-age cutoff, which had the largest proportion of achievement tests (38 of the 86 tests were classed as achievement tests), achievement tests showed a substantially and significantly larger educational effect (6.231 points, SE = 0.339, p = 2.85×10−75) than other tests (3.839 points, SE = 0.412, p = 1.11×10−20; difference SE = 0.371, p = 1.19×10−10)….”

Male-only studies.

Outcome test category.

“…Figure 1 shows the effect sizes in the first two study designs as a function of the participants’ mean age at the outcome test. For the control- prior-intelligence studies, outcome age was a significant moderator: The effect size of education declined by −0.026 IQ points per year of age (SE = 0.012, p =.029). At the youngest age (18 years) the effect size of having had an additional year of education was 2.154 IQ points, whereas at the oldest age (83 years) the effect size was 0.485 IQ points. This effect was smaller but still significant if the largest effect size (> 3 IQ points for an additional year) was excluded (−0.011 IQ points per year of age, SE = 0.005, p =.018). There was no significant moderating effect of age at outcome for the policy-change studies (0.014 IQ points per year, SE = 0.022, p =.543): The effect at the youngest age (18 years; 1.690 IQ points) was not significantly different from the effect at the oldest age (71 years; 2.413 IQ points). There was comparatively little variation in the age at the outcome test for the school-age- cutoff design (SD = 1.6 years; see Table 1); there was no significant age moderation effect (−0.027 points per year, SE = 0.399, p =.947)….”

“…. For the policy-change design, we tested whether the age at which the educational policy change produced an increment in compulsory schooling moderated the intervention effect. This was not the case: The effect size increased by a nonsignificant 0.038 IQ points per year of age at the intervention (SE = 0.228, p =.867). The meta-regression model implied that at the youngest intervention age (7.5 years), the effect size was 1.765 IQ points, and at the oldest (19 years) it was 2.204 IQ points….”

Age at outcome test.

“…For the control-prior-intelligence design, we tested whether the age at which the participants had taken the initial (control) cognitive test, or the gap between this early test and the outcome test, moderated the effect size. The age at the early test, which did not vary substantially (see Table 1) was not significantly related to the effect size (−0.024 IQ points per year, SE = 0.064, p =.706). For the youngest early-test age (10 years), the meta-regression model indicated that the effect size of 1 additional year of education was 1.243 IQ points; for the oldest (16 years), the effect size was 1.099 IQ points. Conversely, the time lag between the tests was a significant moderator of the effect size (−0.031 IQ points per year, SE = 0.015, p =.033). This meta-regression indicated that at the smallest age gap (5 years), the effect size was 2.398 IQ points, whereas for the largest age gap (72.44 years), the effect size was a substantially smaller 0.317 IQ points. Note that this age gap is almost fully confounded with the age at the out- come test (r =.988), assessed as a moderator below….”

Age at intervention.

Age at early test and time lag between tests.

“…We analyzed results from the three most prominent quasi-experimental methods for testing the effects of education on intelligence. We defined intelligence as the score on a cognitive test; see below for consideration of how the test scores might relate to the underlying psychological processes. Each method implements a different approach to minimize effects stemming from selection processes. Full meta-analytic inclusion criteria are reported below, but first we describe each of the three designs, providing a canonical example of each..…. The first research design, which we label control prior intelligence, is a longitudinal study in which cognitive testing data are collected before and after variation in the duration of education. This allows the relation between education and the second test to be adjusted for by each participant’s earlier ability level. An example of this design is the study by Clouston et al. (2012), who analyzed data from three large U.S. and UK cohort studies, all of which had both an adolescent and a midlife cognitive test. Results indicated that completing a university education was linked to higher midlife cognitive ability, above and beyond adolescent intelligence….…The second design, policy change, relies on changes in educational duration that are, by all accounts, exogenous to the characteristics of the individuals. An example of this design is the study by Brinch and Galloway (2012), who used large-scale data from a 1960s educational reform in Norway. This reform increased compulsory education by 2 years; critically, it was staggered across municipalities in the country. This allowed the researchers to estimate the effect of an additional year of school on a later intelligence test, taken by males at entry to military service as part of Norway’s universal military draft. Under the assumption that the policy change affected intelligence only via increasing years of schooling, the authors used an instrumental-variables analysis to estimate the effect of 1 year of schooling on intelligence at approximately 3.7 points on a standard IQ scale (M = 100, SD = 15 in the population)….…The third design takes advantage of a school-age cutoff. These studies use regression-discontinuity analysis to leverage the fact that school districts implement a date-of-birth cutoff for school entry. The first study to use this method was by Baltes and Reinert (1969), but the most highly cited example is by Cahan and Cohen (1989), who, in a sample of over 12,000 children across three grades of the Israeli school system (between the ages of approximately 10-12 years), found that schooling exerted positive effects on all of 12 tests covering a variety of cognitive domains. These educational effects were around twice the effect of a year of age. The strict assumptions of this method are some- times not fully met (Cliffordson, 2010); methodological issues are discussed in more detail below….”

Their Evidence

“…In three separate unconditional random-effects meta- analytic models (one for each study design), we estimated the effect of 1 additional year of education on cognitive outcomes. For all three study designs, there was a significant effect of 1 additional year of education. For control prior intelligence, the effect was 1.197 IQ points (SE = 0.203, p = 3.84×10−09); for policy change, it was 2.056 IQ points (SE = 0.583, p = 4.23×10−04); and for school-age cutoff, it was 5.229 IQ points (SE = 0.530, p = 6.33×10−23). An overall model including all estimates from all three designs found an average effect size of 3.394 IQ points for 1 year of education (SE = 0.503, p = 1.55×10−11)….”

Types of Studies They Looked At

Core of paper, “…Music training has repeatedly been claimed to positively impact children’s cognitive skills and academic achievement (literacy and mathematics). This claim relies on the assumption that engaging in intellectually demanding activities fosters particular domain- general cognitive skills, or even general intelligence. The present meta-analytic review (N = 6,984, k = 254, m = 54) shows that this belief is incorrect. Once the quality of study design is controlled for, the overall effect of music training programs is null (𝑔̅≈ 0) and highly consistent across studies (τ2 ≈ 0). Results of Bayesian analyses employing distributional assumptions (informative priors) derived from previous research in cognitive training corroborate these conclusions. Small statistically significant overall effects are obtained only in those studies implementing no random allocation of participants and employing non-active controls (𝑔̅≈ 0.200, p <.001). Interestingly, music training is ineffective regardless of the type of outcome measure (e.g., verbal, non-verbal, speed-related, etc.), participants’ age, and duration of training. Furthermore, we note that, beyond meta-analysis of experimental studies, a considerable amount of cross-sectional evidence indicates that engagement in music has no impact on people’s non-music cognitive skills or academic achievement. We conclude that researchers’ optimism about the benefits of music training is empirically unjustified and stems from misinterpretation of the empirical data and, possibly, confirmation bias…”

Supporting Evidence

“…This meta-analysis - which includes 38 studies, 118 effect sizes, and 3,085 participants - found an overall effect of𝑑̅= 0.16. It also highlighted that the impact of music training on cognitive skills and academic performance was a function of the quality of the study’s experimental design. Specifically, the magnitude of the music-induced effects was significantly smaller (around zero) in those studies implementing active controls and random allocation of the participants to groups….”

“…The overall effect size of the RVE intercept model was𝑔̅= 0.184, SE = 0.041, 95% CI, m = 54, k = 254, df = 38.36, p <.001, τ2 = 0.041, I2 = 43.16%. Different values of within-study effect-size correlation (ρ) did not significantly affect the results (𝑔̅range 0.184- 0.185, τ2 = 0.041). The random-effect (RE) model (with Cheung & Chan’s correction) yielded very similar estimates:𝑔̅= 0.176, SE = 0.037, p <.001, τ2 = 0.033. Overall, the results showed a small and moderately heterogeneous overall effect of music training on cognitive and academic outcomes. The results were not affected by modeling choices (i.e., ρ values and procedure for modeling nested data)…. Baseline difference and Type of controls were the only two statistically significant moderators (p =.031 and p =.035, respectively) and accounted for part of the true heterogeneity (τ2 = 0.038, I2 = 34.87%). Age was not significant (p =.403), neither was Allocation (p =.518). No significant differences were found across the four broad groups of outcome measures (all ps ≥.624; Holm’s correction for multiple comparisons), nor across the more fine-grained categorization (eight levels, all ps ≥.362), and there was no difference between cognitive skills and measures of academic achievement (p =.981). Duration of training was not significant either (p =.266, p =.952, and p =.662 for hours, weeks, and sessions, respectively).

Giovanni Sala and Fernand Gobet, "Cognitive and academic benefits of music training with children: A multilevel meta-analysis,"Memory & Cognition, July 29, 2020, https://link.springer.com/article/10.3758/s13421-020-01060-2