The IQ gap between Harvard undergrads Harvard Law students

Brandon Adams Comment: “That is an interesting piece. I think it has a few problems though. 1. Based on the piece you just sent (about the Flynn Effect operating disproportionately on the middle and left of the distribution), this adjustment is incorrect. It should be noted however that the WAIS-R was published in 1981, and that the norms were collected from 1976 to 1980. Carson’s study was published in 2003, so presumably the test norms were 25 years old. James Flynn cites data showing that from WAIS-R norms (circa 1978) to WAIS-IV norms (circa 2006) the vocabulary and spatial construction subtest (used in the abbreviated WAIS-R) increased by 0.53 SD and 0.33 SD respectively. These gains would result in the composite score of the abbreviated WAIS-R becoming obsolete at a rate of 0.26 IQ points per year, meaning the Harvard students’ scores circa 2003 were 6.5 points too high. This reduces the mean IQ of the sample to 122.6 (U.S. norms).This adjustment would only be sensible if the distribution shifted but maintained constant shape. That is not what happened. 2. Test Sample: 86 Harvard undergraduates (33 men, 53 women). "It should also be noted that this was a psychology study, and thus a disproportionate number of psych students likely took part. Realistically, us psych majors are not as bright (on average) as harcore STEM majors. Add to this the fact that the abbreviated WAIS only had a ceiling of 150, likely preventing some participants from showing their full potential. Given these two facts it seems reasonable to round up the mean score to 130." I don't know which factor is more problematic, the unrepresentative sample, or the fact that the test caps at 150. 3. The effect of chance on scores would be highly dependent on the details. We would expect the chance factor to be high if every student in the country took the SAT exactly once and then reported this score on their college applications. But there are three sources of complexity that can be introduced: 1) Does the student take the test more than once?, 2) Does the college have a method for adjusting the scores for students who take the test more than once when making admission decisions?, and 3) In the reporting of SAT scores to US News and World Report (or whatever database is used), are they reporting only the best score of admitted students who took the test more than once (or, worse, the best score on each section)? Likely what's happening is that Complexity 3) is the big problem, and therefore the bias effect is real. It's likely the case that if students are taking the test multiple times, but then only their best scores are recorded in the database, then that would result in an even larger bias than a world in which every student took the test exactly once.As a side note, I believe that if a psychometrician were trying to assess the IQ of people that were known ex ante to be highly intelligent, they would use a test where the scores are centered around a relatively high IQ (a test with very hard questions throughout). Attached is an interesting paper from the behavioral finance literature. The authors add one field to a database of hedge fund returns; the field consists of the average SAT score at the manager's undergrad institution. They find that this variable has strong explanatory power for returns. A 200 point SAT difference would equate to.73% higher returns per year, in their sample."”

Ed Conard Comment: “There is likely selection bias, especially of the kind Brandon describes. But there is definitely bias of the kind described by the study. People who luck out and score highly tend to end up at schools that accept high scorers. Again, there is a more comprehensive study of this in the DB.I don’t have a lot of experience with the STEM majors at top schools. My impression of the top student who end up at places like Bain is that they don’t average anything close to 145. Probably 130ish. My experience is that 145 makes you the smartest guy in a room at places like Bain Cap with the smartest business oriented people. …and that 150+ is need to be competitive with the top college professors but not the smartest guy in the room. Summers is probably 155. Again, I would say that to succeed in business you need a lot more than smarts. That’s why student with IQ lower than competitive professor choose business and not academics and vice versa. And perhaps more importantly, software is more important/powerful than hardware. I believe people have horrible s/w."”

Mark Hill Comment: “I have a further issue practical issue: when you recruit Harvard students to take an IQ test via signup sheet and offer an hourly rate, do you expect a representative sample or some significant selection bias?"”

Brandon Adams Comment: “Harvard is a terrible setting to illustrate this general phenomena. This is the starting point of the article: "Jonathan Wai reported that Harvard undergrads had a mean SAT of 1490 which at the time equated to an IQ of 145." That is not a statistically sound approach to the problem. I believe he's saying that "an SAT score of 1490 equates to an IQ of 145," which is not the same thing as saying that "a population whose mean SAT is 1490 equates to a population mean IQ of 145." To be statistically sound, you would need to translate all of the students' individual SAT scores into individual IQ scores, and then take a mean. When doing so, you would need to answer the question, "Conditional on a student getting a 1600, what is their expected IQ?" 20-25% of the sample might fit this profile. A 1490 avg for the institution as a whole likely implies a mean IQ much higher than 145 (though that doesn't mean much bc 1490 is biased to the high side for reasons we have mentioned)."”

Ed Conard Comment: “My strong suspicion is that 1) high-scorers tend to take the test multiple times. That’s an important way to get a high score. (BTW, I took it once and got high the night before.) 2) Colleges don’t adjust. 2a) Often you don’t have to report that you took the test except the one you submit. And 2b) colleges largely accept students so that they can report high scores. The new test optional option allows schools to accept low-scorers without having to report their scores. That’s the case because the schools don’t care about admitting the smartest students. They care about reporting high scores. …and funding their endowments so they can raise their own pay. 3) see 2; the schools are only reporting the highest scores. Columbia had so many tricks for falsely trumping up their scores that they were called out and lost their ranking. The most prestigious colleges (e.g., Harvard Law) are now refusing to participate in the US News rankings because they want to admit low-scorers and not have their scores and post graduate performance monitored.”

Ed Conard Comment: “It’s valuable to have you critique the statistical methods. So I don’t want to discourage it. Bad analysis leads to bad conclusions. That said, there is information in all data and analysis. The author probably couldn’t get the individual scores. So he is doing want he can with the data he has to indicate what he is trying to learn. Notwithstanding your point about top-coding, my guess is that the mean of the sat scores converted to IQ is very close to the scores covered to IQ and then averaged. That seems small not large in this case. Top-coding should make the IQ of the pool as measured by their admission test higher than 145. 128 is a long way from 145 (plus). Surely that large difference means something significant. We should not pooh-pooh what we can learn from this just because the analysis is less than perfect. I agree that a large share of the difference might be the difference between those who took the second test - psych students who may be lower than the Harvard average. But my guess is that those students also had very high SAT score on average and lots of 1600s too. Although, the psych majors may skew towards the ones who discovered they were lucky scorers. Nevertheless, my experience with the top business-oriented grads from the top schools is that their IQs feel closer to 128 to me than 145. They are not lowly psych majors. Most were econ majors with top grade points who excelled in quantitative classes, albeit not classes that included the top math and science majors. Although many were science majors. (Mark, aren’t you a John Harvard scholar in biology/science? Baines are much higher than the average student at the top school in my experience. 145 stands out among smart people as a recognizable difference. Not many people stand out IQ-wise at places like Bain Capital where the average is probably higher than 128, but probably not 135 despite their high scores and GPA is quant-oriented majors. Most people who stand out are top college professors. They are often famous. Again, I think s/w makes a bigger difference than h/w. Worse, they seem inversely correlated at high levels.”

Though this point was interesting"...Circa 2013, Jonathan Wai reported that Harvard undergrads had a mean SAT of 1490 which at the time equated to an IQ of 145. Meanwhile Wai reported that Harvard Law students had a mean LSAT score of 173.5 which also equates to an IQ of 145. However by definition, elite students over perform on the very test used to recruit them, because one of the things they’re recruited for is “good luck” on the admission test. Thus it’s interesting to ask how Harvard students perform on a random test (not used in the selection process) As I’ve noted many times the best data on the subject was obtained by Harvard scholar Shelley H Carson and her colleagues who had an abbreviated version of the WAIS-R given to 86 “Harvard undergraduates (33 men, 53 women), with a mean age of 20.7 years (SD 3.3)… All were recruited from sign-up sheets posted on campus. Participants were paid an hourly rate…The mean IQ of the sample was 128.1 points (SD 10.3), with a range of 97 to 148 points.”..Even though Harvard undergrads and Harvard Law students both score IQ 145 on their respective admission tests, their actual IQs are likely 130 and 136 respectively. This is not to say that the WAIS is necessarily more accurate than the SAT or LSAT; rather it’s to say that the IQ of a group should never be measured by the very test that selected them, because by definition, they likely overperformed on that...."