You're Hired" Mulligan Review

John Cochrane The Grumpy Economist

Date Posted:: October 5, 2020
Is Database:: Database

Medicare Part D expansion inadvertently fueled opioid epidemic by increasing access to prescription drugs, leading to a 345% rise in opioid-related deaths from 1999 to 2016.

Casey Mulligan's analysis highlights how the expansion of Medicare Part D inadvertently contributed to the opioid epidemic by increasing access to prescription drugs, which led to a rise in opioid prescriptions. The economic implications are significant, as the increased availability of opioids through Medicare Part D correlated with a surge in opioid-related deaths, which rose by 345% from 1999 to 2016. This expansion, while intended to improve healthcare access, inadvertently fueled a public health crisis, demonstrating the complex interplay between healthcare policy and economic outcomes. The data underscores the need for policymakers to consider unintended consequences when designing healthcare programs, as the economic burden of the opioid crisis has been estimated to exceed $500bn annually, impacting GDP and labor force participation.

John Cochrane, ""You're Hired" Mulligan Review,"The Grumpy Economist, September 26, 2020, https://johnhcochrane.blogspot.com/2020/09/youre-hired-mulligan-review.html

Cochrane reviews Casey Mulligan’s book, “..the book is really not that much about Trump. It is much more about how the CEA works, how policy is formed in the Trump administration, which is much more like other administrations than you'd think, and a record of some great successes of the Trump-era CEA. (Kevin Hassett, the CEA chair, really deserves more pride of place in this story.) Trump shows up to make the big decisions, but usually on issues the CEA has spearheaded…. On Trump, I think Casey is a bit too soft….However, on documenting the enormous uphill climb this administration faces against an implacably hostile media and federal workforce, this book is very useful. To understand how strong the bubble is, how it fails, and why populism might make some sense and outlive Trump, read on….”

Ed Comment: The "Samaritan's curse" that aid always leads to more of the misfortune"... Medicare part D.. reduced the annual cost of a... 0.75-gram daily habit from $39,420 to $2,677.Early Oxytocin's dealers... were seniors... 'it's like hitting the Lotto if your doctor will put you on OyxContin... people didn't even twice about selling.'...for a three-dollar Medicaid co-pay.. an addict got pills priced at a thousand dollars...In February 2018, CEA informed Federal agencies about the role of opioid prices. At least two agencies refused to allow the findings to be released to the public or be shared with President Trump. Secretary Azar refuses to even to consider that introduction of Medicare Part D might have helped fuel the opioid epidemic."

Are results in top journals to be trusted?

Economist Staff The Economist

Date Posted:: January 25, 2016
Is Database:: Database

A recent study found 10-20% of results in top American journals may be questionable, raising concerns about the reliability of economic research.

A recent study published in the American Economic Journal analyzed 50,000 papers from 2005-2011 in top American journals, revealing that 10-20% of results deemed statistically significant may be questionable. The study identified a peculiar double-humped distribution of z-scores, indicating "missing" results just outside the standard significance cut-off. This suggests potential bias in reporting, as some results may have been manipulated to achieve significance. Such findings raise concerns about the reliability of economic research, emphasizing the need for rigorous scrutiny and transparency in data reporting to ensure the integrity of published results. This has significant implications for economists, policymakers, and business professionals who rely on these studies for data-driven decision-making.

Economist Staff, "Are results in top journals to be trusted?"The Economist, January 21m 2016. Available at:http://www.economist.com/blogs/freeexchange/2016/01/fudging-hell

"....But apaperjust published in the American Economic Journal finds evidence of a different sort of bias, closer to the source. Called "Star Wars, the empirics strike back", it analyses 50,000 papers published between 2005 and 2011 in three top American journals. It finds that the distribution of results (as measured by z-score, a measure of how far away a result is from the expected mean) has a funny double-humped shape (see chart). The dip between the humps represents "missing" results, which just happen to be in a range just outside the standard cut-off point for statistical significance (where significance is normally denoted with stars, though the name may also be something to do with a film recently released—file under 'economists trying to be funny').Their results suggest that among the results that are only just significant, 10-20% have been fudged...."

Ed Comment:A study of 3 of the most prestigious economic journals—American Economic Review, the Quarterly Journal of Economics, and the Journal of Political Economy—suggests 10 to 20% of the results may be fudged. The results suggest researches find ways to push near-significant results over the statistical threshold necessary for publication. Rather than finding smoothly distributed study results across the range of statistical significant the find an odd 2-hump curve where the frequency of results just beyond the threshold.…Is this too hard to understand? A study estimates 20% of economic studies are fudged. The selection of studies by academic journals should logically increase as the statistical significance of the studies’ results increase. This should lead to a logically shaped distribution of studies in journals as a function of their statistical significance. But rather than find such a curve, the study finds, “the distribution of test statistics published in three of the most prestigious economic journals over the period 2005-2011 exhibits a sizable under-representation of marginally insignificant statistics relatively to significant statistics but also to (very) insignificant ones. In a nutshell, once tests are normalized…the distribution has a two-humped camel shape … cannot be explained by selection alone. …10% to 20% percent of tests with are misallocated: there are missing test statistics just before the threshold that…can retrieve after the” The study notes, “The two-humped shape is an empirical regularity that can be observed consistently across journals, years and fields. … Similarly, the two-humped camel shape is less visible in articles with theoretical models, articles using data from randomized control trials or laboratory experiments and papers published by tenured and older researchers. More generally, we find a larger residual in cases in which we would expect higher incentives for researchers to respond to selection,” i.e., “ …evidence that academic economists respond to publication incentives.” The researchers concludes, “Our analysis suggests that the pattern of this misallocation is consistent with what we dubbed an inflation bias: researchers might be tempted to inflate the value of those almost-rejected tests by choosing a slightly more ‘significant’ specification. … Among the tests that are marginally significant, 10% to 20% are misreported. These figures are likely to be lower bounds of the true misallocation as we use conservative collecting and estimating processes.”

Various solutions have been proposed. One is to publish 'pre-analysis plans', where researchers say how they will do their analysis before they actually do it. Another is to encourage more replication. Anew NBER working paperby Marcel Fafchamps and Julien Labonne suggests another, related, method. The idea is that researchers send their data to a third party, who randomly splits the data sample in half. The researchers do their analysis based on the first dataset, finalise their method, and submit for publication. If and when the paper is accepted, the same analysis is carried out on the second sample, and the unadulterated results published. If the initial result only showed up because of manipulation, then the chances of the same result in the second sample are relatively low. To avoid the embarassment of a non-result, researchers should be stricter with themselves when it comes to tweaking their results. When sample sizes are small, this fix is difficult, as halving the sample saps power from tests. But in a world of big data, it could work. The bigger barrier might be getting career-conscious researchers to sign up.

The paper does look at the results split into subgroups, and there seem to be some factors that are associated with a less humpy distribution (which could suggest less fudging). Although the overall pattern holds across all three prestigious journals the paper considers (theAmerican Economic Review, theQuarterly Journal of Economicsand theJournal of Political Economy), papers by older researchers and ones describing randomised control trials have less marked humps—though they are still there. This is worrying for those trying to interpret and communicate the latest research, as it is impossible to tell if there has been foul play in any individual study. But more fundamentally it is worrying for the profession and policymakers making decisions based on economic evidence; fiddling and running multiple, slightly different tests on the same data rapidly sucks meaning from the reported size and accuracy of the final results.

One explanation is that if a result shows up as significant at the 5% significance level (the industry standard) then researchers crack open the champagne and move on to making economicsjokes. But if the result is tantalisingly close to a positive result then perhaps the researchers will fiddle a bit with their method...and celebrate their nice publisher-friendly result. Yanos Zylberbe, one of the paper's authors, explains that in economics it is difficult to conduct controlled experiments, which ultimately gives a lot of freedom to researchers to tweak their methods. Sometimes researchers are tweaking because they want to find the best way of estimating an effect, but sometimes it's in the search for a significant effect. The distinction might be hazy, even in their own minds.

This should skew the distribution of published results, towards more 'significant' findings. But apaperjust published in theAmerican Economic Journalfinds evidence of a different sort of bias, closer to the source. Called "Star Wars, the empirics strike back", it analyses 50,000 papers published between 2005 and 2011 in three top American journals. It finds that the distribution of results (as measured by z-score, a measure of how far away a result is from the expected mean) has a funny double-humped shape (see chart). The dip between the humps represents "missing" results, which just happen to be in a range just outside the standard cut-off point for statistical significance (where significance is normally denoted with stars, though the name may also be something to do with a film recently released—file under 'economists trying to be funny'). Their results suggest that among the results that are only just significant, 10-20% have been fudged.

PUBLICATION bias in academic journals is nothing new. Afindingof no correlation between sporting events and either violent crime or property crime may be analytically top class, but you couldn’t be blamed, frankly, for not giving a damn. But if journal editors are more interested in surprising or dramatic results, there is a danger that the final selection of published papers offer a distorted vision of reality.

Are results in top journals to be trusted?