Macro Roundup

"...From the evaluation, we gained insights about why the initiative didn’t make an impact, which can be used to help future initiatives succeed—if people are willing to pay attention to the lessons learned...."

Frederick Hess, "Straight Up Conversation: RAND Education Chief Darleen Opfer,"Education Next, January 18, 2019, https://www.educationnext.org/straight-conversation-rand-education-chief-darleen-opfer/

Straight Up Conversation: RAND Education Chief Darleen Opfer

Darleen Opfer serves as director of RAND Education and Labor, a division of the nonprofit, nonpartisan RAND Corporation. She leads a 200-person staff that conducts research for major government agencies and private foundations. Along with her extensive work in the U.S., Darleen has researched teacher professional development in countries such as England and Turkey, and served as an education advisor in Norway, Israel, India and South Africa. Before RAND, she was director of research and a senior lecturer at the University of Cambridge. I recently had the chance to chat with Darleen about the state of education research, and here’s what she had to say.

Rick Hess: Darleen, readers probably have a vague notion that RAND Education does research and analysis, but can you explain more concretely what it is you do?

Darleen Opfer: RAND was founded 70 years ago to provide objective analysis on defense issues for the U.S. government. In the late 1960s, we expanded our work to social and economic policy areas such as education and health. For several decades, RAND Education has conducted research on schools and other educational institutions from pre-K through postsecondary, but recently we combined the education and labor research groups. We’re excited about this new structure because it allows us to connect our education research with our significant body of research on workforce skills needed for individuals to be successful. I think what people would find surprising about RAND is the breadth of research that is conducted. Our education and labor research is quite tame compared to work my colleagues are doing to track guns on the dark web, fight the opioid crisis, and clean up space junk.

RH: How did you wind up at RAND, anyway?

DO: RAND found me. I was at the University of Cambridge, which was amazing, but I became increasingly frustrated with research that didn’t necessarily make a difference in the lives of kids and teachers. RAND offered me the opportunity to be part of an organization whose research makes a difference.

RH: What are a couple of the most interesting projects currently on your docket?

DO: We just released our final report about best practices for school districts to implement summer learning programs. This report is a product of a six-year study sponsored by the Wallace Foundation. Through this randomized controlled trial, we assembled the largest dataset out there about summer learning—over four summers, we conducted thousands of hours of observations, surveyed thousands of summer teachers, and did hundreds of interviews. We will be presenting very specific recommendations about how to use time, how to train and recruit teachers, and when to plan for summer programs. Another Research area, which we call the RE: Work initiative, focuses on the future of work and the role of education in powering economic prosperity for individuals. This effort arose out of our decision to bring together education and labor researchers to talk about why the education system and the labor market seem to be leaving so many people behind. Ultimately, we want to help create more pathways to the middle class.

RH: This past summer, RAND issued the final results of a seven-year evaluation examining the Gates Foundation’s massive Effective Teaching initiative. Most observers regarded the results as quite disappointing for a once-heralded effort. Can you talk a bit about that project and the takeaways?

DO: It was unfortunate that the coverage of that report was so negative in the education press. The Gates Foundation deserves credit both for sticking with their intervention and the evaluation. A major problem I see in education research is that people jump from one thing to the next without giving interventions enough time to even have an impact. From the evaluation, we gained insights about why the initiative didn’t make an impact, which can be used to help future initiatives succeed—if people are willing to pay attention to the lessons learned.

RH: And, to your mind, what are a couple of the key lessons that people should learn?

DO: We learned that teachers and principals valued some aspects of the evaluation system, particularly the classroom observation rubrics which not only provided teachers with more detailed feedback than before, but also helped to create a common language around what good teaching looks like. At the same time, once the evaluation measures became high stakes, teachers’ support for them declined, and principals seemed less willing to provide negative feedback—a finding that has been reinforced in other research. We also found that implementing these types of systems places a heavy time burden on principals, which suggests that districts or Charter Management Organizations might want to explore alternative approaches that reduce demands on principals’ time by shortening observations or having someone other than the principal conduct some of them. A final key takeaway is that the significant turnover in the leadership of the districts and CMOs proved to be a barrier to consistent messaging and coherent implementation.

RH: How uncomfortable is it to issue a report that says a client’s big project didn’t pan out as hoped?

DO: It can depend on the client. We never want to take anyone by surprise, so we would make sure the client is briefed before the report is published. We typically give the client an opportunity to review all products and share any concerns, though we ultimately decide whether and how to address them. In the case of the Effective Teaching report, we provided interim reports and communicated informally with the Bill and Melinda Gates Foundation on a regular basis. The Foundation knew what was happening and still stuck with it. I think they understood that it is as important for the field to understand why something didn’t work as it is to understand why something did. If you have a client who cares more about the field than they do about the hit they may take in the press, then putting out a report such as we did for the Effective Teaching project isn’t that difficult.

RH: What kind of safeguards do you all use to reassure outside observers that RAND researchers aren’t catering to clients or pulling their punches to keep clients happy?

DO: RAND’s mission is to produce research that makes a difference and is in the public’s interest. We reserve the right to publish independent research and regularly walk away from doing work when a potential client disagrees with this. Every report is also peer reviewed both internally and externally to make sure what we are doing meets the highest standards of quality. The insistence on both publication rights and the extensive nature of our peer review process is unique in the contract research world but central to RAND as an organization.

RH: Speaking of contract research, the Wallace Foundation recently commissioned a RAND report on Social and Emotional Learning interventions. Given all the hype around SEL, I’m curious: What did you all find—and how much confidence should parents and educators have in these interventions?

DO: As you know, there’s been a proliferation of new instructional programs and materials to promote SEL. Wallace asked RAND to generate guidance for education leaders who want to use their Every Student Succeed funds to pay for SEL programs. We found that there are numerous programs that are evidence-based when applying the ESSA definitions, but several factors—including the uneven rigor of the research designs, the lack of high-quality outcome measures, and the importance of contextual conditions in influencing how programs are implemented—limit our ability to provide definitive advice on what will work in a given school or district. So to answer your questions, the jury is still out on effectiveness of SEL programs. We recently released a follow-up report that provides practical tips on how to conduct an evidence-based SEL needs assessment tailored to local contexts.

RH: When it comes to professional development, we all know how much faith people tend to put in “better PD,” and how skeptical many teachers and researchers are about the value of all this. You’ve studied PD across the U.S., England, and Turkey. What can you share on this score?

DO: PD for teachers is often of low quality, but it doesn’t need to be. PD is a significant investment for districts and states, yet information after the fact about whether teachers implemented what they were taught or whether student learning improved is not collected. Some of our newer work is examining more customized, job-embedded approaches to PD, which are more likely to influence teachers’ practices and improve student learning. But I don’t think we’ll see a sea change in the quality of PD until policymakers decide that PD funding cannot be used for a program that lacks sufficient evidence of effectiveness. Until that happens, many PD providers will not be incentivized to improve what they offer.

RH: So, what does customized, job-embedded PD look like? Is there an example that comes to mind?

DO: This is PD that is tailored to a teacher’s role and context rather than relying on a one-size-fits-all format such as a large group workshop. High-quality PD that is job-embedded provides teachers with opportunities to try out what they’re learning and to get feedback on their practices in real time. This can occur through individual coaching and mentoring, for example. Professional learning communities in which teachers collaborate with their colleagues to plan lessons or review student work can also be a form of job-embedded PD because teachers are learning from one another. We’ve seen these models in many schools we’ve studied, though often conditions, such as limited time or funding, hinder the implementation of these models.

RH: Another intriguing project you all have tackled is a multi-nation examination of instruction in middle school math. What did you learn?

DO: Our middle school math study, the TALIS Video Study for the OECD, looks at the relationship between teaching practices and students’ achievement and social and emotional outcomes in mathematics. A study like this has never been done before because it’s difficult to develop cross-cultural instruments that can measure teaching, and the fielding requirements are complex. An interesting thing we’ve learned already is how much variation in mathematics content exists. Eckhard Klieme, at the Leibniz Institute for Research and Information in Education in Germany, worked with the 8 participating countries’ mathematics experts to map curriculum topics taught to students between 13-15 years old. That mapping exercise resulted in only two common topics—linear equations and quadratic equations. We selected quadratic equations for the focus of the study, but we’re finding out that what counts as quadratic equation content also varies across the countries. This variation is important to understand because too often we expect students to demonstrate understanding when they haven’t even been given the opportunity to learn.

RH: From your perch at RAND, you’ve got a bird’s eye view of so much of the education research happening in the U.S. and around the world. What are a couple things that have caught your eye that may be news to those of us more enmeshed in our day-to-day?

DO: With the growth in technology applications in schools, combined with advancements in data processing and analysis capabilities, we need to think more about what the next generation of ed-tech reforms will look like and how educators can create the conditions to maximize their benefits and minimize harms. But we also need to do so in a way that doesn’t further harm the teaching profession. There has been scope creep, and the demands on teachers to be academic instructors, social and emotional instructors, first responders, and so on can be overwhelming. Advances in personalization of education, while exciting, make lesson planning and progress monitoring even more challenging. As education delivery and expectations change, we need to develop a sustainable and fulfilling teacher career trajectory and be able to forecast needs and demands of the future teacher labor market.

RH: Final question: What’s an especially promising development you see when it comes to education research? And, what’s the one development in education research that you find most disconcerting?

DO: The use of randomized control trials and other rigorous research designs has grown tremendously, resulting in valuable evidence on education programs and practices. At the same time, we are starting to see the limits of these approaches; there has been a replication crisis not just in education but in several fields, where we’ve seen that positive effects often don’t hold up when an intervention is studied by a different research team or in a different context. This points to the need to better understand what’s driving varying results. Because of that, I think implementation is finally getting its due in research. We all know how critical it is to implement a program long enough and well enough to see results, and how often that doesn’t happen in schools. So I’m happy to see increased focus on how educators implement reforms.

But something I find problematic is that even with the increase in quality, we are not seeing a comparable increase in use of research. One of my priorities is to ensure that RAND’s work is conducted and disseminated in a way that will improve decisionmaking—this is RAND’s core mission and is the reason researchers such as myself work here. In order to see more research being used, we need to be clear on the utility of our research. Here at RAND, we’re working on how to improve our ability to tell educators what they can/should do based on the research evidence we produce. The onus is on the research community to help educators use evidence-based practices; if we do that, we can move the needle significantly when it comes to bettering education systems worldwide.

— Frederick Hess

Frederick Hess is director of education policy studies at AEI and an executive editor at Education Next.

“…The share of applying workers whose disability claims were allowed following a medical review in 2016 fell to 48.0%, according to the latest available data from Social Security. That was the first time the rate was below 50% on records dating to 1992. Allowances peaked at 62% in 2001. “A smaller share of people are getting on the program, and that is causing people to not apply and instead to seek alternatives,” said David Autor, an economist at the Massachusetts Institute of Technology. His research was among the first to document the causes and consequences of rising disability claims. Mr. Autor said the recent decrease in disability applications and awards is similar to a crackdown in the early 1980s, which Congress later reversed, setting off a long march higher in claimants….”

Eric Morath, "America’s Hidden Workforce Returns,"Wall Street Journal, January 26, 2019, https://www.wsj.com/articles/americas-hidden-workforce-returns-11548478801

America’s Hidden Workforce Returns

WYOMING, Mich.—Like many adults with disabilities, Nathan Mort has often struggled to find and hold a job. A conservation group once declined his attempt to volunteer. The 37-year-old West Michigan native, who has a high-functioning form of autism, ended up living with his parents and dependent on government payments.

His fortunes turned several years ago when a local food distributor, Gordon Food Service, found itself short of entry-level workers and developed an internship program for adults with disabilities. Mr. Mort was hired from the program permanently to track warranty claims for the company’s trucks and other equipment. That allowed him to stop collecting federal disability benefits and move into his own home.

“A reason why I like my job is because it’s kind of the same thing over and over again. I really like routines. That is part of my autism,” Mr. Mort said. Making his own money, he added, “makes you feel more worth something.”

Mr. Mort is part of a wave of disabled Americans joining or returning to the U.S. labor force, breaking a long-running trend that had pushed millions to the sidelines of work. These workers have benefited from a tight economy with a very low overall unemployment rate—3.9% in December, just above lowest level since 1969—as employers in many sectors tackle a shortage of available workers by becoming more creative about whom they recruit.

“This is really quite a dramatic turnaround,” said Nicole Maestas, an economist at Harvard Medical School who studies the disability system. “Employment among people with disabilities had been declining for a really long time—now it’s not. And it’s not transitory. It’s been increasing for a few years.”

Two factors are at play. First, the tight labor market is causing employers like Gordon Food Service to expand their search for job candidates. Second, Social Security disability benefits—which for many became a de facto second form of unemployment insurance after the last recession ended in 2009—have become harder to access.

When the disability-benefits system found itself on shaky financial footing a few years ago as its rolls swelled, the Social Security Administration pushed new training guidelines to administrative judges who decide which claims will be granted, with the aim of producing more consistent rulings. Before those efforts, judges in some states approved claims at unusually high rates. Members of Congress from both parties in 2011 called for an investigation into the way administrative judges determine whether an applicant qualifies for disability benefits and federal prosecutors began probing the practices of specific judges.

“A smaller share of people are getting on the program, and that is causing people to not apply and instead to seek alternatives,” said David Autor, an economist at the Massachusetts Institute of Technology. His research was among the first to document the causes and consequences of rising disability claims.

Mr. Autor said the recent decrease in disability applications and awards is similar to a crackdown in the early 1980s, which Congress later reversed, setting off a long march higher in claimants.

Before Gordon Food Service hired him in 2014, Mr. Mort spent a decade without regular work, despite earning a degree from Grand Valley State University near his home. He was let go from a part-time mail-clerk job in college because pain in his hands, a side effect of his condition, made him slow sorting envelopes. He quit another job packing croutons at a food-processing plant after a day. The pace was too fast. He was laid off from another job sorting and hanging clothes at a Salvation Army resale shop. The job presented him with too many distractions. “I probably did not work fast enough,” Mr. Mort said.

The program that brought Mr. Mort to Gordon Food Service came from the company recognizing it may be missing a potential pool of candidates, said Jill Day, senior talent acquisition leader. A suggestion to work with Hope Network, a Grand Rapids, Mich., nonprofit that serves the disabled, came from staff members who had disabled family members.

“For many of our entry-level positions, there’s not enough people going into those roles,” she said. “I think that creates a great opportunity for the company to step back and ask ‘What can we be doing differently?’ There’s an enormous number of people with disabilities who are hungry to work and want to contribute.”

Gordon Food Service was able to tap into what some autism experts call a “splinter skill,” something an autistic person does exceptionally well. For Mr. Mort it’s the ability to process complex strings of numbers combined with being a fast typist—his hands don’t bother him on a keyboard.

As vehicle-warranty administrator, Mr. Mort spends his time copying serial numbers, submitting warranty repair orders, tracking claims, and managing databases. After his internship he was hired for a part-time job. He gradually increased his hours and in November was given a full-time role, taking home more than $400 a week plus health benefits.

“He has incredible focus and never seems to get tired of processing claims,” said Jason Derby, Mr. Mort’s supervisor and Gordon Food Service fleet-maintenance manager.

Mr. Mort uses his earnings to pay for maintenance and property taxes on a two-bedroom condo his parents purchased for him last year. He also enjoys reading fantasy books by J.R.R. Tolkien and Brandon Sanderson, playing “Warcraft” videogames, visiting area zoos and attending Civil War re-enactments.

Mr. Mort’s father, Bruce, now worries less about what will happen once he and his wife are no longer able to care for their son. “It was scary for a while,” he said. “During the recession there weren’t many companies willing to give someone with autism a chance.”

Most people exit the Social Security disability system because they either qualify for retirement benefits or die. Disability pays roughly the equivalent of working full time at the federal minimum wage, $7.25 an hour. Unlike many low-wage jobs, it comes with health insurance, Medicaid, and is adjusted for the cost of living.

But people like Mr. Mort are no longer a rarity. In 2017, according to the latest data available, 51,302 people left disability because they found “gainful” employment. That is the most on records dating to 2002 and nearly 30% higher than in 2011.

Last year, Social Security Chief Actuary Stephen Goss told members of Congress that the projected depletion of the disability trust fund was extended by four years to 2032, a shift “based primarily on continuing lower-than-expected disability application and incidence rates.”

The tighter labor market is delivering opportunities to a broad swath of workers who were disproportionately affected by the last recession. Unemployment has fallen sharply for blacks, Latinos, younger workers and those without a college education.

But the deeper jobs pool has been particularly crucial for improving the economic outlook of the disabled Americans who have been denied benefits at a higher rate in recent years. Take Keri Dougherty, of Paris, Tenn., a 23-year-old with autism who qualified for Social Security payments as a child but was denied when she applied for disability benefits as an adult. Her family didn’t appeal the decision made in 2013. “We didn’t have the money to fight it,” said Laura Dougherty, Keri Dougherty’s aunt, who raised her since she was 5. “But that’s OK, we had bigger aspirations for Keri anyhow.”

Keri Dougherty, whose disability prevents her from driving, completed high school and two years ago received employment training at the Tennessee Rehabilitation Center in Smyrna, Tenn. Days after finishing her training at the center, she attended a job fair and was soon hired to work at a trailer that collects donated items for the Goodwill Industries charity.

The job allowed her to move into her own apartment in December and to buy an old car, even though she’s unable to drive it. She leaves it parked near the unheated trailer so she can warm up during breaks. She earns about $9 an hour and works 40 hours a week. “I’ve wanted to live independently for a long time,” Ms. Dougherty said. “I wanted to experience how the real world works.”

Patty Kasbohm, director of operations at Hope Network, who helped place Mr. Mort in his job, said the tougher rules around disability programs are causing hardship for some, including less access to job-training programs that come with the benefits. She said people with some forms of autism are particularly challenged because they have high IQs, which can disqualify them for benefits, but their social skills often prevent them from finding work on their own.

Another factor removing pressure on the Social Security disability program is demographics. When the recession hit in 2007, the baby boomers were in their 50s and early 60s. Older workers are more likely to have impediments, including vision challenges and pain, that may qualify them for disability. Now baby boomers are increasingly eligible for Social Security retirement, shifting the payment burden to that program.

A onetime construction worker, Elton Crouse went on disability in 2002 after he fell 16½-feet down an elevator shaft in Michigan. “The fall was a breeze,” he said. “It was the sudden stop that got me.” The injury has left his neck too weak for physical jobs.

After five years on disability benefits, Mr. Crouse was told by a counselor assisting in his rehabilitation that because he had improved he needed to pursue part-time work or risk losing payments.

Mr. Crouse, now 59, for years bounced between jobs but in 2017 he landed what he considers the perfect role: greeting customers at Duluth Trading Co. clothing store, among other tasks. The company has been accommodative of his disability, reassigning him to mop floors before store openings when short-term memory loss associated with his accident made operating the cash register too difficult.

“They’re really willing to jump through hoops for me,” he said.

Mr. Crouse hopes to work for the store until he can tap old-age Social Security benefits in a few years. By working part-time, but remaining eligible for benefits under a Social Security program known as Ticket to Work, Mr. Crouse earns between $800 and $1,000 more a month than he would otherwise, which helps offset rising prescription drug costs.

In some cases, Americans with disabilities are creating their own jobs. People with disabilities are more likely to be self-employed than the nondisabled, according to the Labor Department.

Mark Burke, who is deaf, found it hard to find new employment in 2014, after he was let go from his job as athletic director at Model Secondary School for the Deaf in Washington, D.C. One constraint was that he didn’t want to move his family, including his in-laws and young daughter, from their suburban Maryland home.

“Emotionally, it was very tough,” he said, speaking through a signing interpreter. “What can I do? Where can I go?” He said he applied for about 300 federal government jobs. He received two callbacks and no job offers.

After a year out of work—when he received Social Security disability benefits—Mr. Burke, now 48, took a job tending bar. The bar that hired the devoted home brewer is located near Gallaudet University, a federally chartered school for the deaf in Washington. Mr. Burke’s ability to sign and deep knowledge of craft beer helped land him the job, where he also waited on hearing customers.

That led Mr. Burke to think he could open a microbrewery. He pitched his idea at a Gallaudet entrepreneurship competition in the style of the television show “Shark Tank.” He didn’t win, but the experience gave him and his co-owners, friends and fellow Gallaudet alumni, the confidence to open Streetcar 82 Brewing Co. in Hyattsville, Md., in September.

Ryan Maliszewski, director of the Gallaudet Innovation & Entrepreneurship Institute, said technology, including lower-cost video interpreting, has opened doors for deaf people to start businesses and take jobs ranging from app developers to accountants, where they can work remotely.

“I’d always dreamed of owning my own business, but when I graduated from college I didn’t think the opportunity existed,” Mr. Burke said, sitting on a barstool across the room from a silver fermenter, where he recently finished brewing a coffee stout. He added that in the 1990s, deaf people were pushed toward jobs in the government or large corporations that were more able to afford accommodations, such as interpreters.

Now Mr. Burke said he regularly uses email, texting and social media to communicate with vendors and other business associates. “A lot of them don’t realize that I’m deaf until we meet face-to-face,” he said. “Back in the day I don’t think people would want to deal with that—but now I think it’s OK.”

James Freeman, "U.S. Income More Equal than Advertised,"Wall Street Journal, December 12, 2018, https://www.wsj.com/articles/u-s-income-more-equal-than-advertised-11544574351

U.S. Income More Equal than Advertised

Remember the 2014 bestseller “Capital in the Twenty-First Century” by French economist Thomas Piketty? Beachgoers in the Hamptons couldn’t be seen without it tucked under an arm if they wanted to be regarded as serious people concerned about the plight of the less fortunate. The dismal tale of exploding inequality and capitalist failure has been a recurring theme in political chatter ever since. But a new report highlights just how poorly Mr. Piketty’s thesis has held up under further study.

This column should note that some scholars saw problems right from the start. In a 2014 Journal op-ed, Harvard economist Martin Feldstein ticked off a series of fundamental errors, including those related to Mr. Piketty’s practice of comparing the incomes of top earners with total national income. “National income excludes the value of government transfer payments including Social Security, health benefits and food stamps that are a large and growing part of the personal incomes of low- and middle-income households,” wrote the Harvard prof.

Then Chris Giles of the UK’s Financial Times examined Mr. Piketty’s work and found that “the rock-star French economist appears to have got his sums wrong.” He added:

The data underpinning Professor Piketty’s 577-page tome, which has dominated best-seller lists in recent weeks, contain a series of errors that skew his findings... The central theme of Prof Piketty’s work is that wealth inequalities are heading back up to levels last seen before the first world war. The investigation undercuts this claim, indicating there is little evidence in Prof Piketty’s original sources to bear out the thesis that an increasing share of total wealth is held by the richest few.
Now a new report from the Urban Institute goes back further, to Mr. Piketty’s influential research with University of California, Berkeley professor Emmanuel Saez published in 2003. Stephen Rose writes:

Piketty and Saez’s findings garnered tremendous attention and were cited repeatedly. But many researchers eventually found problems with Piketty and Saez’s approach and developed income inequality measures that led to different findings.
Phil Magness of the American Institute for Economic Research writes today:

In short, the widely reported explosion of inequality in the past three decades is likely a myth, built upon outdated and flawed statistics... Whereas Piketty and Saez show a massive century-long swing of almost 20 percentage points in the income share of the top 10 percent, the adjusted figures show a much flatter curve with a little over half the variation. Inequality still falls and rises under the revised numbers, but at a comparatively subdued rate. Under the adjustments, the top 10 percent income share seldom strays more than 5 percentage points away from a century-long average of about 35 percent.
Perhaps instead of assuming that we’re in the midst of a surging inequality crisis, we should first settle more fundamental issues affecting the accuracy of our measurements over the past century.

Perhaps we should also focus on ensuring an abundance of opportunity, rather than regarding it as a problem when some people inevitably make more than others.