Does AI boost consultants' output most for the lowest performers?
Core argument: In a pre-registered experiment at Boston Consulting Group covering 18 realistic consulting tasks, consultants using ChatGPT-4 completed 12.2% more tasks, worked 25.1% faster, and produced 40% higher quality results than those without AI.
For the last several months, I have been part of a team of social scientists working with Boston Consulting Group, turning their offices into the largest pre-registered experiment on the future of professional work in our AI-haunted age. For 18 different tasks selected to be realistic samples of the kinds of work done at an elite consulting company, consultants using ChatGPT-4 outperformed those who did not, by a lot. On every dimension. Every way we measured performance. Consultants using AI finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without. [AI] works as a skill leveler. The consultants who scored the worst when we assessed them at the start of the experiment had the biggest jump in their performance, 43% when they got to use AI. The top consultants still got a boost, but less of one.

