AI Training for Productivity and Accuracy: What Five Studies Actually Found

A useful starting point for AI training is what studies actually measured when people did work tasks with and without generative AIgenerative AI — Generative Artificial IntelligenceAI that produces new content — text, code, images, structured data — rather than only classifying or predicting from existing inputs., because those studies found both large gains and a measurable loss of accuracy, in one study within the same knowledge workflow.

This guide covers five studies of AI-assisted work, four randomized experiments and one study of a staggered rollout, with consultants, support agents, writers and software developers, what each did not test, and the training design that follows. For a role-by-role curriculum structure, see the four-tier AI upskilling framework; this article covers the evidence behind what goes into it.

Management consultants. Fabrizio Dell'Acqua and colleagues ran a preregistered experiment with 758 Boston Consulting Group (BCG) consultants, published as Navigating the Jagged Technological Frontier in the peer-reviewed journal Organization Science (online March 11, 2026). Participants were randomly assigned to no AI access, GPT-4 access, or GPT-4 access with a prompt engineering overview. In one arm of the experiment, 385 of the 758 consultants worked on 18 realistic knowledge tasks inside the frontier of AI capabilities, and those using AI completed 12.2% more tasks, completed them 25.1% more quickly on average and delivered solutions of significantly improved quality, according to the Organization Science paper.

Customer-support agents. Brynjolfsson, Li and Raymond studied the staggered introduction of a generative AI conversational assistant, built on GPT-3, using data from 5,172 customer-support agents, and published the results as Generative AI at Work in The Quarterly Journal of Economics (volume 140, issue 2). Access to AI assistance increased productivity, measured as issues resolved per hour, by 15% on average, according to Brynjolfsson, Li and Raymond. Less skilled and less experienced workers improved significantly across all productivity measures, including a 30% increase in the number of issues resolved per hour, according to Brynjolfsson, Li and Raymond, while the most experienced and highest-skilled workers saw small gains in speed and small declines in quality. The authors also report that agents who followed AI recommendations more closely saw larger productivity gains.

Professional writing. Noy and Zhang published an experiment in Science in July 2023 (volume 381, issue 6654) that assigned occupation-specific, incentivized writing tasks to 453 college-educated professionals. With a random half of participants exposed to ChatGPT, the average time taken decreased by 40% and output quality rose by 18%, according to Noy and Zhang in Science, and inequality between workers decreased.

Software developers, large scale. Cui, Demirer, Jaffe, Musolff, Peng and Salz analysed three randomized field experiments that Microsoft, Accenture and an anonymous Fortune 100 company ran, giving a random subset of developers an AI-based coding assistant suggesting code completions, in a paper published in Management Science (volume 72, issue 11, 2026). Each experiment was noisy and results varied across them, but combining the three experiments and 4,867 developers, developers using the AI tool completed 26.08% more tasks (standard error: 10.3%), according to the Management Science paper, and less experienced developers had higher adoption rates and greater productivity gains.

Software developers, small scale. In a randomized trial posted to arXiv as a preprint, not a peer-reviewed article, 16 experienced open-source developers completed 246 tasks that were randomly assigned to allow or disallow early-2025 AI tools. Allowing AI increased completion time by 19%, according to the study's authors at METR, although the developers had forecast beforehand that AI would cut completion time by 24% and estimated afterward that it had cut it by 20%. METR's February 2026 update says it is likely that developers are more sped up by AI tools in early 2026 than its early-2025 estimates showed, but that, because of selection effects in its new experiment, including developers choosing not to participate because they did not wish to work without AI, its new data is only very weak evidence for the size of the increase. METR's page for the original study now says those results are out of date.

The other arm of the consultant study, 373 consultants according to Dell'Acqua and colleagues, worked on a task the researchers designed to sit outside the frontier: a business case in which spreadsheet data seemed comprehensive, but a careful review of interview notes revealed crucial details. Consultants in the control group were correct about 84.5% of the time, while the two AI groups scored 60% and 70.6%, an average decrease of 19 percentage points, according to the Organization Science paper. The group that had received the prompt engineering overview recorded the more pronounced decrease (24.5% versus 13.9% for GPT-4 access alone), according to Dell'Acqua and colleagues.

The authors found that AI-supported participants produced recommendations that were more coherent and persuasive whether or not they were correct, which they say shows that AI improves presentation and argumentation even when the underlying analysis is flawed. They also report that the frontier's location is not immediately obvious to knowledge workers, even on familiar tasks, and that consultants tended to overrely on AI where closer human supervision was needed.

Inside the frontier, the prompt engineering overview helped: according to Dell'Acqua and colleagues, the overview group's quality gain was consistently larger than the GPT-only group's, with the difference below or around the conventional 5% significance threshold. The training lesson is direct: in this study, instruction on how to prompt improved work that AI handled well, and did not protect accuracy on the task AI handled badly.

These results describe their own settings and tasks, not your staff, and several of their limits matter for training.

No study tested a full training program. Of the five, only the consultant study included a training treatment, a prompt engineering overview; the other four compared access to an AI tool with no access. None of them reports the effect of a multi-week course, coaching or a verification curriculum.

The writing study did not check accuracy. MIT News reports that the Noy and Zhang team did not think it feasible to hire fact-checkers to evaluate the accuracy of the outputs, that participants were anonymous so the tasks could not require contextual knowledge, and that each assignment came with explicit instructions, whereas real-world tasks may be more open-ended.

One firm, one outside-frontier task. Brynjolfsson, Li and Raymond emphasize that their findings capture medium-run effects in a single firm. The consultant study's authors call their single outside-frontier task an inherent limitation, note that subjective metrics may contain evaluation bias, and say their results apply most directly to settings that prioritize quality within allocated time frames over speed.

The tools have changed. The consultants used GPT-4 as available at the end of April 2023, the support assistant was built on GPT-3, and METR now describes its early-2025 results as out of date and believes developers are likely more sped up in early 2026 than those estimates showed.

Each element below is tied to a specific finding. No study tested the design as a package, so measure it on your own work.

1. Train on each role's real tasks, sorted against the frontier. The consultant study's authors suggest organizations focus on the knowledge workflow and the tasks within it, and for each task evaluate the value of different configurations of humans and AI. In practice, run a sample of each team's recurring tasks with known correct answers through the approved tools, and label each one: AI drafts and a person checks; AI assists with parts only; or AI is not used.

2. Teach where not to rely on AI, with exercises that fail. The outside-frontier case was designed as a task at which knowledge workers would excel but AI would struggle, and the AI groups got it wrong more often than the group without AI. Include tasks from your own work where the AI output is plausible but wrong, and have staff find the error before they see the answer.

3. Make verification a habit, not an afterthought. The Government of Canada's Guide on the use of generative AI tells federal public servants not to consider generated content authoritative, to review it for factual and contextual accuracy by checking it against information from trusted sources, and not to use generative AI tools as search engines unless sources are provided so the content can be verified. The persuasiveness finding gives the reason: polish is not evidence of correctness, so reviewers should check claims, figures and sources rather than tone. Noy also told MIT News that speed benefits are lesser in the real world because you need to spend time fact-checking and writing the prompts; build that time into expectations.

4. Teach prompt basics, in proportion. The overview improved quality on tasks inside the frontier, and support agents who followed the assistant's recommendations more closely saw larger gains. Prompting is worth teaching; see the enterprise prompt engineering guide for techniques. It belongs alongside frontier and verification training, not in place of it.

5. Set data-handling rules before practice starts. The same Government of Canada guide states that public servants must not input personal information into publicly available online generative AI tools, and should not enter sensitive or personal information into any tools not managed by the Government of Canada. Staff need to know which tools are approved and what data may go into each before they practise on real work. HCC's AI governance and compliance service can help you write those rules.

6. Adjust the emphasis by experience. Three of the studies found larger gains for less experienced or lower-skilled workers, while the most experienced support agents saw small declines in quality. Those findings describe the populations studied; if the same pattern shows up in your own measurements, newer staff may benefit most from guided use, and experienced staff need to see where AI can lower the quality of work they already do well.

7. Measure speed and accuracy on your own tasks, not by impression. In METR's early-2025 study, which METR now describes as out of date, the developers believed AI had made them faster while the measurement showed the opposite. Baseline time and error rates on a sample of real tasks before training, then re-measure; see the AI ROI measurement framework.

8. Refresh the frontier map regularly. The consultant study's authors describe the jagged frontier as expanding and changing, and METR's update shows how quickly an estimate can date. Re-run the task sample when tools or models change.

The Government of Canada's guide, dated 2026-09-24 in its page details as of November 2026, is written for federal institutions; it says institutions should ensure employees can access and take training on the effective and responsible use of generative AI tools, and should support employees in improving their knowledge of topics such as detecting biased and inaccurate content.

If your organization is a provider or deployer of AI systems under the EU AI Act, Article 4, as replaced by Regulation (EU) 2026/1744 (the Digital Omnibus on AI, dated July 8, 2026, and published in the Official Journal on July 24, 2026), says as of November 2026 that providers and deployers shall take measures to support the development of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf. Those measures take into account the technical knowledge, experience, education and training of those persons, the context the AI systems are to be used in, and the persons on whom they are used, and the amended text states that the obligation does not require providers or deployers to guarantee any specific level of AI literacy of any individual.

Pick one team. Test its ten most common tasks with your approved tools against known answers, label each inside or outside the frontier, write the data rules and baseline time and errors. Then train on those tasks, including at least one exercise where AI is confidently wrong, and re-measure.

Plan adoption alongside training; the AI change management guide covers sponsorship, champions and reinforcement.

HCC's Corporate AI Training service can build this kind of program around your teams' own tasks: frontier-mapping exercises, verification habits, prompt basics and data-handling rules.

Dell'Acqua, McFowland, Mollick, Lifshitz, Kellogg, Rajendran, Krayer, Candelon and Lakhani — Navigating the Jagged Technological Frontier, Organization Science (2026), copy hosted by Harvard Business School Brynjolfsson, Li and Raymond — Generative AI at Work, The Quarterly Journal of Economics 140(2) Noy and Zhang — Experimental evidence on the productivity effects of generative artificial intelligence, Science 381(6654) MIT News — Study finds ChatGPT boosts worker productivity for some writing tasks Cui, Demirer, Jaffe, Musolff, Peng and Salz — The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers, Management Science Becker, Rush, Barnes and Rein (METR) — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, arXiv preprint METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (study page) METR — We are Changing our Developer Productivity Experiment Design (February 24, 2026) Government of Canada — Guide on the use of generative artificial intelligence EUR-Lex — Regulation (EU) 2026/1744 (Digital Omnibus on AI)