Harvard study: why coding agents write more code but not more software

A Harvard working paper has been making the rounds since mid October, and it says what many engineering leads already feel. Teams that adopted coding agents write about 30 percent more code. They do not resolve more tickets. The extra code piles up in review, where it waits longer and gets sent back more often. For anyone running agents in a real team, it is the most useful study of the year so far.

What the study found

Fiona Chen and James Stratton, two researchers at Harvard, worked with data from Jellyfish, an engineering analytics platform. Their sample covers 718 companies and around 726,000 workers from January 2021 to March 2026, roughly 300 million events from GitHub, Jira and calendars. Because companies adopted AI tools at different times, the authors could compare each company before and after adoption against those that had not yet switched.

The writing side moved clearly. After a company adopted coding agents such as Claude Code or Cursor’s agent, each engineer produced about 1,500 more lines of code per month, an increase of 30 percent. Commits rose by 20 percent and pull requests by 23 percent. Simpler code assistants without agent mode showed much smaller effects.

The output side did not move. Resolved Jira issues and completed epics showed no significant change, and the estimates rule out an increase of more than 12 percent. The size of the issues did not shift either, so teams were not simply tackling bigger tickets. Employment stayed flat as well.

Where the work gets stuck

The gap shows up in review. After agent adoption, the time from opening a pull request to merging it rose by 3.45 days, 49 percent above the baseline of about a week. The share of pull requests that reviewers sent back for changes nearly doubled, from 13 to about 25 percent. Comments per pull request rose by 35 percent, although the pull requests did not get bigger. And 14 percent more engineers were pulled into reviewing.

The authors describe two stages, writing and checking. Agents speed up the first. As long as the second still runs on the same human time, faster writing only moves the queue. On top of that, the higher share of sent back pull requests points to a second problem: part of the agent’s code needs more correction than human code did. The authors note that the first problem can be solved by moving people into review, while the second needs better tools.

AI review is not the way out yet, at least not as most teams use it today. By March 2026 almost 80 percent of the companies had used AI review tools, but they wrote only 23 percent of review comments and appeared in about one in ten pull requests. In this sample, review stayed human work.

What to keep a sober eye on

This is a working paper, not yet peer reviewed. The data comes from Jellyfish customers who agreed to research use, which skews toward companies that measure their engineering closely. Adoption is measured per company, not per person, so the effect on individual engineers is probably larger than the averages show.

The review figure measures elapsed time from opening to merge, not hours of active reviewing. A pull request that waits three days in a queue counts the same as one that takes three days of work. And the observation window ends in March 2026. Agents and review tools have changed a lot since then, so the size of the effect may already look different today.

What holds regardless is the direction: more generated code does not turn into more finished work on its own. It turns into more to check.

What this means for teams

The study makes one implicit assumption worth questioning: that every change needs the same human review. That is how most teams work today, and it is exactly what turns more code into a longer queue.

The alternative is to review by risk. A change to the payment flow, the login or a data migration deserves a careful human review. A copy change, a dependency bump or a test for an existing function does not, as long as an agent has checked it against the ticket and the gate has passed it. How deep a review goes should depend on what a mistake costs at that point, not on who wrote the code.

The second lever is the share of work that fails review in the first place. In the study it nearly doubled. Often what gets sent back is nothing exotic: the agent missed a convention, misread the ticket or did not know how a module is meant to be used. That is a question of context: an agent that knows the codebase, its rules and its pitfalls, and that only sees what its step needs, produces less that has to go back.

If you want to act on the study, start by measuring the right thing. Lines of code and commits are the numbers that went up. Time to merge, the share of pull requests sent back and resolved tickets are the ones that tell you whether you ship more. Then sort your pull requests into two or three risk classes and decide, per class, who reviews. A simple start: changes to payments, authentication, permissions and data get a human reviewer every time. Changes inside one module with good test coverage get an agent review against the ticket, plus a human spot check on a sample. Everything else passes on the gate alone. After a month, compare the share of pull requests sent back per class and move the lines where the numbers say so.

At Kadmo, specialized agents work through playbooks in which every step ends with a gate, set by an agent or a person depending on what a mistake costs at that point. The agents carry knowledge about your software in their skills, so less work fails review: 92.5 percent of the pull requests our agents deliver are accepted in human review. If you want to see what this could look like for your team, we are happy to show you.

Sources

• https://fion.ac/jellyfish.pdf
• https://www.byteherald.com/2026/10/ai-dev-productivity-is-25-50-not-10x.html
• https://dev.to/jamilxt/ai-agents-write-30-more-code-shipped-software-did-not-move-code-review-is-why-47ii
• https://forkast.news/ai-coding-agents-write-30-more-code-and-resolve-zero-additional-issues-harvard-study-finds/