AI adoption is not AI productivity

The dashboard says almost every engineer used AI this month, against about half of them in the spring. The roadmap is exactly as late as it was in the spring. Both facts are true, and the second one does not contradict the first, because they measure different things.

Why doesn’t higher AI adoption mean higher productivity?

AI adoption counts inputs: people, sessions, tokens, suggestions accepted. Productivity is an output: working software that reaches users, at a cost and quality the business accepts. Between the two sits the whole delivery system (review, testing, release, coordination), and AI speeds up only the part of it where code gets written.

When one step of a system speeds up, the work piles up at the next one. If agents help each developer open twice as many pull requests and the number of reviewers stays the same, the queue in front of review grows. Adoption went up. The rate at which finished work leaves the system may not move at all.

flowchart TD
  accTitle: Where AI speeds up delivery and where work waits
  accDescr: Four steps in a chain. Writing code is faster with AI. Review is where the extra pull requests queue. Testing and release run at their old pace. Finished work leaves the system at the rate of the slowest step.
  W[Write code: faster with AI] --> Q[(Queue of pull requests waiting)]
  Q --> R[Review: same reviewers, same hours]
  R --> T[Test and integrate]
  T --> D[Release to users]
  D --> O[Finished work per week: set by the slowest step]

This is the oldest result in operations research, applied to code: speeding up a step that is not the constraint makes the queue in front of the constraint longer and changes nothing else.

What does the evidence say about AI and developer productivity?

The evidence splits cleanly by what each study measured: controlled tasks tend to show gains, real codebases show mixed results, and organization-level data shows gains being absorbed downstream.

Two findings from that table matter most for anyone running a dashboard. First, the METR perception gap: developers who were measurably slower believed they were faster. Any productivity claim built only on “how much faster do you feel” inherits that error. Second, the telemetry pattern: when adoption is high, the extra output shows up as larger pull requests and longer reviews, and the organization-level numbers stay flat.

Where do the individual gains go?

The individual gains go into the parts of delivery that AI does not speed up, mainly review and rework. The telemetry study measured pull requests 154% larger and review time 91% longer on high-adoption teams. A larger change takes longer to review, and reviewers can only read so much a day.

xychart-beta
  accTitle: Changes measured on high-adoption teams in a 2025 telemetry study
  accDescr: Bar chart. Tasks completed up 21 percent, pull requests merged up 98 percent, pull request size up 154 percent, review time up 91 percent, bugs per developer up 9 percent.
  title "High-adoption teams, change in a 2025 telemetry study"
  x-axis ["Tasks", "PRs merged", "PR size", "Review time", "Bugs per dev"]
  y-axis "Change (%)" 0 --> 160
  bar [21, 98, 154, 91, 9]

Read the five bars as two groups. Tasks and merged pull requests are what a team would celebrate. Pull request size, review time and bugs are what someone else on the team pays for. All five rose together.

Some of the gain also turns into code that has to be dealt with later. A 2025 analysis of 211 million changed lines found an eightfold increase in duplicated blocks of five or more lines during 2024, and the first year in which copy-pasted lines outnumbered moved (refactored) lines. Duplication is fast to write and slow to maintain, so it shows up as speed in one quarter and as cost in the next.

None of this means AI does not help. It means the help lands in one step and the measurement has to follow the work through the rest.

What happens when a team doubles its pull requests and keeps its reviewers?

A small worked example makes the queue visible. The numbers are illustrative. A team of eight developers opens 30 pull requests a week and its reviewers can finish about 34. Then the team adopts agents, and in week three the pull requests opened climb to 45 a week while review capacity stays where it was.

xychart-beta
  accTitle: Pull requests opened, merged and waiting over six weeks
  accDescr: Illustrative line chart. Opened rises from 30 to 46. Merged stays at 30 to 34. Waiting grows from 4 to 50 after week three.
  title "Illustrative team: opened, merged and waiting"
  x-axis ["W1", "W2", "W3", "W4", "W5", "W6"]
  y-axis "Pull requests" 0 --> 55
  line [30, 31, 45, 45, 46, 46]
  line [30, 31, 34, 34, 34, 34]
  line [4, 4, 15, 26, 38, 50]

The blue line is opened, the green line is merged and the copper line is waiting. By week six the copper line is the highest of the three. Merged pull requests rose by four a week, the most review could absorb. An adoption dashboard would show a 50% jump in output. A delivery dashboard would show a modest gain and a queue that, by week six, holds more than a week of the team’s review capacity. Time to merge climbs with it, because every new pull request now waits behind the backlog.

Cutting back on AI rarely helps. What helps is making each review cheaper: smaller pull requests, a review skill or subagent that checks the basics before a person looks, and an explicit agreement on what a reviewer must read and what tests already cover.

How should a team measure adoption and productivity without confusing them?

Keep two separate sets of numbers and read them next to each other for the same period, without computing a causal link between them.

Then read the two columns together. Adoption up and time to merge flat: the gains are reaching delivery. Adoption up and time to merge climbing: review is the constraint, and pushing adoption harder will make it worse. Adoption up and failure rate climbing: speed is being bought with quality.

HubBound Metrics is built to be read this way. It reports adoption (Active with AI, Sessions by tool, AI involvement in commits) and delivery (Pull request outcomes, Time to merge, Lines added) for the same period, each compared with the previous period of the same length. It shows the two side by side and deliberately does not compute a link between them, because a correlation on a dashboard is easily mistaken for a cause.

When is reporting adoption on its own the right call?

Adoption on its own is the right number early in a rollout, when the question really is “are people using it?”. It is also the right number for license decisions: a seat nobody opens is a cost with no possible return, whatever the productivity research says.

It is the wrong number for anything that sounds like “AI made us X% faster”. That claim needs an outcome metric, a baseline from before the change, and honesty about everything else that changed in the same period.

What should engineering leaders take from this?

Put your measurement where the work waits, not where it starts. The research points in one direction: AI speeds up writing code, and the time saved collects in front of review, testing and release. So the most informative single number for an AI rollout is often not an AI metric at all. It is time to merge, or review time, tracked as adoption rises.

If that number holds steady while adoption climbs, the team is absorbing the extra output. If it climbs, no amount of adoption will turn into delivery until the review step changes: smaller pull requests, review skills and subagents that check the basics first, or simply more people reviewing. Our article on how to measure AI adoption covers the adoption side of that pair.