Skip to content
Sprint Radar

Blog ·

What AI coding agents do to your delivery dates (and how to measure it)

Many teams now ship with AI coding agents next to the people. The question for a delivery lead is not whether the agents are impressive. It is what they change in the dates you promise. We do not know the answer for your team, and nobody else does either. You can measure it. This article says what to measure and how much to trust it.

Four ways agents can move a date

These are possibilities to check, not findings.

  • Throughput. The team may finish more items per sprint. Or the same number, with less effort.
  • Cycle time. Items may move from start to done faster. Or slower, if review takes longer.
  • Rework. Items may be reopened more often, or less often.
  • Abandoned work. Cheap attempts may produce more items that are closed without delivery. That inflates counts without moving the date.

Each of these can go either way. A speed-up in cycle time that comes with more reopened items may leave the date where it was.

What to measure

Start by marking which issues were AI-assisted. Use something your team already records: a label, an assignee that is an agent account, a custom field. Then compare the two groups on delivered items per sprint, lead time, cycle time, reopen rate, abandoned work and work in progress. Measure from status changes in Jira, not from estimates.

Add one time comparison: throughput in the sprints before and after the date the team started using agents. And one forecast question: what would the dates be if the AI-assisted completions were left out?

Why sample size decides everything

Sprint-to-sprint variation is large. In the synthetic demo project in Sprint Radar, throughput over the last 6 sprints ranged from 18 to 31 points. That swing of 13 points came from ordinary variation, with no change to the team at all.

An effect smaller than that swing cannot be seen in a few sprints. Comparing two groups of ten items has the same problem. This is why a careful comparison shows the number of items in each group, gives an interval instead of one number, and refuses to compare when a group is too small.

What the data cannot show

Suppose AI-assisted items have a shorter cycle time. That does not show that the agents made them faster. Teams may give agents the easy, well-specified items. Those items would have been fast anyway. The groups differ before the tool is used.

So treat the comparison as description. It tells you what happened, with how many items. It does not tell you why. If you want a cause, you need a design: assign comparable items to both groups, or compare the same team before and after a change that nothing else coincided with.

How Sprint Radar handles it

AI Impact, part of the Business plan, does the comparison above from your Jira data. It reports measured values and the number of items per group. It withholds a comparison when either group has fewer than 20 items. It puts a caution on every comparison. It also reports a forecast without AI-assisted completions, labelled as an upper bound. Read how it works. Usage and cost for Claude Code, GitHub Copilot and Cursor is a separate preview page.

What to do now

Mark AI-assisted work in Jira today, even before you measure. History you did not record cannot be recovered. Keep the forecast honest in the meantime: if throughput changed, the model needs a few sprints of the new pattern before its dates mean much, and a good tool will warn you while it waits.

See a forecast with odds for your own project.