Skip to content
Sprint Radar
Help

Measuring AI impact

How AI-assisted work is told apart from other work, what is compared, the sample-size rule, and why the result does not prove a cause.

AI Impact compares work that AI coding agents helped with to other work, from your Jira data. It reports what it measured and how many items each number rests on. It does not say that AI caused a difference.

1. Tell Sprint Radar what AI-assisted work is

An owner or admin sets rules on the project page, in the AI Impact tab. A rule can match:

  • a label on the issue,
  • the assignee (the current one, or any assignee in the status history),
  • a custom field and its value,
  • the creator of the issue.

Without rules, Sprint Radar suggests labels and accounts that look like agents, with the number of issues each would match. You confirm them. You can also set an adoption date, the day the team started using agents.

2. What is compared

  • Items and points delivered per sprint, and the share of delivered work that is AI-assisted.
  • Lead time and cycle time, measured from real status changes. Cycle time counts delivered issues only.
  • Rework: how often finished items are reopened.
  • Abandoned work: issues that were started and then closed without delivery.
  • Work in progress for each group.
  • Throughput in the 6 sprints before and the 6 after the adoption date, when you set one.
  • A forecast without the AI-assisted completions, labelled an upper bound of the effect.

3. The sample-size rule

A comparison needs at least 20 delivered items in each group. Below that, the screen says “not enough data” and shows no difference. Items of one type (for example Bug) are compared only when both groups have at least 20 of that type. The number of items in each group is shown next to every figure.

Differences carry a 90% interval, computed by resampling. If the interval includes zero, the screen says there is no measurable difference.

4. Why this does not prove a cause

Teams often give agents the easy, well-specified items. Those items would have been fast anyway, so the groups differ before the tool is used. A newer group also looks faster when its slow items are still open. For these reasons every comparison carries a caution, and the wording on the screen describes what happened and never why. To learn a cause you need a design, for example comparable items assigned to both groups.

Issues without an assignee history are counted, because for them a rule on “any assignee” can only see the current assignee.

Plan

AI Impact is part of the Business plan, and the trial includes it. The demo project in every new workspace shows a populated example on any plan. See Plans, trial and billing.