"100% of our code is AI-written." Congratulations. What did it cost?

Every engineering leader now says it. It tells you nothing.

AI-written code is not the same as cheaper or faster delivery. If a developer prompted a hundred times to get a feature that used to take an afternoon, you have not transformed anything. You have changed the typing.

What should you be measuring?

How many prompts it takes to get to a committed change, and whether that number is falling over time. It is all in the GitHub deltas.

Alongside that, the boring metrics that were always right: lead time, cycle time, deployment frequency, change failure rate.

Here is the uncomfortable part. In most large organisations I speak to, the head of engineering can quote DORA metrics for their own area, and almost nobody else in the business knows what DORA is. So there is no baseline. And a transformation without a baseline is not a result. It is an anecdote with a budget.

The decision for the executive: before the AI rollout goes any further, insist on a baseline from the current teams.

The good news is that this is now cheap. Point AI at Jira and it will tell you where the work is slowing down today. Then compare before and after. Anyone who cannot show you that comparison is asking you to take it on faith.

Measurement is built into the way Propel runs AI PDLC engagements, at every step from intent to implementation. If you want to understand more about how this is done in the best organisations, come and talk to us.