Stop Measuring AI By Lines Of Code
Authored by: Alex Ponomarev
Ask most software engineering leaders how AI is going on their team and you’ll get some version of the same answer: “We’re at about 30% of code generated with AI.” Or “AI wrote 40% of last sprint’s PRs.”
These numbers are easy to produce, easy to report, and almost completely useless. They tell you AI is being used. They tell you nothing about whether it’s working.
We’ve been here before:
- Velocity
- Story points
- Lines of code
Every few years our industry picks a new proxy metric, lets it masquerade as progress, and quietly walks it back once the damage shows up. AI is getting the same treatment just faster, because the numbers are bigger and the dashboards are prettier.
Why the easy metric wins
When leadership asks for AI impact numbers, two things usually happen. Someone wants a flashy figure to report upward, and no one knows what else to measure. The convenient metric wins because nothing is competing with it.
The trap is that volume metrics hide real problems. PRs get bigger and ship faster, but reviewers start firefighting changes they can’t reason about. Output climbs, quality drifts, the dashboard keeps glowing green. The team feels worse. The number looks better.
Four signals worth tracking instead
On my team, we still track the same headline number we tracked before AI: useful things shipped per sprint. What changed is where AI shows up inside that number. Four signals have been consistently more useful than any generation-percentage metric.
1. Cycle time from idea to testable code
Building a prototype used to be a project in itself — rough mockups, back-and-forth over weeks. Now, when someone has a hypothesis, they build it. Things that took weeks take days.
Practical tip: If you’re in a legacy codebase, don’t force AI into the old process. Spin the feature up as a standalone app, vibe-code it, validate the idea in a day, then present that. It’s much faster than wedging AI into a system it doesn’t know well, and it gives you something concrete to show.
2. Risk appetite
Teams are now willing to tackle work they used to defer.
Major refactoring that would trigger huge regression testing? That used to get pushed out for months. Now you can spin up a coding agent, review its plan, course-correct, and let it implement. The activation energy for risky work has dropped.
Practical tip: Keep a running list of projects that previously got deferred — major refactors, risky upgrades, migrations. Every sprint, note which of those are now being picked up. That delta is one of AI’s most valuable impacts, and it stays invisible to leadership unless you track it explicitly.
3. Flexibility across the codebase
Engineers used to own specific areas by necessity — the person who knew the payments service stayed on payments because ramping someone else up was too expensive. AI lowers that cost. A backend engineer can now make a reasonable change in an unfamiliar frontend module without a week of context-building first.
Practical tip: Notice who’s shipping in parts of the codebase they didn’t use to work in. Over a quarter, check whether more engineers are contributing to more areas than before. That cross-coverage is what protects you when someone goes on leave or leaves the company.
4. Stage-level speed, not overall averages
The real AI gains are concentrated in specific stages of the lifecycle, not spread evenly across it. For us, technical analysis and architectural planning moved significantly faster. Designers produce wireframes faster. Data engineers write queries faster. PMs read analytics faster. Coding speed itself varies engineer to engineer.
Practical tip: Break your development lifecycle into stages — technical analysis, design, implementation, QA, documentation — and report speedups per stage. A 3x gain in technical analysis is a real finding. Averaging it with a 1.1x gain in QA to land at “1.6x overall” buries the story. Don’t flatten the signal.
Conclusion
Vanity metrics tell you AI is being used. They don’t tell you it’s helping.
Track cycle time, watch what your team is newly willing to attempt, notice where engineers can now move freely across the codebase, and report gains stage by stage instead of as a single average.
The teams that measure honestly will keep finding the next edge. The ones waiting for a cleaner single-number dashboard will be waiting a long time.
Author Bio: Alex Ponomarev has spent 20+ years leading engineering teams across startups and enterprises, and writes about practical engineering leadership at Thriving in Engineering.