AI-generated code quality — the numbers disagree
Duplication up 81%, yet no general increase in code churn. Why the numbers on AI-generated code quality disagree, and what to measure in your own repo.
On this page
One side says that since AI adoption, code duplication has risen 81% and refactoring has fallen 70%. A study that approached the same question differently found “no general increase.” The first is data from a code-analysis vendor; the second is a peer-reviewed paper. What matters is not which one to believe but why they disagree.
Executive summary
The claim that AI is ruining code quality currently rests almost entirely on one company's data. That data is large and its metrics are specific, but it comes from a company that sells code-analysis tools. A peer-reviewed paper on the same question found results pointing the other way.
What the vendor data says
GitClear makes an analysis tool that classifies the nature of code changes commit by commit. In a report released in 2026, it said it had analyzed 623 million real code changes from 2023 to 2026. AI-assisted commits reportedly made up about a quarter of the total.
2.1 Reported metrics
| Metric | Change | Baseline as stated in the coverage |
|---|---|---|
| Duplicated code blocks | +81% | Pre-AI times |
| Copy/paste within commits | +41% | Pre-AI times |
| Code revised within two weeks | +15% | Pre-AI times |
| Moved lines (refactoring) | −70% | Pre-AI times |
| Maintenance of older code | −74% | Since 2023 |
| Cross-file function calls a proxy for code reuse | −35% | Not stated in the coverage |
The most symbolic figure is this: 2024 was the first year in which “copy/pasted” lines within commits outnumbered “moved” lines. In other words, copying code became more common than moving it around to tidy it up. Refactoring's share of all changed lines fell from 25% in 2021 to under 10% in 2024.
These two figures, however, may be re-citations of an earlier study by the same company rather than new results from the 2026 report. I could not open the original, so I could not tell which.
The trap in productivity numbers
An earlier report from the same company contains the most useful passage for this article. Using its own data, it shows how the common claim that “AI users are several times more productive” gets produced.
Diff Delta, the metric used here, was created by GitClear itself. It measures output by counting lines of meaningfully changed code rather than total lines.
4× to 10×
Developers who used AI daily made about 4× as many meaningful code changes as developers who did not use AI, and power users, the heaviest users, made 4–10× as many.
12% to 81%
Regular AI users +12%, power users +81%. Non-users came in at −2%, which was not statistically significant.
This distinction is useful in practice right away. When you evaluate the effect of adopting a tool, do not compare users with non-users. Compare the same people before and after adoption. The first gives you 4×; the second gives you 12%.
The peer-reviewed paper says otherwise
Another study from the same period analyzed the same subject a different way. Self-Admitted GenAI Usage in Open-Source Software is a peer-reviewed paper published in IEEE Transactions on Software Engineering. Its authors are Tao Xiao and six others, and the latest arXiv revision is dated April 5, 2026. (The journal publication date does not appear in the arXiv record, so I could not confirm it.)
The approach is different. The authors looked for cases where developers stated in commit messages, code comments or documentation that they had used AI.
The conclusion reads as follows.
On the same short-term churn metric, the vendor reported a 15% increase and this paper reported no general increase.
Why they disagree
There is no need to assume that one of them is lying. They measure different things.
| Vendor study | Peer-reviewed paper | |
|---|---|---|
| Sample | 623 million code changes. The trend across codebases as a whole | 151 repositories with self-admitted AI use. A tiny fraction of the 200,000 |
| Unit of analysis | Year-by-year trends aggregated across the industry | Per-repository before-and-after comparison, taking the first self-admission as the adoption date |
| Interests | Sells code-quality tools | Academic institutions. Peer reviewed |
| Disclosure | Published as a report. Raw data and methodology not public | Full paper publicly available |
The difference in unit of analysis is especially large here. One side looks at time trends across the whole industry; the other sets an adoption date for each repository and compares before and after it. Even with the same metric, they are answering different questions. The difference between the samples is also decisive. Developers who record AI use in their commits are likely to be relatively strict about development process and record-keeping. Only 156 of the 200,000 repositories did so. The vendor data, by contrast, looks at the overall trend regardless of such self-reports, but how it classified commits as AI-assisted has not been disclosed.
How to measure your own repository
From here on, this is my practical judgment, not research findings. When two studies disagree, the remaining option is to look at your own data.
git log -L only traces one specified line range and does not compute repository-wide ratios, and --find-renames is an option for detecting file renames; it does not count lines of code that moved.
So the list below is ordered by value for the cost.
| Difficulty | What | How |
|---|---|---|
| Easy doable today |
Duplication trend | Hook a copy-paste detector such as jscpd or pmd cpd into CI and record the values. Absolute values depend on the configuration, so look only at the direction under the same configuration |
| Easy | Effect of adopting a tool | Compare the same people before and after adoption. If you compare users with non-users, you will wrongly count as the tool's effect the gap that exists because people who already performed well started using AI first. This is called the selection effect. Any metric will do, as long as you keep using the same one without changing it |
| Medium needs a script |
Short-term churn | Write a script that parses git log -p output and measures the interval between the commit where a line first appeared and the commit where it changed again. Comparisons only work if you define it once and keep it fixed (within how many days, which lines count) |
| Hard | Share of refactoring | Changes have to be classified as “added, modified, moved or deleted.” This is not a value to build yourself; if you need it, use a commercial tool, or you are better off giving up on this metric |
Checklist
- Check whether a figure like “AI users are N× more productive” is the result of comparing different people
- Check what the organization behind the study sells that benefits from that conclusion
- Check whether how AI use was determined is disclosed
- Do not cite metrics that come without a base year
- Measure a baseline once today and write it down
- Hook a copy-paste detector into CI and watch only the trend
- Measure a tool's effect on the same people, before and after
- Check the two-week churn rate every quarter
What this is not yet
- The figures in this article are not our own measurements. They summarize a vendor report and a peer-reviewed paper.
- I could not access the vendor report directly. Access was blocked, so I took the figures from media coverage of it (LeadDev, 2026-01 and 2026-07). That coverage did not explain the methodology in detail either.
- I could not confirm how AI-assisted commits were classified. The maintenance metrics may be year-by-year trends across all changes, in which case they would not depend on the AI classification. Without opening the original, I could not tell which.
- Diff Delta is GitClear's own metric. It is not a standard metric that outsiders can verify or reproduce.
- This article does not use the 47% increase in error masking. The coverage gave no base year for it.
- The baseline in the table could not be pinned down either. None of the material I could reach said which years “pre-AI times” means. Some say the original uses 2023 as the baseline, but I could not confirm it.
- The productivity figures differ from source to source. Power users' improvement is reported both as +81% and as 25%. The cohorts and statistics appear to differ.
- The peer-reviewed paper's sample is also small and biased. Only 156 of 200,000 repositories self-admitted AI use, and developers who disclose it are likely to be more disciplined than average. The longitudinal analysis covers 151 of them.
- Neither study shows causation. Many other things changed over the same period.
- The measurement methods in section 6 are my practical judgment. Neither of the two studies above validated them. An earlier draft said these values could be measured with
git log -Land--find-renames; those commands do not actually perform that calculation, so the text was corrected.
Sources
| Source | What we used it for | Link |
|---|---|---|
| Self-Admitted GenAI Usage in Open-Source Software Tao Xiao and 6 others. IEEE TSE, peer reviewed. arXiv v4 2026-04-05 |
200,000+ repositories, 1,292 self-admissions. Longitudinal analysis of code churn in 151 repositories found no general increase | arxiv.org/abs/2507.10422 |
| GitClear — The Maintainability Gap (2026) Vendor report. Original not checked directly |
623 million code changes. Duplication, copy/paste, churn and refactoring metrics | gitclear.com |
| LeadDev — Code maintainability plummets in the AI coding era Bill Doerrfeld, 2026-07-07. Secondary source |
An article summarizing the report's metrics. The vendor figures in this article were checked here | leaddev.com |
| LeadDev — AI doesn’t create great developers, it amplifies them Bill Doerrfeld, 2026-01-20. Secondary source |
Diff Delta year over year +12% / +81% / −2%, cross-sectional comparison 4–10×, selection effect | leaddev.com |
This document is a technical summary of material that was public as of 2026-09-07. The vendor figures reached us through secondary coverage, so if you need to cite them, check the original report directly.