The broken pipeline: how do you train auditors when nobody starts by programming anymore?
The earlier articles in this series reached a conclusion that is comfortable for anyone who already has experience: human work is moving toward scoping, specifying and auditing, and that demands more judgment, not less. But judgment does not appear on its own. It is built by doing, and much of what people used to do to build it is exactly what AI does now.
That is the paradox of this article: if newcomers no longer write the simple code, fix the easy bugs or get their work reviewed by someone more experienced, where will the people capable of auditing AI output come from ten years from now?
The irony, already 40 years old
In 1983, Lisanne Bainbridge published a short paper on the automation of industrial processes that became a classic: Ironies of Automation. Her argument: when almost everything is automated, the person is left to monitor the system and step in for rare cases. But because they hardly practice anymore, they lose the very skill those cases require. And she ended with the central irony: the most successful automated systems, the ones that rarely need human intervention, are the ones that require the greatest investment in training the people who supervise them.
AI-assisted software development reproduces the same pattern. With one difference that is far from minor: in industry, the supervisor had already learned by operating manually before automation came in. Here, a newcomer may never have done so.
What the evidence says
| Finding | Source | What it shows | What it does not show |
|---|---|---|---|
| Employment of 22- to 25-year-olds in AI-exposed occupations is 19% below where it would be had it kept pace with less-exposed peers; there is no such gap for experienced workers | Brynjolfsson, Chandar and Chen, Stanford, August 2026 | The gap comes from fewer hires of newcomers | That AI causes it: the authors call it a descriptive indicator |
| Of the recovery in US software development job postings, 71% were senior roles | Indeed Hiring Lab, July 2026 | The market is looking for experience | That hiring of newcomers will not come back |
| Students who practiced with unrestricted AI improved 48% during practice and, once it was removed, scored 17% lower on the exam than those who never used it; a tutor with guardrails avoided almost all the harm | Bastani et al., PNAS, 2025 | Using AI as a crutch prevents learning | What happens in the long run |
| Programming students using AI had an illusion of progress: nine of ten who struggled reached a solution, and most believed they understood more than they did | Prather et al., ICER, 2024 | AI hides from learners what they do not know | That the same happens with other ways of using it |
| Among experienced endoscopists, the adenoma detection rate without AI dropped from 28.4% to 22.4% after months of working with AI | Budzyń et al., Lancet Gastroenterology & Hepatology, 2025 | Even with experience, skill erodes when AI exercises it for you | Causation: it is an observational study in four centers |
| Automation complacency and bias appear in both novices and experts, and simple practice does not remove them | Parasuraman and Manzey, Human Factors, 2010 | Monitoring a machine that is almost always right is hard for anyone | How it plays out with generative AI specifically |
The six sources point the same way from different angles. Fewer newcomers are being hired. Learning while AI does the work leaves less learning behind. And people who already know lose skill if they stop using it.
How people used to learn, and what is being lost
Traditional software training was largely implicit. Nobody designed it: it happened because that was how the work was organized.
| What newcomers did | What they learned without trying | Does AI do it now? |
|---|---|---|
| Write repetitive, simple code | Fluency in the language, the project's patterns | Almost always |
| Fix bugs reported by others | Reading other people's code, debugging, understanding the system | Increasingly |
| Get reviews from someone experienced | What counts as correct, and why | If AI does the review, the human model is lost |
| Write tests for someone else's work | Thinking about edge cases | Yes |
| Break something in production and fix it | Respect for consequences | Less, if controls catch it first |
The right-hand column is the problem. The tasks AI does best are precisely the entry-level ones, the ones people learned on.
What an auditor needs to know
In the article on the new roles we listed what an auditor needs: reading code critically, adversarial thinking, knowing how AI fails, specifying without ambiguity, basic statistics and domain knowledge. None of them can be learned by reading about them. All of them require having seen many cases, having been wrong and having been corrected.
The common-cause experiment gave a concrete example: a test property that looked solid proved nothing because of how the code was written. Spotting that requires understanding the code from the inside, not just seeing that the tests pass.
A proposal: learning by auditing
If entry-level work no longer trains people, training has to be designed explicitly. Three ideas from education research serve as a foundation:
- Deliberate practice (Ericsson and colleagues, 1993): expertise comes from practicing just above one's level, with concrete goals and immediate feedback.
- Cognitive apprenticeship (Collins, Brown and Newman, 1989): the expert shows how they think, coaches, provides scaffolding and gradually removes it.
- Desirable difficulties (Bjork, 1994): what takes more effort during practice is what is retained best. A tool that removes all the effort also removes the learning.
With those three ideas, a four-level ladder:
| Level | Duration | Programming without AI | Specifying | Auditing | Working with agents |
|---|---|---|---|---|---|
| 1 | Months 0 to 3 | 50% | 15% | 25% | 10% |
| 2 | Months 3 to 6 | 30% | 25% | 25% | 20% |
| 3 | Months 6 to 12 | 15% | 25% | 35% | 25% |
| 4 | After one year | 10% | 25% | 40% | 25% |
The split is a proposal to discuss and test, not a measurement. What matters is the shape:
- Level 1: build the foundation. Program small problems by hand, without code-generating assistants. Read other people's code daily. Audit AI-generated code with seeded bugs, which gives immediate, objective feedback: you know how many there were and how many you found.
- Level 2: specify before generating. Write user stories and tests before the code, and use AI as a tutor that gives hints instead of answers, which is what prevented the harm in Bastani's study.
- Level 3: audit real work in pairs. Review real low- and medium-risk changes alongside someone experienced who reviews the same change and compares. Measure test quality with mutation testing.
- Level 4: autonomy with a second signature. Audit high-risk changes with a second person who signs off, write threat models and start training level 1.
How to tell whether it works. Auditing with seeded bugs gives an objective metric: what share of the bugs each person finds, and whether their confidence matches their accuracy. If the rate does not rise between levels, the ladder is not working.
If a review had Eseeded planted bugs and the person found Efound, their detection rate is:
Who pays for it
This is the uncomfortable part. Training someone costs experienced people's time, and for months the learner contributes less than they consume. A company that does not hire newcomers and recruits people others already trained avoids that cost. If many do the same, the pipeline dries up for everyone.
It is a collective action problem, and no single company solves it alone. Some existing pieces can help:
- Universities, if they adapt their curricula as we proposed in Teaching software engineering when AI writes the code.
- Internships and supervised professional practice, oriented toward auditing.
- Shared entry-level programs across several companies, or partly funded by those that hire the most experience.
What is measured and what is not
| Claim | Status |
|---|---|
| Fewer young people are hired in AI-exposed occupations | Measured, no established causation |
| Learning with unguarded AI reduces what is learned | Measured in the short term, in high school |
| AI gives programming learners a false sense of understanding | Observed in a small study |
| Experts' skills erode when working with AI | Observed in medicine, observational study |
| A ladder like the one proposed trains competent auditors | Not measured: it is a proposal |
| Without explicit training there will be a shortage of auditors in ten years | Plausible, not shown |
Rival explanations
- The hiring drop is not about AI. It could be the correction after the 2021–2022 hiring boom, or interest rates. The Stanford study finds rates do not explain it and that it persists outside tech companies, but it is still descriptive.
- AI will audit too. Possibly, and to some extent it already does. But the experiment and the common-cause problem suggest someone independent of the AI will still be needed, at least where failure is costly.
- You can audit without having programmed. New tools may make that possible. Today there is no evidence that someone without experience writing code is good at spotting its bugs.
What this article does not claim
- It does not claim AI is bad for learning. It claims it depends on how it is used: as a tutor it helps, as a crutch it gets in the way.
- It does not claim the proposed ladder is the right one. It is a design hypothesis to test and measure.
- It does not claim a shortage of auditors is inevitable. It claims it is likely if nobody designs training explicitly.
- It does not blame companies that do not hire newcomers. It describes an incentive that can only be corrected collectively.
Sources
- Bainbridge, Ironies of Automation, Automatica 19(6), 775–779 (1983).
- Brynjolfsson, Chandar and Chen, Canaries in the Coal Mine?, Stanford Digital Economy Lab, August 2026 version.
- Indeed Hiring Lab, US software development job postings, July 2026.
- Bastani et al., Generative AI without guardrails can harm learning, PNAS 122(26) (2025).
- Prather et al., The Widening Gap, ICER (2024).
- Budzyń et al., Endoscopist deskilling after exposure to artificial intelligence in colonoscopy: a multicenter observational study, The Lancet Gastroenterology & Hepatology 10, 896–903 (2025).
- Parasuraman and Manzey, Complacency and Bias in Human Use of Automation, Human Factors 52(3), 381–410 (2010).
- Ericsson, Krampe and Tesch-Römer, The role of deliberate practice in the acquisition of expert performance, Psychological Review 100(3) (1993).
- Collins, Brown and Newman, Cognitive apprenticeship, in Knowing, Learning, and Instruction (1989).
- Bjork, Memory and metamemory considerations in the training of human beings, in Metacognition (1994).
- Dixi-Project: Verification debt, Scope, specify, audit, Teaching software engineering when AI writes the code, Do AI-written tests catch AI-written bugs? and One maintainer, one agent and 141 PRs with no human review (2026).