Scope, specify, audit: the new human roles in AI-assisted software development

In our article on verification debt we showed that, with AI, a team produces software faster than it can verify it: between January 2024 and September 2026, the number of pull requests merged on GitHub each month grew 5.6×, while those approved by another person grew only 2.6×. The consequence for people is direct. If the bottleneck is no longer writing code but validating it, human work has to move to where the bottleneck is.

This article proposes how. The thesis: people stop being universal code reviewers and take on three functions instead: scoping, specifying and auditing. For each one we describe what it does, which tools it uses, what it delivers and how it is measured. We then compare what a product team looked like in 2023 with what it should look like today, and review what the data says about the actual reduction in headcount, including the evidence that cuts against the thesis.

One clarification first: "stop reviewing code" does not mean stop reading it. It means no longer reviewing all of it, line by line, as the main control. Full human review is still mandatory wherever failure is expensive: payments, authentication, personal data, security.

Why the role has to change

The DVI model from the previous article describes verification debt with one equation:

where G is the number of changes generated per day, α the hours needed to verify each one, V the verification capacity and δ the rate at which pending debt gets more expensive. Add p_c, the probability that code and its tests share the same mistake, and C_i, the cost of a failure reaching production.

Until recently, a developer's hour went mostly into G: writing code. Agents now produce G at almost zero marginal cost, and adding more G without moving anything else only grows the debt. A human hour is worth more in the other terms:

None of the four is new. What is new is the proportion: they used to be the smaller part of a team's work, and now they are almost all of it.

What is measured and what is not

Before proposing roles and team sizes, it helps to separate what the data supports from what is interpretation.

Three readings the evidence supports:

1. The reduction is real, but it comes in through hiring. Fewer people are hired at the start of their careers; experienced people are not being let go at a higher rate.
2. Generating faster does not guarantee delivering more. METR and DORA agree that productivity gains are not automatic; they depend on how work is organized around AI.
3. What is in demand is judgment. The postings coming back are senior and AI-related: people who can decide what to build and judge whether what was built is right.

The three roles

1. Scope: decide what gets built, where AI may act and how bad failure would be

Whoever scopes sets the boundaries AI works within. This role inherits from product management and technical leadership; on a small team one person can cover it.

Responsibilities

• Frame the problem, and what is out of scope, before asking for a single line of code.
• Classify the risk of each change (low, medium, high, critical) and decide which controls each level requires.
• Set a change budget per cycle: how much work in progress fits without exceeding the team's verification capacity.
• Draw the system's boundaries: modules, contracts between them, and what an agent may touch without extra permission.
• Reserve capacity to pay down old debt, at least as fast as it grows more expensive.

Tools: architecture decision records (ADRs), versioned contracts (OpenAPI, JSON Schema, AsyncAPI), domain maps, risk matrices, work-in-progress limits, and permissions and hooks that restrict what an agent can run or edit.

Delivers: written scope, a risk class for every change, recorded decisions, and technical boundaries enforced by machines rather than goodwill.

Measured by: the share of changes with a risk class, changes that cross agreed boundaries, and the ratio of generated to verified work (ρ̂ in the previous article).

2. Specify: state precisely what "correct" means

Whoever specifies turns a need into something a machine can check. This role inherits from business analysis and from much of programming: instead of writing the implementation, you write what the implementation must satisfy.

Responsibilities

• Write user stories with verifiable acceptance criteria.
• Turn those criteria into tests before the code exists, and freeze them so the agent cannot edit them to make them pass.
• Define properties and invariants: what must hold for any input, not just for the examples.
• Maintain the contracts between components and the domain's edge cases.
• Split work into small changes, because a small change is far cheaper to verify.

Tools: user story templates, Gherkin or EARS for requirements, acceptance tests, property-based testing (Hypothesis, fast-check, jqwik), contract testing (Pact), strict type systems and, in the critical core, formal specification (TLA+, Alloy, Dafny).

Delivers: an executable specification: failing tests that describe the expected behavior before the code exists.

Measured by: the share of changes with tests written first, average PR size, and how many defects escape because a specification was ambiguous or incomplete.

3. Audit: check independently that what was built is what was specified

Whoever audits does not review everything: they decide what to review, how and how deeply, based on risk. This role inherits from quality assurance and code review, with one fundamental change: its value lies in being independent of whoever generated the code, including the AI model.

Responsibilities

• Review high- and critical-risk changes line by line.
• Audit the rest by sampling: pick a random fraction of low- and medium-risk changes and review them in depth, to estimate how many defects are slipping through.
• Measure test quality, not just coverage: how many seeded bugs survive (mutation testing).
• Use verifiers that do not share the generator's bias: another model, another person, static analysis, tests written from the specification.
• Build the threat model for features that handle personal data, money or external input, with one test per threat.
• Spot rubber-stamping: approvals too fast for the size of the change.

Tools: mutation testing (Stryker, PIT, mutmut), static and security analysis (Semgrep, CodeQL), secret and dependency scanners, a different AI model from the generator for pre-review, risk-based review guides, STRIDE for threats, and basic sampling statistics.

Delivers: a verdict for every high-risk change, and a periodic estimate of the escaped-defect rate for everything else.

Measured by: mutant survival rate, defects found in production versus before it, and change failure rate (DORA).

The roles that remain

• Operate (platform, SRE): gradual rollout, feature flags, automatic rollback, observability. It lowers C_i: it does not reduce debt, but it makes an escaped failure tolerable. Often shared across teams.
• Design (UX): AI produces interfaces quickly, but deciding what a user actually needs is still research. Also often shared.

Where each role comes from

Skills to develop

What loses relative value: memorizing syntax, writing boilerplate, typing speed, and knowing every library's API by heart. What does not: understanding how software works on the inside. Without that foundation, auditing is impossible.

What a team looks like: 2023 versus today

Take a typical 2023 product team, of the size the industry commonly used for a web product with an app and an API: ten people, four of them dedicated to writing code.

Following the logic of this article, the same product today would need about six full-time equivalents:

How to read this comparison:

1. It is an illustrative composition, not a measurement. No study today measures the size of product teams before and after AI. The data in the previous section measures employment and job postings, not teams.
2. The reduction comes from writing code, not from verification. Verification capacity stays the same or grows. Cutting there, with more code coming in, is exactly what makes debt grow.
3. The number depends on risk. By the same reasoning, a low-risk internal product could run with three people, and a payments or healthcare system might need the same ten as before: the savings in programming get reinvested in verification.
4. It depends on what the company does with the productivity. If it uses it to build more products instead of cutting, total employment may not drop even though each team is smaller. The Stanford authors themselves note that higher productivity can raise or lower employment depending on how much demand grows.

What can go wrong

This reorganization has costs and risks worth stating up front:

1. The talent pipeline breaks. If fewer people are hired at the start of their careers, as the Stanford data shows, where will people with the judgment to audit come from ten years from now? Nobody learns to judge code without having written and broken it first. A team that thinks long term has to keep training people, even if that costs more today.
2. Smaller teams concentrate knowledge. With six people, losing one matters more. Recorded decisions and explicit contracts stop being bureaucracy and become the team's memory.
3. Auditing machine output is tiring and lulling. When the system is almost always right, attention drops and approvals happen without looking. The previous article already found the signal: in its sample, PRs signed by Claude Code received 4.8× fewer reviews per 1,000 lines.
4. Productivity is not guaranteed. METR measured experts taking 19% longer with AI; DORA, lower stability. Cutting people on the assumption of a gain the team does not yet have is taking on debt in advance.
5. Regulated contexts change the math. In sectors with external audits, evidence of human review is a requirement, not an efficiency choice.

How to make the transition

1. Measure before you cut. Work out how many changes are merged for each one reviewed, and estimate the debt with the calculator in the previous article.
2. Classify the risk of each part of the system and write down which controls each level requires.
3. Make tests-first the rule, frozen so the agent cannot edit them.
4. Turn QA into verification engineering: mutation, sampling and independence instead of manual execution.
5. Move people who used to program into specification and auditing, depending on their strengths, with explicit training in the skills above.
6. Reserve capacity to pay down debt, at least as fast as it grows more expensive.
7. Keep an entry-level program: people who start by auditing under supervision and writing specifications, so the judgment exists five years from now.

What this article does not claim

• It does not claim code no longer needs reading. It claims that reading all of it stops being the main control, and that you read where it matters.
• It does not claim AI has already cut software employment across the board. The available evidence shows fewer young people being hired and postings below 2020 levels, with causes still under debate.
• It does not claim six is the right number for every team. It is an illustration for a medium-risk product; the right size follows from risk and verification capacity.
• It does not claim people are redundant. It claims their work moves, and that where it moves demands more judgment, not less.

Sources

• Brynjolfsson, Chandar and Chen, Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence, Stanford Digital Economy Lab, August 2026 version.
• Indeed Hiring Lab, US software development job postings data, July 2026.
• Becker, Rush, Barnes and Rein, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR (2025).
• Peng, Kalliamvakou, Cihon and Demirer, The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (2023).
• DORA, Accelerate State of DevOps Report (2024) and State of AI-assisted Software Development (2025).
• US Bureau of Labor Statistics, Occupational Outlook Handbook: Software Developers, Quality Assurance Analysts, and Testers, 2024–2034 projections.
• Dixi-Project, Verification debt in AI-assisted development (2026): GitHub data and the DVI model.