B2B SaaS team · Series A, fourteen engineers
Reviewing agent-written pull requests without stalling the queue
Roughly sixty percent of pull requests arrived from an agent. Static analysis passed on all of them, because static analysis reads code without running it. Two of the four incidents that quarter came from errors that had been caught and silently discarded.
- 60%
- Of pull requests written by an agent
- 4
- Signature defect classes prioritised
- 1st pass
- Where blocking findings were caught
Four done, one to go.
Why the existing pipeline missed it
The pipeline answered a narrow question: does this compile, does the linter agree, do the existing tests pass. Generated changes that satisfied all three still shipped with credentials in example files and error paths returning empty results.
Nobody could review every line either. Reviewers were approving volume, which is the state in which a codebase acquires defects nobody has read.
The review sequence they settled on
The automated pass ran first and settled the mechanical questions: secrets, execution sinks, swallowed catches, broken markup, placeholder links. Reviewers then read only the entry point, one failure path and the boundary the change touched.
- Automated report attached to the pull request before a human opens the diff.
- Reviewer reads intent and entry point, not every helper.
- One failure path traced end to end, from event to what the user sees.
- Approval states the scope: what was reviewed and what was not.
What changed in practice
The blocking findings became predictable rather than surprising, which meant they could be written into the prompt instead of caught in review. Review time per pull request fell because the mechanical half stopped competing with the judgement half.
The team also started requiring a test for anything touching money or permissions, which is where the remaining incidents had concentrated.
Other engagements
Design and build, eight people
Signing off AI-assisted client work without hiring QA
News and features, thirty staff
Publishing AI images with provenance that survives the CMS
Client names withheld. Figures describe the pattern of these engagements, not audited outcomes.
Your own file
The first report is the useful one.
Run a single deliverable through the console and compare its findings with what your current review already catches.