Engineering

An AI code checklist is what you read before you merge, in order.

Generated code reads well enough that review becomes a skim. A checklist puts the questions in a fixed order so a fluent diff cannot talk you past them.

First: anything that is a live incident

Before design, before style, before naming. Scan for credentials in source, private key blocks, connection strings with embedded passwords, and literals assigned to variables named password, secret or token.

If one is found, rotate at the provider before touching the file. A checklist that starts with formatting has already wasted the review.

Generated code puts secrets in examples, in comments and in the snippet a developer pasted to ask why the connection failed. Search the whole file, not the diff.

Second: what executes, and what it executes on

Look for eval, the Function constructor, document.write, HTML assigned to innerHTML, shell commands reachable from application code and SQL assembled by concatenation.

Then look at the inputs. Generated handlers frequently accept a value and use it without checking, because the prompt asked for the happy path. Ask what a hostile value does at that line.

Check the rendering path too: an interpolation into markup is a defect even when the value looks internal today.

Third: what happens when it fails

Empty catch blocks, bare handlers and defaults returned in place of errors are the signature failure of generated code. Each turns a visible problem into a silent one.

Ask, for every external call: what happens on timeout, on a non-200, on a partial response, on a retry. If the answer is it returns an empty array, the function will report success while doing nothing.

Then check the failure is observable. An error swallowed and logged nowhere is a bug with a delay fuse.

Fourth: whether the API exists

Invented methods are the most expensive generated defect because they pass review and fail at runtime, often in a specific version nobody tested.

Confirm the imports resolve, the method exists on the pinned version, and the option names match the documentation rather than a plausible variant.

Where the code relies on a library behaviour, check whether that behaviour is documented or assumed. Assumed behaviour is the thing that changes between minor versions.

Fifth: residue, duplication and tests

Remove console statements, debugger lines, TODO and FIXME markers, commented-out alternatives and the paragraph of explanation a model sometimes leaves at the top of a file.

Look for the duplication that generated refactors create: the same helper written twice with slightly different handling, one of which has since been fixed.

Finally, ask what was not written. Tests are frequently skipped, error cases are often untested, and the diff that looks complete is the one most likely to have no coverage.

Sixth: who signs, and for what

Routine changes ship with the automated report attached. Changes to authentication, permissions, payments, data deletion or anything customer-facing get a named engineer who reads the file with the findings in front of them.

That reviewer states the scope: reviewed for access control and error handling, for example. A signature without a scope is an approval without a subject.

Keep the report with the pull request so the next reviewer inherits the reasoning rather than repeating it.

Dependencies that arrived with the change

Generated code reaches for a package to solve what a few lines would have solved, and every package is a future upgrade, a licence question and a supply chain risk.

For each new dependency ask what it does that the standard library cannot, who maintains it, when it last shipped, and whether the version is pinned. A package added to format a date is a package somebody will be upgrading in three years.

Configuration and the environment

Check that nothing environment-specific is hardcoded: a hostname, a bucket name, a port, a feature flag. Generated code tends to inline the value from the example it was written against.

Then check the failure mode when a variable is missing. The correct behaviour is a clear error at start-up, not a request that silently goes to the wrong host because the fallback default was real-looking.

Input handling and output encoding

Two questions per endpoint: what arrives, and where does it end up. Input validation is about shape and range; output encoding is about the context the value lands in, which is a different problem for HTML, for a SQL query and for a shell command.

Generated handlers frequently validate nothing and interpolate everywhere, because the prompt described correct input. Ask what a hostile value does at each line where a request field reaches a sink.

Loops, queries and payload sizes

The performance defects in generated code are rarely subtle algorithms. They are a query inside a loop, an unbounded fetch that returns the whole table, or a response that serialises every row when the caller needs a count.

Check the cardinality: how many items the loop will run over in production rather than in the fixture. A loop that is fine with three records is a page timeout with three thousand.

Concurrency, retries and the double click

Anything that changes state needs a second thought about arriving twice: a double submission, a retried request, two tabs open on the same record.

Ask whether the operation is idempotent, whether the database enforces the uniqueness or only the application assumes it, and what the user sees when the second attempt loses the race. Those questions apply to forms, payments, invites and anything that sends an email.

Logging that would actually help

Generated code logs the happy path and stays silent on failure, which inverts the useful arrangement. The log line that matters records what failed, for which identifier, and what was attempted.

Check three things: that errors are logged with an identifier a support engineer can search, that no secret, token or personal record appears in a log line, and that the failure is visible somewhere a human looks before a customer writes in.

What this covers

  • Secrets

    Provider patterns, key blocks, connection strings and named assignments.

  • Execution surfaces

    eval, Function, innerHTML, document.write, SQL concatenation.

  • Error paths

    Empty catches, defaults in place of failures, unobservable errors.

  • Real APIs

    Imports that resolve and methods that exist on the pinned version.

  • Residue and tests

    Debug statements, duplicated helpers and missing coverage.

Questions this page usually raises

Is this checklist specific to generated code?
It is ordered around the failures generated code produces most: secrets in examples, happy-path handlers, invented methods and missing error paths. Most of it applies to any diff.
Can the checklist be automated?
The first five sections are largely mechanical and a browser check covers them in seconds. The sixth is a human decision about scope.
How much time does it add?
Under a minute with an automated pass, and it usually saves more than it costs by removing the mechanical items from a reviewer's attention.
Does it replace tests?
No. A checklist finds patterns; tests establish behaviour. Generated code typically needs both, and the tests are the part most often skipped.
Should every AI-assisted change get a human review?
No. Gate on consequence: routine UI changes with a clean report can ship; authentication, payments and deletion paths should have a named reviewer.

AI code checklist

Run the checks, then decide who has to sign it.

The automated pass is free, needs no account and uploads nothing. Escalate to a named reviewer only where being wrong is expensive.