Process

An AI quality gate is a decision, not a dashboard.

Dashboards measure what happened. A gate decides what is allowed to happen next. The difference is a threshold, an owner and a consequence, and without all three you have reporting rather than control.

The gate needs a decision attached to each band

A gate works by mapping a result to an action. Critical findings block until fixed. High findings require a named reviewer or a written acceptance. Medium warnings are fixed in the next pass. Low notes are informational and never block anything.

The bands matter less than the fact that they were agreed before the first report. A gate invented after a bad release is an argument about one incident; a gate agreed in advance is a standard, and people can plan around it.

Write the bands down in the same place as your code review rules. If the gate lives only in a tool's settings, it will be changed quietly during a rush.

Thresholds: score floors versus blocking findings

There are two ways to gate, and they fail differently. A score floor is easy to explain and easy to game: add a document section, remove a rule, split the file. A list of blocking finding types is blunt and durable, because it names the defects rather than a total.

Use both, in that order of authority. The blocking list decides; the score floor is a tripwire that asks a question when a file is unusually poor across many categories at once.

The mistake is gating on average score alone. A file with one hardcoded key and a clean remainder can score in the eighties, which is exactly the report nobody should ship.

The exemption problem

Every gate gets overridden, and the exemption process is where most of them rot. An override granted verbally, remembered by nobody, becomes a permanent hole in the rule set.

So make the override an artefact: who asked, why, what they accept, and when it expires. A fixture that legitimately contains a key-shaped string is a good reason. A deadline is a reason to record a risk, not a reason to disable the rule.

Expiry is the part people resist and the part that keeps the gate meaningful. An exemption with a date forces the conversation again when the situation has changed, which is usually all anybody needed.

Put the gate where the work already flows

A gate on a separate website is a gate that gets skipped on a busy Thursday. Put it in the pull request template, the CMS publish step, the email builder's send flow, the delivery checklist.

The same principle argues for speed. A check that takes three seconds is run every time. A check that takes a queue slot is batched, and batched checks are run at the wrong end of the process, when fixing costs most.

This is the practical reason for local analysis: the answer arrives while the person is still holding the draft, so the cheapest possible fix is also the most convenient one.

Evidence that the gate actually ran

A gate that cannot show a record is indistinguishable from a gate that was skipped. Keep the report with the change, showing the rules, the findings, the version of the rule set and the decision.

When the decision was to ship with a known finding, record that too, with the person who accepted it. That single line is what turns a future incident review from a memory exercise into a reading exercise.

Retention should be deliberate. The report and the decision are evidence; the raw draft is a liability, and in most cases the excerpt and findings are enough to reconstruct what happened.

Keeping a gate alive past the first quarter

Gates decay in two directions: they get loosened until they mean nothing, or they get tightened until people route around them. Both are visible in the override log, which is why it should be reviewed rather than archived.

A quarterly half hour is enough. Look at the most overridden rules, the most escaped defects and the share of items escalated. Then either fix the rule, add a human step or delete the rule with a reason.

A rule nobody can defend is worse than no rule, because it teaches the team that the gate is noise and that skimming is acceptable.

The human step is part of the gate

Some work should never pass on a score alone: authentication changes, payment flows, deletion paths, regulated claims, anything a customer will follow step by step. For those, the gate includes a named reviewer with a scope and a turnaround.

That reviewer is not a second gate. They are the part of the process that answers the question the rules cannot, and their verdict sits next to the automated report rather than replacing it.

The pairing is what makes a gate defensible: a mechanical baseline that anyone can reproduce, and an attributed judgement about fitness.

What this covers

  • Agreed bands

    Block, review, warn and note mapped to actions before the first report.

  • Blocking defects named

    A list of defect types that always stop the work, independent of score.

  • Recorded overrides

    Who, why, what they accept, and an expiry date.

  • Gate on the existing path

    Checks sit in the PR template or publish flow rather than a separate site.

  • Report kept with the change

    Rules, findings, version and decision travel with the deliverable.

Questions this page usually raises

Should a quality gate block every release?
Only for the defect classes where shipping would be inexcusable. Everything else warns, as long as the warning is read and the acceptance is recorded.
What score should be the threshold?
Pick one that your current output can actually reach after a normal fix pass, then move it only with evidence. A floor that most work fails teaches people to override.
How do we stop teams disabling the gate?
Make the gate fast, put it where the work already flows, and review the override log. A gate that costs thirty seconds rarely gets disabled.
Does a gate need a human reviewer attached?
For high-consequence work, yes. The automated pass proves known defects are absent; it cannot decide whether the change is fit for its purpose.
Can a gate run without sending work to a vendor?
Yes. Deterministic checks can run in the browser against pasted text, so the gate needs no upload and no new processor in your data map.

AI quality gate

Run the checks, then decide who has to sign it.

The automated pass is free, needs no account and uploads nothing. Escalate to a named reviewer only where being wrong is expensive.