AI Generated Code Quality: How to Check and Scan It Before Merge

To check AI-generated code quality, prove the behavior with tests, scan for repeatable residue, fail CI when the scan finds it, and spend human review on intent. This guide covers each step and the commands to scan locally and in CI.

The usual stack of tests, a linter, and one approval was built for a person editing a few files. An agent can add a hundred lines that compile and pass the existing tests while leaving residue behind, and it writes faster than reviewers read. The check has to get cheaper per change. For which product does which job, see the code quality tools guide.

Four things decide whether a change is good

1. Behavior. Tests that fail when the feature is broken, not tests that assert the mock.
2. Residue. Named AI slop: swallowed exceptions, leftover comments, dead code, unsafe casts. Automate this.
3. Dependencies and secrets. Verify every new package. Do not merge placeholders.
4. Intent. A person confirms the patch matches the request and the architecture.

The first three can be checked by machine the same way every time. The fourth needs judgment, so keep it separate and save people's time for it.

The problems that hurt most are rarely syntax errors. Empty catches turn failures into wrong answers, an as any silences the compiler without fixing the type, unused helpers and TODOs pile up with no owner, agents weaken tests to stay green, and rewrites grow too large to verify. All of these pass a formatter, and linters catch only the subset they were built for.

Check code quality in four steps

1. Write the bar before you generate

Before the agent edits, name the behavior, the files it may touch, and the tests that would prove it. Without that, review argues about taste. Keep the patch small enough that a reviewer can hold it in their head; a gate cannot recover intent the prompt never stated.

2. Automate the repeatable checks

Run the project's format, lint, type, and test tools on the changed files before anyone reads the diff. Then cover what those tools ignore: empty catches, as any, leftover TODOs, unused helpers, and hallucinated imports often pass ESLint and the test suite. That gap is what a deterministic scanner is for.

3. Score the residue and fail CI on it

A scanner such as aislop gives a 0 to 100 score and named findings, and the same code always gets the same score. Details are in the scoring docs. Run it in CI so the merge is blocked when the score drops below the team's threshold or an error-severity finding appears.

Raise the threshold gradually: record the current score, fix or suppress noisy rules, then raise ci.failBelow. A 90 gate on a codebase that scores 40 will only block everything.

4. Keep human review for intent

After tests and the gate, the remaining diff should be smaller. Reviewers check whether it matches product intent, fits the architecture, invents a dependency, or weakens a test. The AI-written code review checklist covers this layer. If the diff is still huge, shrink the prompt and the patch rather than asking a person to absorb a generated rewrite.

How to scan AI generated code

A slop scan is a deterministic pass for named AI slop patterns plus format, lint, complexity, and security checks. It names residue, scores the tree, and can fail CI, so quality no longer depends on who had time to read the diff.

Local CLI

From the repo root:

$ npx aislop@latest scan
$ npx aislop scan --json

The first command scores the whole tree; the second emits machine-readable output for scripts. Full walkthrough: npx aislop scan.

Changed-file mode

Agent output arrives as a diff, so scan the diff: the files you just touched, or the files that differ from a base branch. Run it before opening the pull request.

$ npx aislop scan --changes
$ npx aislop scan --changes --base origin/main

CI gate

npx aislop init writes .aislop/config.yml and a GitHub Actions workflow. aislop ci exits non-zero below failBelow or when an error-severity finding is present. To gate only the pull request diff:

$ npx aislop@latest ci --changes --base origin/main

Setup details are in the CI / CD docs. The hosted gate on every PR is scanaislop, and for a public GitHub URL, try aislop runs a real scan with no install. Both use the same engine.

Limits

A passing scan shows the change did not trip the rules you encoded. It does not show the feature is right, matches the request, or fits the architecture, so do not treat a score of 90 as an architecture review. Open-ended LLM review can suggest logic issues, but it is not a stable merge rule; keep it advisory until you have evidence (see deterministic AI code review). To compare scanners on what they catch, use the AI code quality checker checklist.

To start, run npx aislop scan --changes on your next agent change. CLI release notes are in the changelog.