Engineering April 21, 2026 · 5 min read · reads

What 25 real projects taught us about aislop 0.5

How do you know a release is actually ready? Fixture tests say yes. Real repos say it depends. Here is what we landed on before tagging aislop 0.5: we ran it against 25 real projects from our own backlog, and it told us what to fix before users did.

When is a release actually ready to tag? Some teams ship once the fixtures pass. Some wait for a beta user to flinch. Some cut the release, then hot-fix all week. For aislop 0.5 we wanted to know before we shipped, so we pointed it at 25 real projects from our own backlog and watched what broke. Here is what we found, what we fixed, and what changed about how we build the tool.

The validation run

Fixture tests prove the tool passes. Real projects prove the tool does not corrupt. So we ran aislop scan and aislop fix -f against 25 production codebases in the Skivelane backlog. A mix of React and Next apps, Expo projects, a Chrome extension, Remotion video templates, and a handful of Node backends.

Outcome	Count	Notes
PASS	15	Scan and fix ran clean. No source corruption.
PASS_WITH_WARNINGS	3	Ran clean. Flagged real pre-existing issues.
GAP	4	Destructive corruption from external fixers.
SKIPPED	2	Install failed before we could scan.
PARTIAL	2	Hung on `expo install --fix`.

Median score improvements across the PASS set:

before: 15

after fix: 32 (+17)

after fix -f: 47 (+32)

A few standouts:

+ chrome-extension: 9 → 68 → 88 (+79)
+ dailyapp-backend: 27 → 87 → 94 (+67)
+ buildwithkenny: 47 → 58 → 100 (+53)

The four GAP projects are the interesting ones. Every single gap came from a destructive auto fix that broke TypeScript in a way no fixture test had caught.

The four bugs

1. Dangling colon on aliased destructure removal

Project: zingo-web. The user's code:

const { languages, isLoading: languagesLoading } = useLanguages()

After oxlint --fix stripped the unused alias:

const { languages, isLoading: } = useLanguages()

// TS1003: Identifier expected.

What we did. We stopped asking oxlint --fix to remove unused destructure identifiers. aislop owns that rewrite now. When the alias is unused, we remove the whole key: alias pair through the TypeScript compiler API. Not a regex.

2. Invalid rest-element rename

Project: snappymenu. aislop's own regex-based rename helper hit rest parameters:

({ ...props }) => <ChevronLeft />

The naive foo: _foo rename produced:

({ ...props: _props }) => <ChevronLeft />

// TS2566: A rest element cannot have a property name.

What we did. Threw the regex rename out. The new implementation walks the AST and classifies each binding. Positional parameter. Shorthand property. Aliased property. Rest element. Catch parameter. Array binding. Rest elements rename in place: ...foo becomes ..._foo. Each shape has its own rewrite. No regex touches source code any more.

3. Typed shorthand rename breaks property binding

Projects: buildwithkenny, joiner-landing-page.

type Props = { durationInFrames: number }

const Foo: FC<Props> = ({ durationInFrames }) => {}

Renaming the shorthand property renamed both the property key and the local binding:

const Foo: FC<Props> = ({ _durationInFrames }) => {}

// TS2339: Property '_durationInFrames' does not exist on type 'Props'.

What we did. Convert shorthand into aliased form instead of renaming. { foo } becomes { foo: _foo }. The property key foo still matches the type. The local binding gets the underscore prefix. Unused-var rule is satisfied.

4. `react-hooks/exhaustive-deps` autofix causes TDZ

Project: joiner-landing-page.

useEffect(() => { if (script) loadScript(script); }, [searchParams]);

const loadScript = (s: string) => { ... };

oxlint appended loadScript to the dep array without checking hoisting order. The result failed to compile. TS2448. "Block-scoped variable 'loadScript' used before its declaration."

What we did. Disabled the autofix for react-hooks/exhaustive-deps in aislop's oxlint config. The warning still fires. Developers still get told about the missing dependency. We just don't apply a fix that assumes a human reviewer is about to reorder declarations.

All four gaps closed. The common thread. External auto fixers assume a human will review the diff and catch what they got wrong. Run them unattended, which is exactly what aislop fix does, and you ship broken TypeScript.

The moment the tool beat the tool maker

Somewhere in the middle of all this, we added a new rule: ai-slop/narrative-comment. It catches decorative separators, phase headers, multi line preambles, cross reference commentary, and JSDoc paragraphs that summarize what a function does.

Then we ran aislop against its own source. Two fix passes removed 82 comments. They were all ours. Put there by an AI agent working with one of us during the rehaul. The agent writes prose. The tool removes prose. The commit is clean. We have a feedback loop that works even when we're not paying attention.

The honest version. We wrote a linter for AI slop. Then we used AI to build features. Then we caught ourselves producing the exact slop the linter was designed to flag. If the tool builder needs the rule, you probably do too.

What we learned about AI-generated code

AI agents produce distinct comment patterns. Decorative section banners. Phase numbered preambles. JSDoc that narrates implementation. Comments that cross reference other functions by name. Humans rarely write these. They take too long and add no information. Agents write them because training data rewards visible effort.

External fixers are designed for humans, not pipelines. oxlint --fix and knip --fix both assume a reviewer will look at the diff. Wire them into an unattended pipeline, which is where aislop lives, and their "fixes" corrupt the exact destructure patterns React and TypeScript code is full of. Three of our four GAPs came from this assumption.

Regex based source mutation is a bug factory. aislop had a regex rename helper from 0.4. It worked on 90% of cases. The 10% it broke were real user code. Moving to AST based rewrites was the direct response. Every transformation now runs through the TypeScript compiler, classifies the binding shape, and parse verifies the output before writing. Revert on any failure. Always.

The broader move with 0.5. We stopped delegating destructive operations to external tools. Detection we'll happily take from anywhere. But when something mutates user code, we want to own the rewrite, know the edge cases, and guard the file with a parse check.

Where aislop goes next

0.5 is tagged. Next up. A formal release to npm. Hydrogen style recipes for common stacks like Next, Expo, and Remotion. And the scanaislop GitHub app so teams can enforce a score threshold on every PR without touching a workflow file.

Try it on your repo

$ npx aislop scan

$ npx aislop fix -f

Run it on your own code. If it breaks anything, open an issue. We'll pin your project to the next validation matrix.

Star the AI Slop CLI on GitHub

if you want releases to show up in your feed.

← All posts