The AI coding agent code-quality benchmark.
Same tasks across every major agent, scored deterministically by aislop. No LLM in the scoring loop — so the leaderboard is reproducible and auditable, unlike benchmarks where one model grades another.
What it measures
Not whether the code runs — whether it’s clean enough to inherit. The patterns a maintainer would flag in review, scored the same way for every agent.
AI-slop density
Named anti-patterns per task: narrative comments, swallowed exceptions, unsafe casts, todo stubs, generic naming.
Structural quality
Function size, complexity, unreachable code, and dead code in the generated solution.
Security constructs
Risky primitives and unsafe patterns introduced while satisfying the prompt.
Composite score
A single 0–100 aislop score per agent, per task — same deterministic engines teams run locally.
Method
- 01Pinned tasks, published in full.
- 02Pinned agent versions and models.
- 03Clean-room runs — no shared state.
- 04Deterministic aislop scoring — no LLM judge.
- 05Raw output published so anyone can re-score.
LLM-judged — opaque, non-reproducible. aislop — deterministic, published raw output.
At launch
- Claude Code
- Cursor
- Codex
- Gemini
- Aider
- OpenCode
Nominate a task or get notified.
Suggest a task that exposes how agents really write code, or follow research for the first published run.