Repository navigation
ConventionRuleDiffReview
title: Convention-Rule Diff Review radar_quadrant: Techniques radar_ring: Assess radar_position: inner
Code written by AI agents often passes linting and tests yet still breaks a team's conventions, for example by swallowing exceptions, duplicating tool registrations, or adding product-specific branches to shared code. Written style guides get skimmed, and asking a chat model to review a diff gives inconsistent answers. Convention-Rule Diff Review turns each convention into a testable question that a decision model scores against every change (blog post).
Each team rule is written as a named mistake, such as "the change catches an exception and returns None". Rules live in rules.yaml files next to the code they cover, each with a fix and a reference file. Every rule becomes one yes/no question put to the decision model with the diff or screenshots as state. The score is flipped so that a high number means the change follows the rule, and anything under 80% becomes a hint.
Before coding, a command lists the rules that apply to the files about to change, with no model call. During coding, lint and tests run as usual. After coding, one review call checks the whole change against all applicable rules. The agent fixes real problems or dismisses false alarms with a written reason, for at most three rounds.
Rules are tested like code. Each has labeled violating and compliant cases, and a margin measures how cleanly they separate. A rule is reworded until the margin passes. The author found that rules phrased as mistakes work better than rules phrased as virtues, and reports one rule's margin improving from 0.24 to 0.96 after rewording.
The author proposes a ladder for where a convention belongs: make the mistake impossible in code first, then use lint or tests, then a model judge, and only then a guide entry.
Code goes to a third-party provider, the provider can fail, the plumbing is real work, and human judgment is still needed for new kinds of mistakes. The author reports reviews costing cents, with the largest around $0.04, and taking about a minute.
Convention-Rule Diff Review is placed in Assess within Techniques. It extends the sensor idea in Maintainability Sensors for Coding Agents with team-specific, testable rules, and the evidence comes from the author's own team. No first-person production use has been confirmed, so the ring remains Assess.
