Home

Astryx review loop

An agent that drafts PR reviews for facebook/astryx, checks its own draft against a rules file, and hands a human something to post. It never posts by itself.

Status: SHADOW MODE. Every review it writes is read by a human before it goes anywhere. It has drafted ~22 real reviews and is calibrated to one maintainer's taste. Treat its output as a draft to judge, not a review to trust. Posted reviews carry [Reviewed by Robohands] as their last line.

This wiki is the source of truth. Edit it — corrections belong here rather than in someone's local copy.

The pages

Page	What it is
Reviewer Brief	the reviewer's job — framing, what to look for, what to return
Critic Rules	R1–R15. Where corrections accumulate
Evidence Templates	layout / API / behavior evidence formats
Loop Mechanics	roles, passes, escalation

Run it

Save the four pages as local .md files, then:

export KIT=~/astryx-review-loop        # where you saved them
export ASTRYX_MAIN=~/astryx            # any checkout; the loop only reads it

Spawn an agent with:

Read $KIT/Reviewer-Brief.md and $KIT/Critic-Rules.md in full and follow them. Then review facebook/astryx PR #NNNN. Do the loop: draft → critique yourself against R1–R15 → rewrite → critique the rewrite. Read-only: post nothing, push nothing. Return the output block the brief specifies.

One PR per agent. That is the whole interface — no install, no dependencies beyond gh and whatever the repo already needs.

The three ideas worth stealing even if you hate the rest

1. Order of consequence. Four levels; the review spends itself on the highest unsettled one.

1. Is this the right thing to do at all?
2. Is this the right way — shape, where behavior lives, what surface it adds?
3. Does it actually work?
4. What else does it need — docs, tests, changeset?

When level 2 is open, level-4 findings are churn against code that may not survive. Say that instead of listing them.

2. Run the thing. Every genuinely useful bug this found came from executing code, not reading it:

• a mask engine that turned 555 into 11555, because a literal digit in the pattern round-tripped as data
• a number input that committed 1239 when you typed a then 9 — the rejected keystroke threw the caret to the end
• a token retune that made three shipped themes unreadable (contrast 8.33 → 1.40)
• an text-overflow: ellipsis that never fired once across 11 layout cases, because the parent flex child could not shrink

None were visible in the diff.

3. Hard word caps. Approve summary ≤30 words, request-changes ≤150, inlines ≤20. Drafts fail this constantly, and the fix is always to delete a finding, never to compress the prose.

What it is bad at

• Taste in what to flag. It converges on form fast and on judgment slowly. On one PR it missed the finding a maintainer caught in ten seconds.
• It over-blocks. Left alone it turns "would be nice" into "must", demands refactors before merge, and leads with duplication nobody cares about. Half the rules in the critic exist to stop that.
• It escalates too readily. A reviewer that sends every design question to the TL is the same bottleneck with extra steps.

---

Calibrating it to your taste

Critic-Rules is calibrated to one maintainer: their verdict policy, their ~50-word review as the length anchor, their rulings on component-vs-hook and consolidation. Some of that is Astryx-wide; some is personal.

Do not hand-write your own rules. Nobody has reliable access to their own taste in prose form. Mine it from what you already wrote.

Step 1 — have your agent mine your review history

Give it this prompt verbatim.

You are calibrating a review agent to my taste. My GitHub handle is <HANDLE>. Work read-only; post nothing.

Collect. Pull every review and review comment I have written on facebook/astryx:

gh api repos/facebook/astryx/pulls/<N>/reviews  --jq '.[]|select(.user.login=="<HANDLE>")|{state,body,submitted_at}'
gh api repos/facebook/astryx/pulls/<N>/comments --jq '.[]|select(.user.login=="<HANDLE>")|{path,line,body,created_at}'

Take the most recent 40–60 reviews; older ones reflect a system that has changed. Note each PR's author, size and area.

Measure, do not impressionize. Report actual numbers:

• median and max WORD count of my summaries by verdict, and of my inlines
• approve vs request-changes rate, split by whether the author is in .github/ENGOWNERS / .github/DESIGNOWNERS or an outside contributor
• distinct findings per review (median, max)
• how often I use a code block, a file:line pointer, a closing question
• my opening move: thank, name the hard part, or state the verdict first?
• hedged vs flat phrasing — count "ideally", "probably", "might", "hmm" against imperatives

Then find the rules, each backed by evidence. Quote 2–3 of my real comments per rule:

• What do I consistently flag that a generic reviewer would not?
• What do I consistently ignore that a generic reviewer would flag? This is the most valuable one and the hardest — look for findings plainly visible in the diff that I said nothing about.
• What makes me block versus merge-and-follow-up?
• When I disagree with a design, do I rule, prefer, or ask?
• Where do I defer to someone else instead of deciding?

Sanity-check against the diffs. For five reviews, read the PR diff and ask what I could have said and didn't. Silence is data.

Output a numbered rule list in the style of Critic-Rules, each rule ≤4 lines, each quoting me. Mark anything inferred from fewer than three examples [thin evidence]. Do not invent rules to fill gaps.

Step 2 — replace only the personal half

Keep (Astryx-wide, not personal): R1b order of consequence · R5 state the problem at the level it exists · R6 never let one instance justify a system change · R13 never charge a contributor for inherited debt · R14 confidence gate · R15 layout needs a matrix.

Replace with yours: R2/R2b length and register — your measured word counts, your own review as the anchor · R1 verdict policy · R7/R8/R9 how hard you push consolidation · R10 strictness about half-finished APIs · R12 voice.

Step 3 — verify, do not trust

The step people skip, and the only one that proves anything.

Pick 3 PRs I reviewed that are not in the set you mined. Run the loop on each without looking at my review. Then fetch mine and report, per PR: did you match my verdict · which of my findings did you get · which did you miss · what did you say that I did not · word count yours vs mine. Then name the one rule that would close the biggest gap — and change only that one.

Repeat until the misses stop being about taste and start being about the code. Three rounds took one calibration from a 185-word blocking review to a 47-word approve that matched the maintainer's.

Step 4 — keep it fed

Every correction becomes a numbered rule in Critic-Rules, quoting what was actually said. Rules accumulate in the critic; the brief stays a description of the job, not a changelog.

If several people share the loop, keep one critic file each and point your spawn prompt at yours. A merged taste file is worse than either one alone.

---

Astryx conventions the loop relies on

• Author bucket drives the verdict. .github/ENGOWNERS and .github/DESIGNOWNERS → approve-with-nits; everyone else → request-changes, because they cannot merge and the review has to carry their loop.
• The rubric is the bar. The Component Audit Rubric wiki page is the single source of truth for what counts as a defect. The agent cites check ids to itself and never puts them in a posted comment.
• The PR Analysis Report already posts bundle size, the a11y audit and preview links, so any evidence comment is gap-filler only. Screenshots are usually the real gap.
• Never review from a stale checkout. git fetch origin and read via git show origin/main:<path>.
• Screenshots publish to an assets/pr-<NNNN> orphan branch on your own fork, never on the upstream repo.

Attribution

Every posted review ends with exactly:

[Reviewed by Robohands]

Nothing else — no "generated by", no disclaimer, no model name.