-
Notifications
You must be signed in to change notification settings - Fork 686
Blog Review Rubric
How we review Astryx docsite blog posts (apps/docsite/src/content/blog/posts/<slug>.md). Blog posts are human-authored prose (see the blog folder README); this rubric protects that bar. It is advisory — a review produces a scorecard and specific, quotable findings with concrete fixes. It never auto-blocks merge and never rewrites the author's voice.
The same rubric backs three surfaces so a post gets the same read everywhere: this page (canonical), the Copilot reviewer at .github/instructions/blog.instructions.md (auto-applies to blog-post PRs), and the blog-review reviewer skill for local/interactive checks.
Do not detect "AI vs human." LLMs are unreliable at authorship detection, and "this feels AI-written" is neither verifiable nor actionable. Judge observable writing quality — generic, low-effort, unsupported, repetitive, off-voice — and give the author something concrete to fix. Never output a human-vs-AI verdict.
Read it end to end before judging. Then read the frontmatter type — it sets the bar. A changelog digest and a philosophy essay are good in completely different ways; do not judge one by the other's standard. Match the post to a profile:
type |
Profile | What "good" looks like | Do NOT penalize for |
|---|---|---|---|
update |
Changelog digest | Scannable; accurate command/version list; each change stated once with its user-facing effect. Brevity is a feature. | Lacking narrative, story, or deep technical journey. |
engineering |
Technical deep-dive | Specifics: real numbers, named systems, code, the journey (what was tried, what broke, trade-offs). | Being long if the length carries detail. |
design |
Design rationale | Clear reasoning, the why behind decisions, trade-offs, principle → application. | Fewer hard numbers than an engineering post. |
guide |
How-to | Correct, complete, actionable steps a reader can follow and succeed. | Being instructional/dry rather than narrative. |
perspective / story
|
Opinion / narrative | A real point of view, authentic specifics, community-forward warmth. Personal and loose is fine. | Lacking benchmarks, code, or command tables. |
If type and content mismatch (a post tagged engineering that's actually a philosophy piece), note it — the tag or the content should change.
In your own words, write (a) why this post exists — the job it does for the reader — and (b) the 3–5 key takeaways. Derive both from the content, not the title. This is your ground truth for scoring. A post that can't answer "why does this exist?" has the deepest problem a reviewer can surface (a gratitude/PR post with no point beyond "thanks" is the classic case).
Cite specific lines/quotes for every deduction. Two things make the score diverge by type: an accuracy gate that can cap the grade, and per-type category weights.
A blog post ships to the world; a wrong command or invented API misleads real readers, so correctness is a floor, not a nice-to-have. The reviewer verifies checkable claims against the current branch (grep the CLI/source) — this is where the reviewer is strongest.
- One confirmed verifiable error — a stale/removed command, a nonexistent component/prop/flag, a wrong version, a dead link, or a specific number stated as fact with no support — caps the grade at C. Mark the post ⛔ blocking: do not publish until corrected and lead the review with it.
- Multiple such errors, or one that would seriously mislead — cap at D.
- A confirmed error means you checked it against the current branch or a cited source. If a claim simply can't be verified from the repo, it does not trigger the gate — list it as "needs author/maintainer confirmation" instead.
The gate is why "clean prose, broken command" can't score well: fix the facts first, then the writing is judged.
Read the type, pick the column, score each category out of its weight. This is what makes a changelog and a deep-dive genuinely different reviews — not just different advice, different math.
| Category | update |
engineering |
design |
guide |
perspective / story
|
|---|---|---|---|---|---|
| Substance & purpose | 20 | 30 | 30 | 25 | 30 |
| Accuracy | 30 | 25 | 20 | 30 | 10 |
| Information efficiency | 25 | 15 | 15 | 20 | 15 |
| Craft | 15 | 20 | 20 | 15 | 20 |
| Voice & fit | 10 | 10 | 15 | 10 | 25 |
Rationale: a changelog lives or dies on accuracy + scannability (readers copy commands from it) and needs little voice. An engineering / design post is mostly substance — the journey, the reasoning, the trade-offs. A perspective post is mostly voice + substance and carries few verifiable claims, so accuracy is light. A guide must be accurate and complete enough to follow.
Grade after the gate: A 90+, B 75+, C 60+, D 40+, F <40.
1. Substance & purpose. Does the post have a clear reason to exist and deliver on it? Is each takeaway earned the way its type demands — engineering/design want specifics, mechanism, the journey, trade-offs (assertion without evidence is the failure); an update wants the change and its user-facing value stated clearly; a perspective wants a genuine point of view, not generic sentiment. Does the title/description match what the body delivers? Flag over-claiming, and flag the inverse (the best material buried).
2. Accuracy. The gate above is the teeth; this category's points reflect how much verifiable surface the post got right. Verify commands, component names, props, and CLI flags against the current branch; check figures against an in-repo source where one exists; confirm links resolve and code samples are correct. Only score what's verifiable — don't penalize honest subjective statements, and mark unverifiable claims "needs author/maintainer confirmation."
3. Information efficiency. Flag repetition (the same fact in intro + middle + conclusion is a structural problem — name where it belongs, don't just call it wordy), padding (throat-clearing intros, the "sounds nice, deletes clean" sentence that survives deletion with no information loss), and dead clichés (Orwell: cut every word you can; kill metaphors "you're used to seeing in print"). Judge information per word, calibrated to the profile — a digest should be tight; a deep-dive earns length only if each section carries new detail.
4. Craft. Observable quality of the writing — not an authorship test. The failure mode (per the Field Guide to AI Slop) is surface polish with nothing underneath. Each item below is a signal, not a ban — weigh the whole texture, never dock for one instance:
- Editorial labels doing the content's work — "comprehensive," "robust," "elegant," "seamless," "powerful" as praise. Cut the label; does a fact remain?
- Hollow hedging / vagueness — "can help," "may improve," "plays a key role."
- Over-signposting — "In today's landscape," "It's worth noting that," "Let's dive in," restating a heading as the first sentence.
- Defensive framing — "this isn't a hack," "it's not X, it's Y," pre-empting an objection no one raised.
- Self-labeled significance — "Key insight:," "The headline finding:" — telling the reader it matters instead of showing it.
- Puffery / peacock terms (Wikipedia's Signs of AI Writing) — "stands as a testament to," "rich tapestry," "plays a vital role," flattery that adds no fact.
- False range — "from startups to enterprises," "from X to Y" that sound complete but bound nothing real.
- Vague authority — "industry reports suggest," "studies show," "many developers agree" with no named source.
Clarity & structure lens (Gopen & Swan). Beyond word-tells, judge sentence structure: old-before-new — a sentence should open with familiar/context info and land the new payload in the stress position at the end (burying the new point mid-sentence, or opening with it cold, is the most common readability drag; matters most for update digests — lead with the change, land the user-facing effect at the end); and topic position — the start of a sentence/paragraph should name what it's about.
Antidote signals (raise the score): concrete numbers, named people/tools, real anecdotes, admitted trade-offs or failures, a specific opinion, an unexpected structure, dry humor.
Signal, not ban. A strong writer uses em-dashes, rules-of-three, and the occasional "seamless." Never dock for one token. Wikipedia's own guide says it plainly: this is not a ban on any word or on em-dashes — the models learned from human writing, so no single token proves anything. What you're scoring is polish-with-nothing-underneath, described as a writing-quality issue with a concrete fix — never as "this is AI."
5. Voice & fit. A light check — do not homogenize the author's voice. Aim for community-forward (warm, direct, talks with readers not at them, credits people, invites participation) while preserving each author's register. Astryx voices range from punchy-staccato to warm-reflective — all valid.
- Read-it-aloud test. If a passage sounds like a marketing page or a school essay, flag it. If it sounds like how the author would explain this to a coworker, it's right. This catches most voice problems fast.
- Community-forward where it fits (especially
perspective/story/update), or does it read as corporate broadcast? - Relax on clearly human, distinctive posts. A strong idiosyncratic voice outranks the rubric — reserve notes for genuinely off moments (a personal post that lurches into press-release register, jarring shifts).
Give the author a mirror. These two notes are not graded and never change the score — they just report, honestly and briefly, how the piece is likely to land. Keep them descriptive, not prescriptive.
- How a reader might feel — the dominant felt experience after one pass (energized, informed, warmly included, impressed, confused, skeptical, sold-to, bored), from the perspective of the post's actual audience. Candid but kind.
- What the reader would actually take away — the one thing that sticks after they close the tab, stated as the reader would say it. Then compare it to the intended takeaways from step 2: if what sticks differs from what the post was trying to land, say so — that gap is the single most useful signal an author can get.
# Blog Review: <title>
**Type**: <type> (<profile>) · **Author(s)**: <handles> · **Grade**: <LETTER> (<score>/100)
<if gated:> ⛔ Blocking: do not publish until corrected — <the accuracy issue>
## Purpose & takeaways (my read)
- Why this post exists: …
- Key takeaways (intended): 1) … 2) … 3) …
## Reader reflection (unscored — a mirror for the author)
- How a reader might feel: … (one candid sentence)
- What actually sticks: … <if it differs from the intended takeaway, name the gap>
## Scorecard (Max = this post's TYPE profile)
| Category | Score | Max (this type) |
|---|---|---|
| Substance & purpose | X | <weight> |
| Accuracy | X | <weight> |
| Information efficiency | X | <weight> |
| Craft | X | <weight> |
| Voice & fit | X | <weight> |
| Total | X | 100 |
Gate applied? <no / capped at C / capped at D — and why>
## Findings
- Accuracy (report FIRST): what you checked against the repo — verified / stale / unverifiable
- Substance: does it have a point; which takeaways landed vs were asserted (quote lines)
- Efficiency: specific repetition/padding (quote the duplicated pair)
- Craft: specific tells with quotes + the concrete fix (never "this is AI")
- Voice: community-forward? read-aloud check (only flag genuinely off notes)
## Top suggestions
Concrete, prioritized fixes. Never a full rewrite — respect the author's voice.
-
The
typesets the bar. Never fault a changelog for lacking a journey, or a philosophy post for lacking benchmarks. - No authorship verdicts. Describe writing quality and give a fix; never label a post human or AI.
- Accuracy is where the reviewer is strongest — verify checkable claims against the repo; that's more valuable than any vibe read, and it gates the grade.
- A strong human voice beats a clean rubric. Suggest, don't impose. The blog is human-authored by principle; the reviewer defends substance, not conformity.
- Borrowed lenses, not new laws. The tells come from the Field Guide to AI Slop, Wikipedia's Signs of AI Writing (which itself says: not a ban on any word or em-dashes), Gopen & Swan (old-before-new, topic/stress position), and Orwell (cut every word you can; kill dead clichés). Use them to see more precisely — never to mechanically penalize a token.
- Contributing Templates — the template grading rubric (sibling reviewer)
- Design Conventions · API Conventions — the component-side review bars