Skip to content

fix: README line-by-line audit — internal consistency + accuracy#20

Merged
justi merged 2 commits into
mainfrom
fix/readme-line-audit
May 10, 2026
Merged

fix: README line-by-line audit — internal consistency + accuracy#20
justi merged 2 commits into
mainfrom
fix/readme-line-audit

Conversation

@justi
Copy link
Copy Markdown
Owner

@justi justi commented May 10, 2026

Summary

Five fixes from a critical line-by-line audit of the README — internal consistency between the sample document, the cutoff date, and the marketing punchlines.

Findings

# Severity Where Problem Fix
1 HIGH Line 30 "done arguing about Rails 7 syntax on a Rails 8 app" — sample shows cutoff 2026-01-31, by which point Claude already knows Rails 8 Reworded to match the data: "it sees the 8.1.3 patch and the CVE before it touches your code"
2 HIGH Line 27 Sample CVE bullet referenced 8.0.2.1 + CVE-2025 alongside 8.1.x deltas — confusing Replaced with a hypothetical 8.1.3 advisory + CVE-2026 stamp consistent with the post-cutoff window
3 HIGH Line 114 "Why I built this" had same Rails-7-on-Rails-8 mismatch Reworded to a realistic patch-level scenario
4 HIGH Line 133 "GitHub Discussions" — repo has Discussions disabled (gh repo view --json hasDiscussionsEnabled returns false) Replaced with "Open an issue tagged enhancement, or send a PR"
5 MEDIUM Line 128 "adapter surface is small (~4 functions)" — real contract is seven (parser + 3 local/HTTP pairs) Reworded to spell out the real shape

What I verified to ground the audit

  • ✅ Rails sub-gem count: README claims 12, code confirms exactly 12 (RUBY_RAILS_SUBGEMS in lib/adapters/ruby/local.sh)
  • ✅ Test count: README claims ~205, full suite reports exactly 205 (10+8+6+6+35+13+7+4+23+16+11+22+11+33)
  • ✅ Version: README claims v0.3.1, bin/postcut:4 confirms VERSION="0.3.1"
  • ✅ Cutoff date for claude-opus-4-7: 2026-01-31 matches model's actual training cutoff
  • ✅ All flags listed in Usage exist in bin/postcut
  • ❌ Adapter function count: README said ~4, real is 7 — fixed in this PR
  • ❌ GitHub Discussions: README pointed there, but repo has them disabled — fixed in this PR (alternative: enable Discussions on the repo and revert this part of the fix)

Stats

  • 5 insertions, 5 deletions — surgical
  • 138 lines (unchanged)
  • No code touched, no tests touched

Test plan

  • git diff --stat — only README.md, 5/5
  • Full test suite still 205/205 (unchanged)
  • Manual eyeball of each edit in context

Open question for the maintainer

If you want GitHub Discussions instead of issue tags for "ideas / use cases", enable Discussions in repo Settings → Features and revert the line 133 edit. Otherwise this fix is the safer default.

🤖 Generated with Claude Code

justi and others added 2 commits May 10, 2026 20:22
Five fixes from a critical pass over the README:

HIGH:

- Sample punchline (line 30) said "done arguing about Rails 7 syntax on a
  Rails 8 app" — but the sample shows model cutoff 2026-01-31, by which
  point claude-opus-4-7 already knows Rails 8.x. The real gap the
  example demonstrates is 8.1.2 → 8.1.3 patch + CVE landed after cutoff.
  Reworded to match the data: "it sees the 8.1.3 patch and the CVE
  before it touches your code."

- Sample CVE bullet (line 27) referenced version 8.0.2.1 alongside
  8.1.x deltas, with a CVE-2025 stamp. Confusing in isolation. Replaced
  with a hypothetical 8.1.3 advisory + CVE-2026 stamp so the bullet
  fits the post-cutoff window the rest of the example uses.

- "Why I built this" footer (line 114) had the same Rails-7-on-Rails-8
  mismatch — model with current cutoff would not suggest Rails 7
  methods on a Rails 8.1 app. Rewrote to a realistic patch-level
  scenario: "model confidently called an API the latest patch had
  quietly changed."

- Community section (line 133) pointed users to "GitHub Discussions"
  but the repo has Discussions disabled (verified via gh repo view).
  Replaced with "Open an issue tagged `enhancement`, or send a PR."
  Enabling Discussions is a separate decision left to the maintainer.

MEDIUM:

- Roadmap line (line 128) said "adapter surface is small (~4 functions)"
  but the real contract is seven: a lockfile parser plus three
  local/HTTP function pairs (metadata, notes, advisories). Underselling
  contributor work invites a "wait, this is more than you said"
  reaction once they open the codebase. Reworded to be specific.

No code changes. Tests untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second-pass codex review caught four things the first audit missed:

MEDIUM:

- Sample CVE bullet (line 27) had a fabricated GHSA-ID and CVE-ID
  (GHSA-h4wq-7r2x-9j3p / CVE-2026-12104) introduced by the previous
  fix to dodge the version-mismatch problem. A made-up but
  real-looking advisory undermines the document's factual posture
  more than the original mismatch did. Replaced with explicit
  placeholder IDs (GHSA-xxxx-xxxx-xxxx, CVE-2026-XXXXX) so the
  reader knows it is illustrative shape, not a real Rails advisory.

- GITHUB_TOKEN scope (line 109) overstated the token's reach. README
  said "rate limit on registry/advisory calls" but the token is only
  attached to GitHub API calls (releases + advisories) in
  lib/core/github.sh. RubyGems registry calls in
  lib/adapters/ruby/fetch.sh and lib/adapters/ruby/local.sh are
  unauthenticated. Reworded to match the actual scope and to match
  the CLI help text in bin/postcut.

LOW:

- Install one-liner (line 44) said "symlinks into PATH" without the
  caveat from install.sh: when no writable bin dir is found on PATH,
  the installer prints an `export PATH=...` hint instead. Added the
  conditional and the alternative.

- Sample document (lines 18-21) omitted the two blockquote guidance
  lines that lib/core/save.sh actually emits before the per-gem
  sections ("Each section below describes..." and "Use the
  post-cutoff version's API..."). Section is titled "What it
  generates", which implies exact output — added the two lines.

No code changes. Tests untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@justi
Copy link
Copy Markdown
Owner Author

justi commented May 10, 2026

Second codex pass on the README (resumed prior thread, confronted with what it missed) — codex acknowledged the prior misses with a postmortem (reading-pass failure / assumption error / scope issue / verification failure / codebase-scope miss) and surfaced four follow-up findings, all addressed in commit dea58f6:

# Severity Where Fix
1 MEDIUM Line 27 (sample CVE) Fabricated GHSA + CVE IDs from the previous fix swapped for explicit placeholders (GHSA-XXXX-XXXX-XXXX / CVE-2026-XXXXX). A made-up real-looking advisory undermines factual posture more than the original mismatch.
2 MEDIUM Line 109 (GITHUB_TOKEN) Was "rate limit on registry/advisory calls" — token actually only affects GitHub API (releases + advisories), not RubyGems registry. Reworded to match lib/core/github.sh scope and CLI help text.
3 LOW Line 44 (install) "symlinks into PATH" was unconditional — install.sh prints an export PATH=... hint when no writable bin dir found. Added the caveat.
4 LOW Lines 18-21 (sample) Sample omitted two blockquote guidance lines that lib/core/save.sh:96-97 actually emits before the per-gem sections. Added them so "What it generates" stays a faithful sample.

The pattern across both audit rounds: copy-driven inconsistencies were caught quickly; technical/factual claims required either grepping the codebase or running gh repo view. Worth keeping that gap in mind for future review prompts.

@justi justi merged commit 8fa1319 into main May 10, 2026
2 checks passed
@justi justi deleted the fix/readme-line-audit branch May 10, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant