feat(evidence): check citations, not URLs in code, and warn on dead external links (#245) - #248
Conversation
…xternal links (#245) A URL inside an inline code span or a fenced code block is content — a probe target unreachable by design, a verbatim command or error string — and milestone evidence no longer scans it. An unreachable github.com citation still refuses EVIDENCE_UNREACHABLE; an unreachable external one prints a warning line and the verb passes. The walk is factored into checkEvidence/reportEvidence so the classification is tested without a tracker. Adopts #222 (M12-R4). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QsJKZXNhs7rsovhb6jXCRE
…245) The SPEC row, the qa and doc-synthesizer contracts and the introduction's refusal-code list say what milestone evidence checks: URLs in prose or in Markdown links, never inside code; github.com failures refuse, external ones warn. CHANGELOG entry under Unreleased (M12-R4, #222). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QsJKZXNhs7rsovhb6jXCRE
483fa09 to
03e1fb8
Compare
|
Rebased onto main at cbf32fe (PR #247): head is now 03e1fb8. One adjustment — the CHANGELOG conflict under |
There was a problem hiding this comment.
Verdict: request changes on radiusred/gh-codecrew PR #248.
Findings:
-
roles/qa.md still says "Every link the record cites must resolve before you test against it" even though the PR intentionally lets unreachable external citations pass as warning lines. The QA contract should say the refusing condition is an unreachable github.com citation, and external warnings are weighed by QA.
-
internal/cli/cli.go help still says "verify every cited link in the milestone's record resolves". The built help prints that same old rule, but the new behavior can pass with an unreachable external citation. Update the help text to match the github.com-refusal / external-warning split.
-
internal/cli/evidence_test.go covers ordinary and indented fenced blocks, but not the explicit "fence inside a list item" shape from the review brief. Add a scanner test with a list item containing a fenced URL, preferably with a language-tagged fence, so this requirement is locked down directly.
Verification:
- Reviewer identity minted as radiusred-checky; config App ID 4719924 matched
gh api /apps/radiusred-checky --jq .id; PR author is radiusred-cody[bot]. - Read reviewer, implementer, and contributing contracts.
- Inspected diff before the PR description.
- Read task #245, milestone #207 as requested, milestone #241 for M12-R4, and capture #222.
gofmt -l .: clean.go test ./...: passed.go vet ./...: passed.go build -o ./gh-codecrew ./cmd/codecrew: passed../gh-codecrew milestone evidence 12: passed against the live hub../gh-codecrew milestone evidence 11: refusedNOT_FOUNDbecause M11 issue #233 is closed and the verb only searches open milestones.
…ist items (#245) Checky's three findings on PR #248: the qa contract names the refusing condition (an unreachable github.com citation; external warnings are QA's to weigh); the CLI help for milestone evidence describes the same split instead of "every cited link"; and a fenced block that opens inside a list item — indented under the bullet or on the marker's own line, with a language tag — is recognised as code, with fixtures for both shapes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QsJKZXNhs7rsovhb6jXCRE
|
Addressed all three findings in c5b702e (one follow-up commit on the same base, main at cbf32fe):
gofmt, vet and |
There was a problem hiding this comment.
Round two approved. The three requested changes are addressed: QA wording now names github.com citations as the refusing condition, built help matches the github.com-refusal/external-warning split, and the extractor test now covers list-item fences including marker-line fences that fail without stripping the list marker. Follow-up diff from 03e1fb8 is scoped to the expected four files; CI and local gofmt/test/vet/build are green.
Closes #245
Closes #222
Summary
milestone evidencechecks the links a record cites, not every URL in it (M12-R4, adopts #222).[text](url)outside code. A URL inside an inline code span (any backtick-run length, CommonMark matching) or a fenced code block (``` or ~~~) is content — a probe target that is NXDOMAIN by design, a verbatim command or error string — and is not scanned.extractURLsstrips code first (`stripCode`, `fenceOpener`, `stripSpans`).github.comlink that does not resolve is stillrefused[EVIDENCE_UNREACHABLE]. Any other host printswarning: external link does not resolve — <url> (<err>) — cited on <ref>and the verb passes; the summary line counts both (3 of 4 cited links resolve … the 1 external link above does not, and QA weighs it — evidence is reachable). The refusal's numerator counts github.com failures only; the denominator is every citation.checkEvidence(records) evidenceReportandreportEvidence(w, report, issues)so the classification is tested without a tracker;milestoneEvidenceonly gathers the records.Tests
TestExtractURLsSkipsCode(inline span, double-backtick span holding a backtick, backtick and tilde fences, indented fence, longer/shorter closers, unclosed fence, unclosed span is literal, Markdown link outside code);TestIsGitHubLink;TestCheckEvidenceClassifiesCitations— a code-span URL and a fenced-block URL are never checked or counted, a plain external link that 404s against a realhttptestserver is a warning andreportEvidencereturns nil, a github.com link that 404s (stubbedcheckAPI) is the refusal with1 of 4 cited links. The #138 case that read a backtick-wrapped URL as a citation flips: an unclosed backtick still terminates a URL; a closed span is not a citation.go test ./...,go vet,gofmtclean.Docs
SPEC's
milestone evidencerow;roles/qa.md(first act: what a citation is, how to weigh awarning:line);roles/doc-synthesizer.md(new obligation: a link is a citation, code is content — put a non-evidence URL in code rather than editing a comment to hide it from the scanner);docs/introduction.md'sEVIDENCE_UNREACHABLEbullet; CHANGELOG under[Unreleased].Decisions
github.comhost, sodocs.github.com,raw.githubusercontent.comand codecrew.works are external (warn, not refuse);warning:rather than the CLI'snote:prefix because the line is a finding for QA to weigh.No deviations from the plan.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QsJKZXNhs7rsovhb6jXCRE