feat: rebalanced tier-weighted scoring and structured audit trail (0.2.0)#5
Merged
Merged
Conversation
…rror-tier cap Notice-tier checks now carry weight 1 instead of being excluded, so adopting emerging standards is what lifts a site from B into A. TIER_WEIGHT is error 4, warning 2, notice 1; grade bands are extracted to a named table (A>=90, B>=72, C>=50, D>=30, else F). Any non-inferred error-tier failure caps the grade at C so a site that cannot serve a load-bearing signal never reads as A or B. Under the old error-tier-only model a Cloudflare-shaped site (every established signal, zero emerging ones) scored A/100; it now lands at C/59.
…issue) Every CheckResult now carries additive audit-trail fields alongside the evidence summary that stays: goal (what good looks like) and resources (authoritative RFC/spec links) copied from the check meta, a one-line issue on a verified failure, and a transcript of the HTTP steps the check made. followRedirects records each hop as an HttpStep (method, absolute URL, status or null on a network error/timeout, one-line detail); fetchRaw/fetchText take an optional steps collector so the ~14 callers stay a minimal diff. The shared robots.txt prefetch records its steps once and hands them to robots-txt, ai-bot-rules and content-signals via ctx. mpp makes no probe and carries no transcript. New HttpStep/CheckResource types are exported from the package entry point.
Bump agentradar and agentradar-core to 0.2.0, add a CHANGELOG covering 0.1.0, 0.1.1, and 0.2.0 (flagged as a data-shape change for consumers), and correct the stale README sentences that still described error-tier-only scoring.
…pt, copy shared robots transcript, precise CHANGELOG breaking note Review findings F-1, F-2, F-5: dns-aid no longer overwrites the network-error detail on null-status steps; robots-derived checks each get their own copy of the shared prefetch transcript instead of aliasing one array; CHANGELOG now names the required CheckMeta.goal field as breaking for custom check authors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rebalances the composite score so the grade finally differentiates sites, and adds a structured audit trail to every check.
Before: the deployed 0.1.0 scored only the single error-tier check, so any site passing markdown negotiation got A/100 (cloudflare.com: A/100 with 11 failing checks). Now: tier-weighted ratio (error 4x, warning 2x, notice 1x), bands A>=90 / B>=72 / C>=50 / D>=30, and a failing error-tier check caps the grade at C.
Live calibration under the new model: cloudflare.com C/59, codixus.com C/52, vercel.com D/30. A perfect site is 100/A; an A now requires real emerging-standard adoption.
Audit trail
Every
CheckResultgains optionaltranscript(per-hop HTTP steps recorded centrally infollowRedirects, bounded by MAX_REDIRECTS),goal,issue, andresources(spec links).evidencestays the one-line string summary, so the CLI human output and existing JSON consumers keep working. New types (HttpStep,CheckResource) are exported.CheckMeta.goalis required, which is breaking only for external custom-check authors (noted in the new CHANGELOG).Verification
Plan: ~/.claude/plans/agentradar-scoring-audit-trail.md (approved 2026-07-04). Both packages bumped to 0.2.0; publish is manual after merge.