docs: add benchmark proof and distribution assets#42
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (2)
📝 WalkthroughWalkthroughAdds benchmark proof artifacts and a styled benchmarks landing, marketplace submission docs, and a GitHub Pages workflow; introduces sanitization (hashing) of path-derived identifiers in PR-impact payloads and updates tests to verify sanitized prompts and benchmark artifacts. ChangesBenchmark Proofs & Marketplace Distribution
PR-Impact Payload Sanitization
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Review rate limit: 9/10 reviews remaining, refill in 6 minutes. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/pages.yml:
- Around line 27-40: Replace floating action tags with immutable commit SHAs for
each `uses:` entry in the workflow: change `actions/checkout@v6`,
`actions/configure-pages@v5`, `actions/upload-pages-artifact@v4`, and
`actions/deploy-pages@v4` to their corresponding pinned commit SHAs (e.g.,
`actions/checkout@<sha>`). Locate these `uses:` lines in the
`.github/workflows/pages.yml` snippet and replace the tag syntax with the full
commit SHA fetched from the action's GitHub repo (ensure you pin the exact
commit you tested), committing the updated workflow.
In `@docs/benchmarks/2026-05-01-govalidate-pr-review/compact-prompt.txt`:
- Around line 395-403: The benchmark payloads currently commit
workstation/username-derived node_id values (e.g., path fragments like
users_mohammednaji_...) so update the serialization in mergeRunSections() to
sanitize node_id before persisting: detect node_id values that contain
filesystem/username patterns and replace them with a stable, non-PII identifier
(e.g., a deterministic hash or slug derived from the original id) and use that
sanitized id in the committed payload; apply the same sanitization logic to the
other payload assembly sites referenced around the same area (the blocks that
produce node_id entries) so all emitted node_id fields are PII-free and stable
across runs.
In `@docs/benchmarks/styles.css`:
- Around line 154-157: The code selector's font-family list contains a
duplicated and quoted token ("SFMono-Regular" and SFMono-Regular) which causes
stylelint errors; edit the code rule in styles.css (selector: code) to remove
the duplicate and the unnecessary quotes so the font-stack reads SFMono-Regular,
ui-monospace, Menlo, Monaco, Consolas, monospace (i.e., keep one SFMono-Regular
entry without quotes).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a286f439-6fb2-428e-8dd0-4ac2d335af3b
📒 Files selected for processing (17)
.github/workflows/pages.ymlREADME.mddocs/benchmarks/2026-04-30-govalidate/index.htmldocs/benchmarks/2026-05-01-govalidate-pr-review/README.mddocs/benchmarks/2026-05-01-govalidate-pr-review/compact-answer.txtdocs/benchmarks/2026-05-01-govalidate-pr-review/compact-prompt.txtdocs/benchmarks/2026-05-01-govalidate-pr-review/index.htmldocs/benchmarks/2026-05-01-govalidate-pr-review/report.jsondocs/benchmarks/2026-05-01-govalidate-pr-review/verbose-answer.txtdocs/benchmarks/2026-05-01-govalidate-pr-review/verbose-prompt.txtdocs/benchmarks/2026-05-01-govalidate-pr-review/verify.shdocs/benchmarks/index.htmldocs/benchmarks/styles.cssdocs/distribution/marketplaces/README.mddocs/distribution/marketplaces/awesome-mcp.mddocs/distribution/marketplaces/smithery.jsondocs/distribution/marketplaces/smithery.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/infrastructure/review-compare.ts`:
- Around line 86-87: The sanitization allowlist REVIEW_PROMPT_ID_FIELDS is
missing ID-bearing "node" fields so add "node" to the set (and any other
plural/array variants used for impact items) so path-derived identifiers like
per_node_impact[].node get replaced; make the identical update to the other
allowlist instance around PATH_DERIVED_ID_TOKENS usage (the second set defined
later) to ensure both REVIEW_PROMPT_ID_FIELDS and the later token/field set
include "node" (and related per_node_impact key) so these values are sanitized
before persisting prompts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9d714646-5b17-44e4-8975-50cbd0df2872
📒 Files selected for processing (16)
README.mddocs/benchmarks/2026-05-02-govalidate-pr-review/README.mddocs/benchmarks/2026-05-02-govalidate-pr-review/compact-answer.txtdocs/benchmarks/2026-05-02-govalidate-pr-review/compact-prompt.txtdocs/benchmarks/2026-05-02-govalidate-pr-review/index.htmldocs/benchmarks/2026-05-02-govalidate-pr-review/report.jsondocs/benchmarks/2026-05-02-govalidate-pr-review/verbose-answer.txtdocs/benchmarks/2026-05-02-govalidate-pr-review/verbose-prompt.txtdocs/benchmarks/2026-05-02-govalidate-pr-review/verify.shdocs/benchmarks/index.htmldocs/distribution/marketplaces/README.mddocs/distribution/marketplaces/smithery.jsondocs/distribution/marketplaces/smithery.mdsrc/infrastructure/review-compare.tstests/unit/benchmark-artifact.test.tstests/unit/review-compare.test.ts
✅ Files skipped from review due to trivial changes (10)
- docs/benchmarks/index.html
- docs/benchmarks/2026-05-02-govalidate-pr-review/index.html
- docs/benchmarks/2026-05-02-govalidate-pr-review/README.md
- docs/distribution/marketplaces/smithery.md
- docs/benchmarks/2026-05-02-govalidate-pr-review/compact-prompt.txt
- docs/distribution/marketplaces/README.md
- docs/distribution/marketplaces/smithery.json
- docs/benchmarks/2026-05-02-govalidate-pr-review/report.json
- docs/benchmarks/2026-05-02-govalidate-pr-review/compact-answer.txt
- README.md
Comment on lines
+86
to
+87
| const REVIEW_PROMPT_ID_FIELDS = new Set(['node_id', 'from_id', 'to_id']) | ||
| const PATH_DERIVED_ID_TOKENS = new Set([ |
There was a problem hiding this comment.
Sanitization allowlist misses ID-bearing node fields.
Line 86 currently limits replacement to node_id/from_id/to_id, but pr_impact also carries IDs in fields like per_node_impact[].node. Those values can remain unsanitized and leak path-derived identifiers into persisted prompts.
Proposed fix
-const REVIEW_PROMPT_ID_FIELDS = new Set(['node_id', 'from_id', 'to_id'])
+const REVIEW_PROMPT_ID_FIELDS = new Set(['node_id', 'from_id', 'to_id', 'node'])
function sanitizePersistedReviewPayload<T>(value: T): T {
@@
- if (
- REVIEW_PROMPT_ID_FIELDS.has(key) &&
+ if (
+ REVIEW_PROMPT_ID_FIELDS.has(key) &&
typeof entryValue === 'string' &&
entryValue.length > 0 &&
isPathDerivedIdentifier(entryValue)
) {
return [key, sanitizePersistedIdentifier(entryValue)]
}Also applies to: 263-271
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/infrastructure/review-compare.ts` around lines 86 - 87, The sanitization
allowlist REVIEW_PROMPT_ID_FIELDS is missing ID-bearing "node" fields so add
"node" to the set (and any other plural/array variants used for impact items) so
path-derived identifiers like per_node_impact[].node get replaced; make the
identical update to the other allowlist instance around PATH_DERIVED_ID_TOKENS
usage (the second set defined later) to ensure both REVIEW_PROMPT_ID_FIELDS and
the later token/field set include "node" (and related per_node_impact key) so
these values are sanitized before persisting prompts.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/benchmarks/via GitHub Pages and link them from the READMETest Plan
Summary by CodeRabbit
New Features
Documentation
Tests