docs(artifacts): file 14 oracle-verified bug-hunt findings (REQ-110..123) + REQ-109 decision#344
Merged
Merged
Conversation
…123) + record REQ-109 decision Outcome of a 5-persona oracle-style bug hunt (cross-command report consistency, git/remote semantics, evidence-correctness, path-URL leakage, F2 silent-failure), each candidate adversarially verified through 3 distinct lenses (correctness / user-meaning / code-archaeology) and confirmed only on >=2-of-3 majority. 20 candidates → 14 confirmed, 6 refuted (the refuted set includes a stale "HTML export absolute links" finding correctly killed because REQ-105 just fixed it — evidence the verification gate works). Filed REQ-110..123, dogfooded as rivet artifacts with falsifiable acceptance steps. Clusters: - Coverage counting (REQ-110/111): the coverage HTML card + JSON `total` sum PER-RULE denominators, double-counting artifacts that satisfy multiple rules, under a label that says "artifacts covered" — the same class as the earlier 768/929 observation, generalized, and semantically clashing with stats' distinct `total`. HIGH. - Git/remote semantics (REQ-112/113/114): cmd_commits excludes externals while validate includes them; resolve_external_dir returns an unsynced cache path; sync local/remote state reported inconsistently. - Path/URL leakage beyond REQ-105 (REQ-115/116/117/118): zola export emits absolute artifact links (2 sites), HTML export embeds a hardcoded localhost oEmbed tag, document markdown emits absolute paths. - F2 silent-failure (REQ-119..123): ReqIF enum/dir-import/field fallbacks swallow drops; external git-checkout + symlink-removal failures silently ignored. Also records the REQ-109 (variant document scoping) DESIGN DECISION: yes, via an optional `documents:` list on the existing binding (no new directive), default-in for unbound docs, validate always sees the full set. Implementation is a tracked follow-up. Refs: REQ-004, REQ-083
📐 Rivet artifact delta
Graphgraph LR
REQ_109["REQ-109"]:::modified
REQ_110["REQ-110"]:::added
REQ_111["REQ-111"]:::added
REQ_112["REQ-112"]:::added
REQ_113["REQ-113"]:::added
REQ_114["REQ-114"]:::added
REQ_115["REQ-115"]:::added
REQ_116["REQ-116"]:::added
REQ_117["REQ-117"]:::added
REQ_118["REQ-118"]:::added
REQ_119["REQ-119"]:::added
REQ_120["REQ-120"]:::added
REQ_121["REQ-121"]:::added
REQ_122["REQ-122"]:::added
REQ_123["REQ-123"]:::added
classDef added fill:#d4edda,stroke:#28a745,color:#155724
classDef removed fill:#f8d7da,stroke:#dc3545,color:#721c24
classDef modified fill:#fff3cd,stroke:#ffc107,color:#856404
classDef overflow fill:#e2e3e5,stroke:#6c757d,color:#495057,stroke-dasharray: 3 3
Added
Modified
Posted by |
…-reqs # Conflicts: # artifacts/requirements.yaml
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.
| Benchmark suite | Current: 5a553e5 | Previous: 6c66af5 | Ratio |
|---|---|---|---|
validate/10000 |
18842253 ns/iter (± 3686650) |
15014467 ns/iter (± 542013) |
1.25 |
This comment was automatically generated by workflow using github-action-benchmark.
The bug-hunt REQ-110..123 blocks carried 56 trailing-space errors that failed the YAML Lint CI gate (error-level, not warning). Pure whitespace cleanup — no YAML-semantic change; `rivet validate` still PASS. Trace: skip
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Outcome of a 5-persona oracle-style bug hunt (cross-command report consistency, git/remote semantics, evidence-correctness, path-URL leakage, F2 silent-failure). Each candidate was adversarially verified through 3 distinct lenses (correctness / user-meaning / code-archaeology), confirmed only on ≥2-of-3 majority.
20 candidates → 14 confirmed, 6 refuted. (The refuted set includes a stale "HTML export absolute links" finding correctly killed because REQ-105 just fixed it — the verification gate works.)
Filed as REQ-110..123, dogfooded as rivet artifacts with falsifiable acceptance steps.
Confirmed clusters
totalsum per-rule denominators (double-count artifacts in multiple rules) under a label saying "artifacts covered"; clashes with stats' distincttotal. The 768/929 class, generalized.commitsexcludes externals whilevalidateincludes them;resolve_external_dirreturns an unsynced cache path; sync local/remote state reported inconsistently.localhostoEmbed tag; document markdown emits absolute paths.Also
Records the REQ-109 (variant document scoping) design decision: yes, via an optional
documents:list on the existing binding (no new directive), default-in for unbound docs,validatealways sees the full set. Implementation tracked as follow-up.Verification
rivet validate— PASS, no duplicate IDs.rivet docs check— PASS.Next
These are filed for prioritization. The coverage-counting cluster (110/111) is the highest-value + lowest-risk fix (it's the exact "929" class) and a good first target.
🤖 Generated with Claude Code