spec: Proposal to wire Scorecard to ExecutionContext - #353
Open
justaugustus wants to merge 1 commit into
Open
Conversation
…itdevorg#194) Draft proposal (specs/021-scorecard-execution-context/proposal.md) for giving the shared ExecutionContext.get_or_run_tool infrastructure from darnitdevorg#189 its first real consumer, using OpenSSF Scorecard as the motivating heavy tool. Captures two maintainer-confirmed design decisions: - Invocation backend: uwu-tools/scorecard-mcp (MCP over stdio) behind a swappable ScorecardBackend seam, so a live-scan CLI backend can be added later without touching caching or mapping logic. - Verdict policy: evidence-only. Scorecard never changes a control's status; it only attaches evidence/hints. This matches both darnit's conservative-by-default constitution and scorecard-mcp's own framing (cached/opt-in/partial data; 'heuristic signals, not a verdict'). Also documents: - Existing latent infra (get_or_run_tool plumbing, normalize_scorecard_output, evidence-on-INCONCLUSIVE support) with file:line anchors. - Backend seam + evidence-only handler design, per-repo cache key, and candidate Scorecard-check -> OSPS-control mappings. - Mapping to darnitdevorg#194 acceptance criteria; criterion 2 (cache_key consumed by adapter dispatch) flagged as the one open item, since the live audit path is the sieve-handler path and CheckAdapter dispatch is vestigial. - Phased implementation plan, open questions, and non-goals. Refs: darnitdevorg#194 Depends-on: darnitdevorg#189 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Stephen Augustus <foo@auggie.dev>
ExecutionContext
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft proposal for #194 — giving the shared
ExecutionContext.get_or_run_toolinfrastructure from #189 its first real consumer, using OpenSSF Scorecard as the motivating heavy tool.This PR adds only the proposal doc (
specs/021-scorecard-execution-context/proposal.md) — no code yet. It records two maintainer-confirmed design decisions so we have them written down before implementation:uwu-tools/scorecard-mcp(MCP over stdio) behind a swappableScorecardBackendseam, so a live-scan CLI backend can be added later without touching caching or mapping logic.The proposal also documents the existing latent infra (with
file:lineanchors), the backend seam + handler design, candidate Scorecard-check → OSPS-control mappings, a phased implementation plan, and open questions — notably #194's acceptance criterion 2 (cache_keyconsumed by adapter dispatch), flagged because the live audit path is the sieve-handler path andCheckAdapterdispatch appears vestigial.Type of Change
Framework Changes Checklist
If this PR modifies the darnit framework (
packages/darnit/):docs/architecture/framework-design.md) if behavior changeduv run python scripts/validate_sync.py --verboseand it passesN/A — proposal doc only; no framework code changes.
Control/TOML Changes Checklist
If this PR modifies controls or TOML configuration:
N/A — no control/TOML changes in this PR.
Testing
uv run pytest tests/ -v)N/A — documentation only.
Additional Notes
Requesting review from @mlieberman85.
Two things worth deciding before this graduates to a full
spec.md/plan.md:cache_keypath. Proposal recommends amend.Refs: #194 · Depends-on: #189