ScaffoldScope 0.3.1 is an immutable research-instrument snapshot.
Initial public release from the current repository. Version 0.3.1 is used
because GitHub's immutable-release protection prevents reuse of a release tag
from the deleted predecessor repository.
Added
- A readable, zero-runtime-dependency Python harness for paired coding-agent
ablations across context policies, tool surfaces, and treatment instructions. - Four built-in context treatments: no compaction, reactive summarization,
periodic summarization, and budget-aware selective retention. - Deterministic planning, hard experiment budgets, resumable trials, provider
usage provenance, and intention-to-treat reporting. - Local and network-disabled Docker execution backends with recorded runtime,
task-source, implementation, plugin, and image provenance. - SWE-bench import, full-matrix export, and immutable official-evaluation
overlays without rewriting raw generation results. - Integrity-checked traces, patches, reports, and deterministic evidence bundles
with redaction-aware context commitments. - A starter project, a zero-cost scripted demonstration, a versioned plugin API,
operator documentation, and a statically rendered Next.js project website.
Analysis contract
- Summary schema version 2 separates infrastructure-valid generation accounting
from evaluator-valid outcome analysis, so pending external evaluations retain
their token, cost, latency, and context-exposure records. - Inferential labels require a preregistered primary contrast, at least 20
independent tasks, and at least 98% pair coverage. Scripted runs and panels
below 10 tasks do not receive intervals.
Known limits
- ScaffoldScope 0.3 is an alpha research instrument, not a security boundary or
a guarantee that a benchmark is uncontaminated. - The built-in scripted demonstration checks the workflow; it is not evidence of
model capability. - Imported SWE-bench generations remain outcome-pending until official evaluator
results are ingested.
Read the documentation, verify downloads with SHA256SUMS, and report vulnerabilities through private vulnerability reporting.