Repository navigation
Releases: NewMe505/SCIO
Releases · NewMe505/SCIO
Release list
SCIO v0.9.0
Added
scio dashboard [--root DIR ...] [--port 8765](issue #68, ADR 0017): a live, read-only web view of every SCIO thread across workspaces, on 127.0.0.1 only. A threads list with the stop waiting for you and crashed threads; the build view (the milestone graph with the current step lit, feature runs with tier, status, iterations, escalations, cost and links to their threads, integration lines, the PR, the timeline, cost by role and by feature); the feature and bootstrap views. It reads the checkpoint DB withmode=roconnections, the cost ledgers andprogress.json, reuses thescio reportbuilders, and pushes changes over a WebSocket. Plain HTML/JS with no build step; the optionaldashboardextra (FastAPI, uvicorn) installs it.docs/OBSERVABILITY.md: the ways to watch SCIO (GitHub, an OpenTelemetry trace viewer withSCIO_OTEL_EXPORTER=otlp, the dashboard, the CLI views).scio report --thread new-...reports ascio newthread (phase, product, revision rounds, bootstrap commit, cost by role, timeline) instead of crashing (I-108).- Always/Never guardrails with enforced guards (issue #61):
## Guardrails (Always / Never)at the top ofdocs/LESSONS_LEARNED.md, each rule citing its ledger id and naming its guard;check_guardrailsinscripts/check_docs.pyfails when a resolved ledger entry from I-104 on is cited by no rule or a named guard (symbol, test, check, hook) does not exist. - A push-gate Claude Code hook (
scripts/hooks/push_gate.py, wired in.claude/settings.json): agit pushwaits while the branch's latest CI run is red or still running (SCIO_ACK_CI=<run id>pushes a fix on purpose), and a baregit stashis refused (issue #61, I-104); agit commitwaits until the changed Python files passruff checkandruff format --check(I-106). - CLAUDE.md and CONTRIBUTING protocols 4 and 5: Always/Never rules each backed by a guard, and a bottleneck audit after every pass (issue #61).
Fixed
- On the Claude Code worker backend a feature's
pythonis the product's environment, with its approved dependencies, not SCIO's own: the worker turn's PATH starts with the product interpreter's directory (I-114). - The advisor-cap halt names the cap that fired: the feature's total, or the one call per step (I-115).
skipat a halted feature defers it to the next milestone instead of dropping it, and the release refuses while a feature inPRODUCT.mdis not done; removing a feature fromPRODUCT.mddrops it (I-116, the owner's decision).scio reporton a thread whose run is still going no longer dies on the cost ledger's half-written last line:CostLedger.load_entriesreads only complete lines (a complete line that does not parse still raises), and a worker transcript caught mid-write is read with its broken character replaced (I-127).scio build --ghno longer fails at its first issue on a repository without theproduct-modelabel: the GitHub client creates a missing label before it names it (an existing one is left as it is), and a GitHub refusal stopsscio buildwith the error and how to resume instead of a traceback (I-113).- A hand edit of
PRODUCT.mdis no longer lost: the milestone's sync commit records it, a fresh feature attempt commits one made while the build waited, SCIO's own checkouts and resets carry it, and the next milestone of the same thread re-reads it (I-109). - A quality bar can no longer promise tools the stack's profile does not use: each profile names its fixed tools, the interviewer and the planner see them, and the normaliser replaces a contradicting line with the profile's fact, quoting the original (I-107).
- A thread's recorded spend spans every CLI invocation: a resume carries the checkpoint's spend into the new process, so
scio newand resumedscio buildthreads no longer report only their latest step (I-110).
SCIO v0.8.0
Added
scio upgrade --product DIR(issue #58): brings a product bootstrapped by an older SCIO up to the current product bar, additively and from the current scaffold for its ownPRODUCT.md: a missing Dockerfile and.dockerignore, the README's marked run block (under its## Runheading),[tool.coverage.run]for a python product, and the CI's README run step where the CI is still the old scaffold's; anything the product made its own is kept and reported. One marker commit; a clean tree is required; a second run does nothing.
Changed
- The worker's
run_apptool runs the app where the sandbox runs: inside a Docker sandbox it starts detached, its health is probed with the profile's command from inside, and it is stopped by pid, so it no longer refuses a non-local sandbox; an app that exits reports its real exit code. The app-start code moved fromscio.product.acceptancetoscio.aci.apps, which both share (issue #58). - SCIO's own commits, merges and tags carry its identity (
scio) on the command line, so a product clone or a machine without a git identity builds and upgrades (I-104). - CI runs the test suite in parallel (pytest-xdist, twice the runner's cores);
pytest-xdistjoins the dev group (issue #58). - With
--gha product's release goes through a release PR, as SCIO's own do:releasecuts the changelog onrelease/v<version>(with the last progress record), pushes it and opens the PR;release_merge_waitstops until GitHub shows it merged;release_publishtags the merge commit, so the tag is onmain's history, and publishes the GitHub Release. Offline the release is unchanged (I-103). New state fieldsReleaseRecord.pr_numberandpr_url.
Fixed
- A node-ts product can build in the Docker sandbox: the sandbox image carries Node 22 next to Python, each profile names a capability probe (
sandbox_probe; node-ts: Node with TypeScript type stripping) that the readiness check runs inside the image, andscio buildreadsPRODUCT.md's stack before choosing the backend, falling back to local with the reason when the image cannot run the stack (I-101). - In GitHub mode a squash- or rebase-merged milestone PR no longer breaks the next milestone: after a confirmed merge
scio buildresets localmainto origin's when only SCIO's own bookkeeping commits are local and origin'sprogress.jsonstill records them, and halts withsync_failed(naming the commits) when a human's commit exists only locally (I-102). - The Node docker-live test skips on the plain Python image when no sandbox image is named, and still fails when a named image cannot run Node (I-105).
SCIO v0.7.0
Added
check_released_section_frozeninscripts/check_docs.py: the newest released CHANGELOG section must match what its tag shipped, which catches a branch's[Unreleased]entries that a merge across a release cut moved under the new version (I-089).- M11 coverage floor (issue #54, I-091):
SCIO_COVERAGE_FLOOR(default 80) is filled into the profiles' coverage commands; thenode-tsprofile gains a coverage check (Node's built-in--experimental-test-coveragewith--test-coverage-lines). A quality check markedverify: true(coverage in both stacks,typecheckinnode-ts) also runs in every product feature's verify once the suite and lint pass; a miss is the failure kindquality, sent to the debugger and never escalated, so a feature meets the barintegrateholds the milestone to. - M11 release checks (issue #54):
integrateadds a secret scan of the product's tracked files (prefixed token formats, private-key headers, and credential-named assignments whose value looks random; a line markedscio: allow-secretis skipped) and a release-files check (README, CHANGELOG, LICENSE, Dockerfile). Both are SCIO's own code (scio/product/release_checks.py), so they always run and a finding fails the milestone. Both scaffolds ship aDockerfileand.dockerignore: the image runs the same command the acceptance runner starts (python -m <pkg>ornode src/main.ts), with the approved dependencies installed; CI'sdocker-livejob builds and runs all four (python and node, cli and http). - M11 README check (issue #54): the scaffold's README carries its run commands between
<!-- scio:run -->and<!-- scio:run-end -->;integrateruns each one ([readme]; only commands that start with one of the profile's command words, no free-form shell) and the product's CI runs the same block in aREADME run commandsstep. - M11 release gate (issue #54): once no milestone remains,
scio buildrunsrelease_check(everyintegratecheck onmain, acceptance included, and the version the product's own--versionprints), stops atrelease_decisionfor a human (--decision releaseorstop; the stop is its own node, I-065), thenreleasecuts the product's changelog (chore(release): v<version> [scio:release:v<version>]), tagsv<version>, keeps<name>-<version>.tar.gz(agit archiveof the tag, Dockerfile included, plus the build check's output when there is any) under.scio/sandbox/release/, and with--ghpushes the tag and publishes a GitHub Release with the artifacts (GitHubClient.find_release,ensure_release). Every step is an effect with a world probe, so a resumed release neither tags nor publishes twice; a failing check ends the build withrelease_failed; a version already tagged ends it with nothing to do;merge and stopstill ends atdone. New stateReleaseRecord;scio buildprints the release line. scio new --spec PRODUCT.md --dir DIR(M11): bootstrap from a givenPRODUCT.mdas written, with no interview and no planner call (the product graph starts atbootstrapwhen the spec arrives approved).- M11 product eval suite (issue #54):
python -m evals.runner --suite product. A task (grader: product) carries a fixedPRODUCT.md(stack and interface pinned) and ahidden/directory;evals/product_solver.pybootstraps it with--spec, builds every milestone to the release gate (answeringmerge,releaseand a green gate'sproceedas the owner would,stopfor anything else, every answer counted), then runs the hidden tests with SCIO's own interpreter from a scratch directory against the released product:APP_CMDandAPP_DIRfor a CLI,APP_URLfor a server started from its## Runtime. p001 (CLI todo with tags and JSON export) and p002 (REST todo API with a YAML store) pass on the Fake; a stricter hidden test fails the same build. Trial spend includes every feature's child harness (I-095). - p003 (M11): a todo web app (one HTML page with the list and an add form,
http.server, a JSON store) graded in a real headless browser: the hidden tests use Playwright with Chromium (playwrightin the dev group;python -m playwright install chromium; CI'schecksjob installs it, elsewhere the browser tests skip). The runner gains--task ID(repeatable) to run one product eval.
Changed
- The http walking skeletons read
HOST(default127.0.0.1); the Dockerfile sets0.0.0.0so the container answers on its published port. - The coverage floor measures the product's own code, run inside the test process: the python scaffold sets
[tool.coverage.run] sourceand omits__main__.py, the node check excludestests/**. Both walking skeletons test their app in-process (node'ssrc/cli.tsexportscreateApp()) and mark the process lifecycle (serve) with the stack's coverage-ignore comment. Products bootstrapped before this change (bookmarks_api,notes_service) counted their tests toward coverage and test through a child process, so their nextscio buildmay stop at the honest floor (I-091).
Fixed
- On the Claude Code backend a structured call whose answer Claude Code's schema check refused within the turn cap is re-asked once with the required keys and length limits spelled out; and a provider failure inside one feature ends that attempt (recorded, with its spend) and the build with
feature_failedinstead of a traceback, so the nextscio buildstarts the next attempt (I-100). - A budget stop in the documenter's PR-body call, or at routing, is SCIO's graceful
budgetexit instead of an exception that ended a whole product build; a scan test keeps every model call in a node inside aBudgetExceededguard (I-098). - On the Claude Code backend token caps scale by
SCIO_CLAUDE_CODE_TOKEN_FACTOR(default 10), measured from the live product evals at about 380,000 billable tokens per estimated dollar, instead of the dollar factor 4, which made the token cap stop a healthy feature at about half its dollar cap (I-099). - Verify no longer accepts a feature whose selected tests all skipped:
run_testsreportsno tests ran: every selected test was skipped(a profile'sskipped_onlypatterns: pytest's summary, TAP's counters), verify fails the feature with a hint that acceptance tests run only at integrate, and the acceptance run and the eval graders fail an all-skipped run too (I-097: the first live p002 build halted on it). scio new's crash-resume probe for the bootstrap commit searchedgit log --grep=[scio:bootstrap]as a regex, a character class that matches nearly any commit message; it uses the fixed-stringfind_commitnow, and a scan test keeps every--grep=inscio/fixed-string (I-090).scio buildcommits the install phase's manifest and lockfile changes (npm install --saverewritespackage.jsonand writespackage-lock.json) on the milestone branch aschore: install approved dependencies [scio:install:<M>:<node>], before the first feature's reset could delete the lockfile; a merged milestone no longer leaves a dirty tree (I-085).load's sync ofprogress.jsonwithPRODUCT.mdis committed the same way ([scio:sync:<M>:plan_milestone]) instead of being reverted by that reset.- The node scaffold no longer writes a non-registry dependency (
file:,github:, a URL or a path) intopackage.jsonas a package of that name; the install phase writes it. Such a spec is also its own name on the approved list, so it only matches itself ininstall_deps(I-088).
SCIO v0.6.0
Added
check_release_cadenceinscripts/check_docs.py: the docs check fails when two milestones thatdocs/PRODUCT_MODE.mdcalls done are named only under[Unreleased](I-086: M8 to M10 waited two weeks behind v0.5.0); CONTRIBUTING gains the release-after-every-milestone step and a merged-on-GitHub check before any post-merge step (I-087).- M10 live milestone on the Claude Code backend:
notes_service(TypeScript, node:http plus zod) bootstrapped byscio new(the planner chosenode-tsfrom the stack menu) and built through M1 (three features, one advisor escalation,npm installthrough the install phase, eslint,tsctypecheck and build,npm auditand black-box acceptance all green, merged locally), $4.61 estimated, nothing billed; the done criterion of issue #51 holds on the Fake and live. Open: the install phase leavespackage.jsonandpackage-lock.jsonchanges uncommitted (I-085). - M10 Second stack (issue #51, ADR 0014): a
node-tstoolchain profile (configs/profiles/node-ts.yaml, TAP parsers inscio/toolchain/node_ts.py): Node 22 runs TypeScript directly,node --testis the runner,tsc, eslint andnpm auditthe quality bar (skipped while absent),npm installthe install phase; one data-drivenProfileToolchainloaded from yaml with aStackParsersset per stack; protocol members for the product path (acceptance_dir,skeleton_command,command_words,app_target,syntax_check_argv,health_probe_argv,detect);Runtimegeneralised (env_dir,env_marker, optional create step and interpreter); profile detection from marker files (Settings.toolchain=autois the default,RunContext.adopt_toolchainon the clone, the product load and the bootstrap); anode-tsscaffold (TypeScript walking skeleton withnode:testsmoke and acceptance tests, eslint flat config, CI withsetup-node); the normaliser, acceptance runner,run_app, tool descriptions and Claude Code allow-rules speak the profile's language; the toolbox context's toolchain reachesedit_fileandrun_tests, andedit_fileruns the profile's syntax check (module.stripTypeScriptTypesfor TypeScript, I-083); planner prompt v2 lists the available stacks;resolve_commandfinds npm's.cmdshims on Windows (I-081). The Faketodo_apispec builds its first milestone in Python and in TypeScript (test_same_spec_builds_in_both_stacks); CI installs Node 22 for the Node-dependent tests. - M9 (issue #49), first slices:
delete_filetool on every sandbox backend (Sandbox.delete; the coder and debugger have it; a new file rejected by the edit lint gate is now removed instead of left empty); the python profile declares a quality bar (checks:coverage with an 80% floor, wheel build, dependency audit) thatintegrateruns after the suite and lint, reporting a check whose tool is absent as skipped and failing the milestone on a real failure; coverage artifacts added to the profile's cache excludes; the Claude Code worker backend now refuses a non-local sandbox. - M9 runtime (ADR 0013): a per-product environment (
.scio/sandbox/venv, a plain venv with SCIO's own environment linked through a.pthfile on a local sandbox, the profile'stest_toolsinstalled on any other, I-074) created by the install phase with the network open for that call only (Sandbox.exec(network=True)) and the approved dependencies fromPRODUCT.mdinstalled;install_depsworker tool gated by that list (an unapproved package is refused with the escalation hint; context-aware tools viaToolDef.needs_context/ToolBox.context);run_appstartspython -m <module>, waits for a local health URL or the exit, and stops it (local sandbox only); the python profile'sruntime:block. - M9 live milestone on the Claude Code backend:
bookmarks_api(http.server pluspyyaml) bootstrapped byscio newand built through M1 (four features, one advisor escalation, black-box acceptance against the started server, coverage passed, merged locally), $5.38 estimated, nothing billed; the done criterion of issue #49 holds on the Fake and live. - M9 Docker default (ADR 0013 decision 10):
scio buildruns in Docker when a daemon answers and thedocker/sandbox.Dockerfileimage (Python plus git) is there, and in the local sandbox otherwise or on the Claude Code worker backend, with the first ledger line saying which and what a local sandbox does not enforce (SCIO_PRODUCT_SANDBOX_BACKEND=auto|local|docker).DockerSandboxis one persistent container per sandbox on an internal network (no route out),pythonrewritten to the product environment, a timeout inside the container keeping it;exec(network=True)attaches an egress-proxy sidecar (scio/aci/sandbox/egress_proxy.py, CONNECT-only,Limits.network_allowlist) for the duration of the call, so the install phase reaches the registry hosts and nothing else.run_acceptancestarts an http product inside a non-local sandbox and probes it from inside. CI gains thedocker-livejob that exercises all of it on the runner's daemon. - M9 acceptance (ADR 0013):
PRODUCT.mdgains a## Runtimesection (kind: cli|http, module, start arguments with{port}, health path) that the product planner fills and the normaliser checks; the scaffold shipstests/acceptance/(black-box: a subprocess for a CLI, the health endpoint for an http product, which the scaffold now serves withserve --port);integratestarts an http product on a free port, runs the acceptance tests withAPP_URL, stops it, and reports[acceptance]beside the other checks. The scripted Fake producttodo_api(an http server withpyyamlon its approved list) runs through the whole milestone loop offline: install phase, two features, server started for the black-box tests, merge. The scaffold's test directories are packages, so unit and acceptance tests may share basenames (I-073). - Claude Code backend (ADR 0012, issue #47): SCIO's model work can run through the unmodified
claudebinary on the owner's Claude subscription instead of the Messages API.SCIO_PROVIDER_OVERRIDE=claude_codesends every tool-less role throughClaudeCodeProvider(one restricted, tool-lessclaude -pper call,--json-schemafor structured outputs);SCIO_WORKER_BACKEND=claude_codesends every tool-using turn throughClaudeCodeWorker(onestream-jsonsession in the sandbox root with per-role--allowedTools, then a typed summary on--resume), with the transcript rendered in the shapescio reportreads and the usage folded into the ledger and budget byHarness.account. Verify, commits and the gate are unchanged.scio/loop/backends.make_workerchooses the backend at every worker call site;SCIO_CLAUDE_CODE_BINnames the binary. - On the
claude_codebackend every tier cap inbudgets.yamlis scaled bySCIO_CLAUDE_CODE_BUDGET_FACTOR(default 4.0, the measured ratio of Claude Code's estimate to the API price, I-069); the binary's own budget stop is SCIO's gracefulbudgetexit; aclaude -prun that fails without a message is retried (I-068); over-long prose fields in a typed summary are trimmed rather than failing the turn (I-070).scripts/nightly_live.py --provider claude_coderuns the nightly suites on the subscription, one trial at a time. scio build --product DIR [--gh owner/name] [--decision "<text>"]andscio status --product DIR(Product Mode M8, ADR 0011): the milestone loop. The orchestrator takes the first open feature ofPRODUCT.md; an Opusmilestone_plannergroups up to four into a milestone with a tier each (normalised to a consecutive prefix); a GitHub milestone and one issue per feature are created idempotently by title (offline: files under.scio/product/issues/); each feature runs through the task graph as a subgraph on its own checkpoint thread (build-<product>-<pk>-a<attempt>) on branchmilestone/<n>with the tier pinned and its own budget; the profile's full suite and lint run on the branch; an Opusmilestone_gateverdict that any failing check overrides; one milestone PR (cost, escalations, plan versus done,Closes #n) stops for the human merge;progress.jsonandNOTES.mdare updated onmainand the loop continues. A crash between features or after the PR resumes with no duplicated issue, commit or PR; a feature that halts stops for one decision (retry,skip, stop).merge and stoprecords the merged milestone and ends the thread; build threads are named per starting milestone (build-<product>-m<n>) andscio buildstarts a fresh one once the previous ended. New stateProductBuildState,FeatureRun,MilestonePlan;GitHubClient.ensure_milestone,ensure_issue,find_issue,pr_merged;Harness.child;scio reportrenders a product view forbuild-*threads; the product graphs are registered diagrams in ARCHITECTURE.md.
Changed
- After
scio newevery feature inprogress.jsonistodo, the walking skeleton included: the scaffold proves the toolchain and P1 is verified by its own feature run (ADR 0011 decision 1, the M7 open question).
Fixed
- The local sandbox no longer sets
RLIMIT_ASon POSIX: it capped virtual address space, which V8 reserves by the tens of gigabytes, and Node's TypeScript stripper died with "Cannot allocate Wasm memory" on CI (I-084); the CPU-time and wall-clock limits stay, andLimits.memory_mbis enforced by Docker. - On the Claude Code backend a
--json-schemaanswer is aStructuredOutputtool call that Claude Code bounces with a schema error when it misses the schema; with--max-turns 1the model had no turn to retry and the first live http milestone crashederror_max_turnsin the explorer's summary (I-077). Schema calls (worker summary, provider structured) get three turns and a capped summary is re-asked once with the required keys named. scio new --thread X --decision ...without--dircontinued in a default directory instead of the one the thread was started in, so the first...
SCIO v0.5.0
Added
scio new --prompt "<idea>" --dir <path> [--gh owner/name](Product Mode M7, ADR 0010): a product graph that interviews the user (two to four Opus questions), writes a normalisedPRODUCT.md, stops for approval, then bootstraps a repository with the profile's scaffold (package with--version, smoke test, CI, README, CHANGELOG, LICENSE,.gitignore,NOTES.md,.scio/product/progress.json), runs lint and tests as ground truth and commits with a[scio:bootstrap]marker. Every human stop is an interrupt resumed withscio new --thread ID --decision "<text>"; bootstrap steps are idempotent effects. New rolesinterviewerandproduct_planner; new stateProductState.
Changed
- Toolchain profiles (ADR 0009, Product Mode M6): every stack-specific command and parser (pytest argv and exit codes, ruff lint, syntax check, test-path rules, cache excludes, dependency files, error signatures, the prompt phrases that name tests) moved behind
scio.toolchain.Toolchain, with thepythonprofile inconfigs/profiles/python.yamlreproducing the pre-M6 behaviour byte for byte.Settings.toolchainselects the profile;test_toolchain.pyguards the boundary. scio run --tierpins the route as well as the budget (I-053); without--tierthe router decides and a default budget is resized for the tier it picks (RunContext.apply_tier_budgets); explicit eval budgets are untouched.- Assumption F-8 ("targets are Python") superseded by ADR 0009; PROMPT.md's "No UI" non-goal narrowed to SCIO itself (ADR 0007).
Fixed
SCIO_SANDBOX_BACKEND,SCIO_SANDBOX_DOCKER_IMAGE,SCIO_SANDBOX_E2B_TEMPLATE,SCIO_EFFECTS_PATHandSCIO_RUN_LIVE_TESTSwere declared and never read (I-054, I-059);make_sandboxselects the backend,RunContext.createhonours the effects path, the live-test gate readsSettings.
SCIO v0.4.0
Changed
- Product Mode roadmap (
docs/PRODUCT_MODE.md, M6–M11) and ADR 0007: a product orchestrator over the task graph, toolchain profiles for any stack, milestone approvals; two defects logged for M6 (I-053, I-054). - A passing verify whose diff is high impact (more than 8 files, dependency or schema files, a removed public symbol) now gets one advisor review before commit (ADR 0008):
review_decision, atriageroute from a passing verify, and anadvise -> git_checkpointedge forcontinue. Until now every advisor trigger required a failing verify, and no live run had ever called the advisor (I-056).
Added
- t010_wide_rename (T1, pinned):
format_centsbecomesformat_money(cents, currency)across nine caller modules; the correct diff passes verify and trips the impact detector, so the advisor reviews it. Fake script edits each caller in one whole-file edit becauseedit_filelints every edit (I-057).
SCIO v0.3.0
Changed
- E2B is no longer a planned live tier: no account (I-029, wontfix). Local backend for own fixtures, Docker for untrusted repositories (ADR 0003).
Added
- t008_snapshot_coupling: a formatting change that breaks a CLI snapshot test and a README example test outside the coder's filtered run; the first regression task built to make verify's full-suite step fail and the debugger run live.
- t009_verify_only_lint (T1, pinned): an undefined name in a module no test imports; the coder's
run_testsis green and only verify's lint step fails, so the debugger must run and wire the constant rather than delete the module (hidden tests check both). First live debugger run: one turn, $0.06 of a $0.34 trial, fix plus sweep. Fake scripts gainedStep.debug_editsfor the debugger's turn. tier_pin:on an eval task runs that tier without consulting the router, which rated t008 T0 and skipped the nodes the task exists to exercise (I-050).RunContext.tier_pin;routerecordsclassifier_model="pinned".- Two realistic regression tasks: t006 (a change spanning the store module, the CLI and their tests) and t007 (a fixture whose own CI is red on
ruffwith the repo config and onpytest); hidden tests run the fixture's own lint rules. One live trial each passed ($0.35 and $0.40). - Suite cost cap:
evals/runner --max-usd; the report records spend, cap and skipped tasks. - Nightly live suite:
scripts/nightly_live.py(profile auth, cost cap, rows indocs/EVAL_HISTORY.md) registered as a Windows scheduled task on the dev machine with--max-usd 12; the GitHub nightly workflow gained the cap, parallel trials and auth-token secret support. scio reporttool signals, counted from the worker transcripts with no model call: calls and error rate per tool, repeated identical calls within a turn,view_filere-reads, and whether the documenter edited docs when code changed. The documenter's transcripts are now kept aslogartifacts like the coder's.
Fixed
- Eval trials leaked their temp directory on Windows (636 directories): read-only git objects and an open effect-log handle, both hidden by
rmtree(ignore_errors=True).remove_treeclears the bit,RunContext.close()releases the handle (I-049). - Eval fixtures are copied without
__pycache__and tool caches, so bytecode left by running a fixture's tests in place can no longer become tracked content that shows up in the agent's commits (I-048).
SCIO v0.2.0
Fixed
- Token budgets count billable-equivalent tokens (cache reads at 0.1x, writes at 1.25x) so prompt caching lowers the count it enforces (I-041).
- Prompt caching now covers the conversation: a third breakpoint on the last message block makes each worker turn re-read its transcript at cache price (I-040,
docs/IDEAS_TRIAGE.md). - A tool crash (e.g.
view_fileon a directory) is returned to the model as an error result instead of ending the run (I-032). - Budget pre-check reserves half of
max_tokensinstead of all of it; explorer loops capped at 6 model calls / 8 tool calls; small task budgets raised to the tier the live router picks (I-033, I-035). tests_addedwording: the tests that prove the features, existing or new; bug-fix tasks no longer loop on "add tests" (I-034).- T1 verify adopts the worker's reported tests when the plan's guessed test names match nothing (I-037); worker turns capped at 12 model calls and transcripts trimmed from 30k characters (I-038); the planner no longer emits exploration steps and
normalize_stepsdrops no-op steps and covers missing features (I-039). - verify runs each reported test selector correctly:
path::namenode ids and file paths as positional pytest targets, bare words as-k; found by the first live Anthropic run (I-031). - pdoc no longer warns on PEP 695 generics:
scio/prompt/repair.pyandscio/loop/agent_loop.pydrop the__future__annotations import (I-008).
Added
evals/runner --parallel N: a task's trials run concurrently (independent sandboxes, checkpoints and budgets).- Live Anthropic acceptance recorded: t001 pass^5 = 1.00 (5 trials) via an
ant auth loginprofile. - Live sandbox checks:
tests/live/test_docker_live.pyruns wherever Docker is available (CI runners included) andtests/live/test_e2b_live.pyruns whenE2B_API_KEYis set (I-016).
SCIO v0.1.0
Added
- Observability: per-run report (
scio report), persisted cost ledger per thread,RunReportwith cost by role, escalations and timeline. - A/B role configurations (
evals/configs/*.yaml,SCIO_ROLE_OVERRIDES_PATH); eval runner--config; capability suite with scenario tasks (fixture,scenario,expect_pass); nightly evals workflow posting per-config tables. evals/tune.pyescalation threshold sweep;EscalationConfigenforcesmax_stall >= repeat_threshold - 1; ADR 0006.- Release workflow and
scripts/release_notes.py;SCIO_FAKE_SCRIPT=1offline demo mode for the CLI; the CLI runs the unified graph (router picks the tier). - README acceptance-status table; first real GitHub end-to-end run recorded.
Fixed
prepare_repowrites repo-local git excludes so test caches are never committed (I-028).- T1 and T2 tiers in one graph builder (
scio/graph/build.py):scope, parallelexplore(LangGraphSend, read-only),localize,plan(plan-owned test filters),post_plan(issue checklist),implement(one step per cycle),debug,advise,review; generatedt1/t2diagrams. - Escalation policy (
scio/loop/escalation.py,scio/loop/impact.py): six signals, never-escalate list, per-step and per-run caps,advisor_capexit with human interrupt;TriageDecisionin state;EscalationRecordwith trigger, brief, verdict, tokens, $ and outcome; advisor guidance injected into the next worker turn; ADR generation in the target repository. ToolBox(role=...)on spans;WorkerOutput.sweepsurfaced in the ledger; PR body carries the escalation log and reviewer notes.- Five real regression tasks (t001–t005) with fixture repositories, hidden tests and a data-driven
ScriptedResponder; the runner reports advisor call and resolution rates from the graph. - Postgres checkpointer built with the serializer allow-list; the gate runs the docs check directly.
- ADR 0005: escalation policy implementation.
- Context layer (
scio/context): artifact store, per-role prompt projections, size-triggered transcript trimming. - Loop layer (
scio/loop):AgentLoop(tools until done, tool/model caps, typedWorkerOutputsummary) and exit conditions. - T0 graph (
scio/graph/tiers/t0.py): intake, route, prepare_repo, t0_loop, verify, git_checkpoint, document (docs commit, push, draft PR), gate with ground-truth override, pr_ready, merge_wait interrupt, halt.scio run/scio resumeCLI. WorkerOutput.tests_added; verify requires changes, feature tests, a passing suite and clean lint.- Eval runner drives the real graph for
grader: teststasks and grades with hidden tests; t001 fixture repository;ScriptedResponderfor deterministic Fake runs. - Generated
t0Mermaid section indocs/ARCHITECTURE.md; LangGraph serializer allow-list for state types. - ADR 0004: loop and verification design.
- State backbone (
scio/state): the M0 schema as Pydantic models with reducers,WorkerOutput.sweep, error signatures stable across line numbers and paths, progress fingerprints and stall counting. - Graph runtime (
scio/graph): checkpointer selection by DSN (memory, SQLite, Postgres), idempotent side effects keyed byhash(thread_id, node, intent)with a durable effect log and world probes, Mermaid rendering with doc sections checked in CI, graph registry. - Agent-computer interface (
scio/aci):Sandboxprotocol with local, Docker and E2B backends;repo_search,view_file,edit_file(parse- and lint-gated),run_tests(failures only),git_checkpoint; output truncation with hints; capability tags and the compile-time trifecta check;ToolBoxdispatcher with tracing and a call cap. GitHubClient(scio/harness/github): idempotent branch, issue-comment and draft-PR writes throughgh, default-branch guard, write log.- Durability test: a 3-node graph with file and git side effects killed mid-run and resumed with zero duplicated effects.
scripts/gen_mermaid.py;check_docs.pynow verifies generated graph sections.- ADR 0003: idempotent side effects and sandbox tiers.
- Harness layer (
scio/harness):ModelProviderprotocol; Anthropic, OpenAI and deterministic Fake adapters; single retry layer with backoff andretry-after;BudgetGuardchecked before every call; per-call cost ledger; prompt-cache breakpoint placement; OpenTelemetry GenAI spans with opt-in content capture. - Prompt layer (
scio/prompt): versioned per-role templates (12 roles at v1); Pydantic → provider-safe wire schema; structured-output validate-and-repair loop (max 2 repairs). configs/models.yaml(the only place model IDs live) andconfigs/budgets.yaml(per-tier budgets).- Eval skeleton (
evals/): task loader, n-trial runner, pass@1 / pass^k / tokens / $ / latency / advisor-rate metrics, Markdown and JSON reports, three placeholder regression tasks. - CI (
.github/workflows/ci.yml): ruff, mypy, pytest on the Fake provider, docs checks, regression evals, API docs artifact.scripts/check_docs.py,scripts/gen_api_docs.py. - ADR 0002: harness conventions.
- Continuous-improvement protocols:
CLAUDE.md,docs/ISSUE_LEDGER.md(I-001 to I-010),docs/LESSONS_LEARNED.md(M0 and M1 retrospectives), Debugger template v2 with systemic sweeps, PR template checklist. - Build spec and generation prompt (
docs/PROMPT.md). - Research basis with sources (
docs/RESEARCH.md). - ADR 0001: single writer, Sonnet 5 worker, Opus 5.5 planner/advisor.
- Repository docs: README, CONTRIBUTING, issue and PR templates.
- Milestone 0 architecture (
docs/ARCHITECTURE.md): graphs per tier, State schema, agent contracts, directory tree, milestone plan, assumptions and open questions.