Repository navigation
v1.0.0-rc.4
Pre-release
Pre-release
·
436 commits
to main
since this release
Fourth release candidate, themed "Agent-native": coding agents operate attest without the website through --json output with published schemas and stable exit codes, offline guides (attest guide), installable skills and plugins, and an MCP server whose closed contract grammar drafts and checks contracts. attest mutate measures how much of an agent's behavior the suite would actually catch.
Added
attest mutate <fixture.attest.json...> [--contracts DIR] [--seed N] [--max-mutants N] [--json]tests the tests: it builds behavior mutants of each fixture (drop, duplicate or reorder tool calls, perturb arguments, inject tool errors, alter world effects, truncate output), evaluates the suite's deterministic contracts on every mutant at no model cost, and reports a mutation score, each surviving mutant and a verified contract that would kill it. The mutant set is identical across runs for a fixed seed.--jsonvalidates againstcli/mutate.schema.json; exit 0 report produced, 1 no contract applies to any fixture, 2 usage or load error. Seedocs/guides/mutation.md.ATTEST_CONTRACTS_DIRmakes each engine process record the assertion specs it evaluates (one0600NDJSON file per process, no traces or outputs) soattest mutate --contractscan replay a suite's contracts.evaluate_batchtakes an optionalcontext: {test_id, file, line}that the pytest plugin and the Vitest integration send for the running test; the engine records it next to each spec and does not otherwise use it.- The policy rule
min_mutation_score: {score, severity}withattest policy evaluate --mutation <mutate.json>: a score below the threshold exits 1 (block) or 2 (warn). A policy with the rule and no--mutationis a load error (exit 3). attest guide [<topic>]prints the shipped guides (docs/guides/*.md) from the engine binary, offline;--jsongives{topics:[{name,title}]}or{topic,title,markdown}. New guides:cli(agent-facing CLI reference, exit-code table) andagents(skills,init --skills, plugins). Guides are staged into the engine module bygo generate ./internal/assets/(make generate) and embedded; a test ingo test ./...fails when the staged copies drift.- Every engine command and subcommand accepts
--jsonand then writes exactly one JSON document to stdout on success; on a usage, load or runtime error it writes nothing to stdout and the exit code plus stderr describe the error. Schemas are inproto/attest/v1/cli/(doctorkeepsproto/attest/v1/doctor.schema.json) with valid and invalid fixtures inprotocol-tests/cli/, tested in Go, Python and TypeScript.capture,replayandworldwith-- <command>send the command's stdout to stderr under--jsonand print one result object that carries itsexit_code.attest help --jsonlists every command with its schema and exit codes.attest init --json(both SDKs) prints{created, updated, skipped}, and the SDKs addinittoattest help --json. - Each command declares its exit codes in the registry; the table in
docs/guides/cli.mdis checked against it by a test. attest skills list|install [--dir D] [--json]writes the embeddedattestandattest-verify-generatorskills to.agents/skills/(Codex, Cursor) and.claude/skills/(Claude Code), touching only attest-owned files. It validates every destination before writing, follows directory symlinks that resolve inside--dir(writing each real file once) and refuses symlinks that resolve outside it or symlinked skill files, writing nothing.attest init --skills(Python and TypeScript, alone or with--ci github) calls it. Theattestskill routes to writing contracts, capturing an incident, debugging a failure, CI setup and migrating from Promptfoo or DeepEval with aMIGRATION.mdprogress file;attest-verify-generatorinterviews a repository and writesverify-<agent>skills.attest mcp [--root D]serves attest to coding agents over MCP on stdio with a fixed toolset:trace_check,evaluate,explain_failure,draft_contract,run_contractandfixture_info. It is read-only and offline (deterministic layers only; judge and embedding assertions returnnot_evaluatedwithprovider_not_configured) and confined to the project root: paths are resolved through symlinks, files are capped at 32 MiB.run_contractparses statements such astool_cardinality("issue_refund", per="order_id", at_most=1)with a closed grammar, never as code, and returns per-statement results plus the Python and TypeScript test lines.attest mcp --tools --jsonprints the tool list;attest mcp install --dir D --command <json argv>andattest init --mcp(Python and TypeScript) write.mcp.json,.cursor/mcp.jsonand.codex/config.tomlwithout touching other servers. Guide:mcp.- Claude Code, Cursor and Codex plugin manifests (
.claude-plugin/,.cursor-plugin/,.codex-plugin/) publish the sameskills/; they are generated fromVERSIONand one metadata source bygo generateand checked for drift. In Claude Code:/plugin marketplace add attest-framework/attest.
Security
- JSON Schema assertions (
type: schema) resolve$refinside the schema document only:file://andhttp://references no longer load, for every engine caller. Previously a schema could read any local file the engine could, and its compile errors could echo part of it. The compiled-schema cache is bounded.
Changed
attest policy evaluateexits 3 on usage errors and when the policy, report or baseline fails to load (previously 2 and 1), so they cannot be read as a warning or a block. Verdicts keep 0 pass, 1 block, 2 warn.attest cache,baseline,versionandpolicy evaluatereject unexpected arguments;cacheusage errors exit 2 (previously 1).attest capture --refreshalso refuses to prompt (exit 2,--yeshint) whenCIis set or--jsonis given.
Published components
| Component | Status |
|---|---|
| Engine binaries and checksums (this release) | published |
PyPI attest-ai==1.0.0rc4 |
published |
npm @attest-ai/core@1.0.0-rc.4 |
published |
npm @attest-ai/vitest@1.0.0-rc.4 |
published |
| Website sync | dispatched (release-sync to attest-framework/attest-framework.github.io) |