Skip to content

Releases: devtranger/ChangeSafe

ChangeSafe v0.5.0

Choose a tag to compare

@devtranger devtranger released this 07 Aug 18:25
c1ae07e

ChangeSafe v0.5.0

A trust-model fix in @changesafe/core, a new Terraform policy that replaces
a prose-only skip with a real one, changesafe eval gaining Kubernetes
support, and two security-relevant fixes to the Kubernetes analysis prompt.

Policy behavior changed. CORE_POLICY_VERSION moves to core-v0.2.0 and
TERRAFORM_POLICY_VERSION moves to terraform-v0.2.0. A v0.5.0 receipt is
not directly comparable to a v0.4.x one — that is the point of the version
existing, not an oversight.

Universal-policy skip legitimacy, enforced in core

A domain adapter's skippedUniversalPolicies could previously name any of
the five universal policies, not just the two whose shape assumes a
model-authored proposal or an in-place patch (ROLLBACK_COMPLETE,
VERIFICATION_REQUIRED). The only check against this lived in the app's own
runtime registration (features/domains/runtime.ts), not in published core —
so a hand-written adapter, or changesafe gate run directly against one,
got no enforcement at all. A probe adapter that skipped all five universal
policies passed evaluatePolicies cleanly with zero findings and LOW risk,
despite the proposal carrying an injection attempt, invented evidence, no
rollback, and no verification.

replacedBy was also a bare string, so nothing verified a skip's claimed
replacement existed. Terraform's VERIFICATION_REQUIRED skip named "the pull request review" — prose, not a policy.

Fixed:

  • evaluatePolicies and policyOrder both validate every skip before doing
    anything else, and throw for an illegitimate policyId, a duplicate skip,
    or a replacement that names a policy the adapter does not declare (or that
    collides with a universal policy id, which policyOrder's id filter could
    not otherwise distinguish from the skip itself). Only
    ROLLBACK_COMPLETE/VERIFICATION_REQUIRED may ever be skipped —
    PATCH_SCHEMA, BLAST_RADIUS, and UNTRUSTED_INSTRUCTION are structurally
    answerable by every domain and no adapter may skip them.
  • SkipReplacement is a typed shape ({ kind: "domain-policy", policyId })
    instead of a string. A replacement that named a non-mechanical process
    rather than a real policy would have produced no finding of its own — the
    gate would pass with a genuine verdict gap rather than merely an
    honestly-labeled one — so that option does not exist at all.
  • Every receipt now carries policyCoverage: the exact ordered policy ids
    that ran, plus what was skipped and why. A verifier reads this from the
    signed payload instead of fetching and trusting the adapter's source to
    interpret an absent policy id. The field is optional on
    ChangeReceiptSchema so a receipt issued before it existed (v0.4.1 and
    earlier) still parses for hash and signature verification —
    createReceipt always sets it going forward, and canonicalize drops an
    undefined property, so a legacy receipt's hash is unaffected.

Terraform gains PLAN_CONTEXT_REQUIRED

Terraform's proposal is derived mechanically from the plan with no model
involved, so it can never declare its own precondition or postcheck steps —
that is the honest reason it skips VERIFICATION_REQUIRED. What replaces it
used to be prose. It is now a real policy: PLAN_CONTEXT_REQUIRED warns when
a destructive change carries zero PR or commit context, because the pull
request review this domain relies on for verification needs something to
review. A plan that destroys nothing, or that carries any context entry,
passes.

changesafe eval measures Kubernetes too

The benchmark previously imported networkAnalysisPrompt and networkDomain
only, so it measured one domain while the corpus spans three. Kubernetes now
has its own hardened prompt (packages/ai/src/prompts/kubernetes.ts) and is
registered in ANALYSIS_DOMAINS; eval --domain kubernetes resolves it
instead of the hardcoded network path. Terraform stays out by design — its
plan already is the proposal, so there is nothing for a model to propose.
Report schema moves to version 3, recording corpus.domain, so a report is
only ever compared against another from the same domain.

Two fixes in the same change

Landed as part of getting Kubernetes eval support production-honest, not
found independently:

  • Kubernetes eval input is normalized. The strict, already-normalized
    KubernetesSnapshotSchema was being parsed directly against raw,
    collector-shaped scenario fixtures — every bundled Kubernetes scenario
    failed before ever reaching the provider. eval's Kubernetes domain now
    tries the strict parse first and falls back to normalizeSnapshot, the
    same boundary changesafe gate already used.
  • Pod-label and Service-selector values stay inside the untrusted
    block.
    The Kubernetes prompt's trusted preamble — the section the model
    is told to trust, ahead of <untrusted_snapshot_data> — was echoing
    podLabels and selector values into it. Those are Record<string, string> with no charset restriction beyond length, so an instruction-like
    value there bypassed the untrusted-content boundary the system prompt
    describes. They are no longer copied into the preamble; the full snapshot
    already carries them inside the untrusted block.
  • Rollback references are validated too. The Kubernetes prompt's local
    cross-check validated a forward replace against the snapshot but never a
    rollback replace, and accepted any rollback remove regardless of
    whether it undid something the proposal actually added. A model response
    with an ungrounded rollback reference was counted accepted by the
    benchmark before the gate's PATCH_SCHEMA policy caught it downstream —
    inflating the grounded-output metric for a proposal that was always going
    to be blocked.
  • Prompt/adapter pairing is typed. The analysis domain registry paired
    each prompt with its adapter through two independent as unknown as never
    casts, so nothing prevented registering, say, the Kubernetes prompt against
    the network adapter. A single generic function now binds both under one
    type parameter, so a mismatched pairing fails to typecheck.

Known limits

  • Terraform still cannot catch command smuggling: it is an external-diff
    domain with no simulator, and no Terraform policy inspects planned
    attribute values for command payloads. Tracked, not papered over with a
    scenario the gate would not actually refuse.
  • K8S_SERVICE_SELECTOR still only re-checks Services that existed before a
    change; whether it should also check newly created Services is an open
    decision, not yet made.

ChangeSafe v0.4.1

Choose a tag to compare

@devtranger devtranger released this 07 Aug 01:22
bafdeeb

ChangeSafe v0.4.1

This release completes v0.4.0. It contains no product changes: the code is
v0.4.0 plus a fix to the release workflow. Policy versions are unchanged, so
receipts from v0.3.1, v0.4.0, and v0.4.1 all remain comparable.

Read RELEASE_NOTES_v0.4.0.md for what actually
shipped — the multi-domain workbench, the 26-scenario corpus, and the four
security and hygiene fixes.

Why this version exists

v0.4.0 published three of its five packages and then stopped:

Package v0.4.0
@changesafe/core published, with provenance
@changesafe/domain-network published, with provenance
@changesafe/domain-terraform published, with provenance
@changesafe/domain-kubernetes not published
changesafe not published

@changesafe/domain-kubernetes had no trusted-publisher configuration on npm
— it was created by a manual publish during the v0.3.0 bootstrap and never
wired up — so the registry answered 404 on PUT, and the loop exited before
reaching the CLI. That configuration is now in place.

Use v0.4.1. The three packages published at 0.4.0 are genuine and carry
provenance, but 0.4.0 is not a complete set: the CLI and the Kubernetes domain
do not exist at that version. Nothing published at 0.4.0 is broken or
withdrawn — @changesafe/core@0.4.0 and the two domains that shipped with it
install and work — but only 0.4.1 gives you all five.

What changed in the release machinery

Publishing the workspace is five separate registry calls, and the workflow
refused to run if any version already existed. That made a partial publish
unrecoverable except by hand, which is how v0.3.x lost its provenance. A
version already on the registry is now skipped rather than refused, so a
resumed run sends only what is missing and says which packages it skipped.

The selection lives in scripts/select-unpublished.sh with test coverage for
the states that matter — nothing published, a partial failure, everything
published, a version mismatch, and a registry lookup that fails — because the
failure mode here is silent: skipping a package that was never published
produces an incomplete release that still reports success.

One thing worth recording for whoever recovers the next partial publish: a
release event runs the workflow file at the tagged commit, not the one
on main. Fixing the workflow after tagging does not fix that release. That
is why v0.4.0 could not simply be re-published, and why this is a new version
rather than a retry.

ChangeSafe v0.4.0

Choose a tag to compare

@devtranger devtranger released this 06 Aug 23:30
b6691f9

ChangeSafe v0.4.0

This is the first published release of the multi-domain Change Review
Workbench, plus a corpus that grew from nine Network scenarios to 26 across
three domains. Policy behavior did not change: CORE_POLICY_VERSION,
NETWORK_POLICY_VERSION, and TERRAFORM_POLICY_VERSION all stay put, so a
v0.4.0 receipt and a v0.3.1 receipt remain directly comparable.

The workbench is multi-domain

/ is the Network public replay. Terraform lives at /workbench/terraform,
Kubernetes at /workbench/kubernetes, and the optional authenticated client
at /workbench/self-hosted. Domains register against an app-local
runtime/presentation registry and a domain-neutral review controller, so the
shell does not know what a network is.

The authority split is enforced in domain and server logic rather than in the
UI:

  • Public replay evaluates and displays. It creates no decision, no
    simulation record, no durable review, and no receipt.
  • The self-hosted API carries OIDC approver identity, recomputes findings
    server-side, signs receipts, and appends to the ledger before responding.

changesafe serve now wires the durable review queue, so a self-hosted
deployment persists reviews rather than holding them in memory.

Breaking for app deployments: the exact /workbench route and
/api/analyze are retired and return 404. Use the domain subroutes and the
versioned replay API instead. No published package API was renamed.

Corpus

26 scenarios, 12 adversarial, across network, Terraform, and Kubernetes —
each with an expectations.json that CI verifies against the real engine.
docs/SCENARIOS.md is generated from the corpus and CI fails if it drifts.

New in this release, scenario-z-orphaned-canary-service is Kubernetes' first
created-resource change and its first silent safety regression:
K8S_SERVICE_SELECTOR only re-checks Services that existed before the change,
so a newly added Service selecting a pod label no workload carries clears
every policy and is caught only by the sandbox. That is the failure mode the
gate is honest about not catching, which is why the scenario exists.

Security fixes

  • 401 for an undecodable JWT signature. A signature segment that is not
    valid base64url escaped the OIDC decoder as a DOMException and surfaced as
    HTTP 500. It is now the same AuthenticationError every other malformed
    segment produces, refused before any JWKS fetch. (#24)
  • Provider calls are bounded. Every OpenAI, Anthropic, and Ollama call now
    runs under a deadline and a 2 MiB response cap enforced during the read,
    composed with — not replacing — a caller-supplied AbortSignal. The default
    is 60s for hosted APIs and 600s for a local Ollama model, because CPU
    inference and a cold model load are slow for reasons that have nothing to do
    with a stalled connection. changesafe analyze --timeout <seconds> and
    changesafe eval --timeout <seconds> override both, as does timeoutMs on
    AnalyzeOptions. This is a behavior change for library consumers who
    previously relied on an unbounded wait. (#26)
  • SECURITY.md matches what shipped. The authenticated server, ledger
    integrity, and signature-vs-integrity boundaries are now named in scope,
    and the two runtimes have separate documented boundaries. (#25)
  • Dependency advisory cleared. The high-severity brace-expansion
    advisory in the ESLint tree is resolved by pinned overrides; npm audit
    reports zero vulnerabilities. (#27)

Known limits

  • Terraform cannot currently catch command smuggling: it is an external-diff
    domain with no simulator, and no Terraform policy inspects planned attribute
    values. This is tracked in #55 rather than papered over with a scenario the
    gate would not actually refuse.
  • changesafe eval still runs against the Network corpus only.

ChangeSafe v0.3.1

Choose a tag to compare

@devtranger devtranger released this 29 Jul 08:35
da6dbfa

ChangeSafe v0.3.1

This patch release republishes the v0.3 Kubernetes domain with valid
Node-resolvable ESM import specifiers.

@changesafe/domain-kubernetes@0.3.0 was built from raw TypeScript output
during its first manual publication, so importing the package directly in
Node failed on extensionless relative imports. The CLI bundle was not
affected, but the library package was.

v0.3.1 adds the Kubernetes domain to the installed-tarball import smoke test,
bumps the workspace as one release set, and keeps all Kubernetes gate and
collector behavior unchanged.

Use @changesafe/domain-kubernetes@0.3.1 or later. The broken 0.3.0
library version is deprecated on npm.

ChangeSafe v0.2.0

Choose a tag to compare

@devtranger devtranger released this 28 Jul 23:10
19e696a

ChangeSafe v0.2.0

The engine is installable. v0.1.1 made the GitHub Action usable by tagging
it; this release puts the library and the CLI on npm, which is the adoption
path the roadmap has always described — an embeddable library and a
CI-friendly command-line gate
.

Nothing about the trust model changes. No policy behavior changes, and no
verdict changes.

npx changesafe gate --domain terraform --input tfplan.json
npm i @changesafe/core @changesafe/domain-terraform
Package What it is
changesafe The CLI: gate, analyze, eval, verify, keygen, ledger, serve, scenario. One bundled file, no dependencies
@changesafe/core The domain-agnostic gate: proposal contract, universal policies, risk derivation, workflow state machine, receipts, the DomainAdapter contract. Depends on zod alone
@changesafe/domain-network The network domain: declarative state, allowlisted transactional patches, deterministic reachability, sandboxed simulation
@changesafe/domain-terraform The Terraform domain: normalizes terraform show -json and polices destruction, protection, and reversibility. Never runs Terraform

@changesafe/ai, @changesafe/ledger, and @changesafe/server stay
unpublished for now; they are the self-hosting pieces and join in v0.3.0.

These four tarballs were published by hand. npm retired the classic publish
tokens the release workflow was written against, so 0.2.0 went out from a
maintainer's machine and carries no provenance attestation — the registry
records the publisher, not the workflow and commit that built it. Verify this
release the way the repository has always asked you to instead: check out the
v0.2.0 tag and reproduce it (see Verification below).

Publishing moves to npm trusted publishing (OIDC) from the next release, which
attaches provenance automatically. @changesafe/core already has the
publisher configured.

Embedding the gate

import { evaluatePolicies, hasBlockingFinding } from "@changesafe/core";
import { networkDomain } from "@changesafe/domain-network";

const { findings, riskLevel } = evaluatePolicies(networkDomain, incident, proposal);
if (hasBlockingFinding(findings)) {
  // Nothing in this library can approve it. That is the point.
}

A DomainAdapter teaches core what a change means in your world; the
universal policies then work unchanged. packages/core/tests/standalone-domain.test.ts
implements a complete toy domain in one file if you want the shape of it.

What changed since v0.1.1

The packages ship compiled JavaScript and declarations instead of
TypeScript source, so npm i works and editors resolve types. The workspace
still develops against source — no build step joined the test loop.

The Action no longer builds itself in your CI. It previously ran
npm ci on this whole repository inside every consumer's workflow before
evaluating a single policy. The CLI bundle is committed and executed
directly; CI rebuilds it and fails if the committed bytes differ from the
source, so what runs in your pipeline is what was reviewed here.

changesafe gate no longer prints an experimental-SQLite warning on every
run. The ledger's node:sqlite import is loaded when a ledger is opened.

--app-version joins --receipt-id and --created-at for regenerating
or re-verifying an audited snapshot with a later build. Ordinary runs record
this build's own identity and need nothing.

Version numbers, and one that stayed put

Package versions and build identity move to 0.2.0: receipts now record
appVersion: "changesafe-cli-0.2.0".

The policy versions did not move — core-v0.1.0, network-v0.1.0,
terraform-v0.1.0 — because no policy behavior changed. They answer a
different question from the release number: which gate decided, not which
binary ran
. A receipt from v0.1.1 and one from v0.2.0 therefore remain
directly comparable, and policyVersion still means what it has always meant.
tests/integration/version-sync.test.ts enforces both halves of that.

The published v0.1.0 verification snapshot still reproduces byte for byte
(npm run verify:v0.1.0) across this bump.

Upgrading

Your situation Do
Using the Action Move the pin to @v0.2.0 (or @v0 to track patches). Nothing else changes
Vendoring the repository to get the engine npm i @changesafe/core and delete the vendored copy
Running the CLI from a clone npm i -g changesafe, or npx changesafe. The clone still works
Self-hosting the decision server No change. @changesafe/server is still consumed from source until v0.3.0

Verification

git checkout v0.2.0
npm ci
npm run verify:v0.1.0   # the published v0.1.0 snapshot still reproduces
npm test                # 448 passing, 2 skipped
npm run build:cli && node packages/cli/dist/changesafe.js scenario check   # 9/9

The red-team corpus still refuses every proposal it is supposed to refuse,
including when the gate is reached through the installed packages rather than
this workspace — tests/integration/publishable-packages.test.ts gates the
route-leak scenario through unpacked tarballs and requires CRITICAL.

ChangeSafe v0.1.1

Choose a tag to compare

@devtranger devtranger released this 27 Jul 05:26
3d62475

ChangeSafe v0.1.1

A security and correctness patch. Nothing about the trust model changes: AI
proposes, deterministic code validates, a human decides, and ChangeSafe still
executes nothing.

This is ChangeSafe's first tagged release. v0.1.0 was documented but
never tagged, so uses: wonkwonlee/ChangeSafe@v0.1.0 never resolved for
anyone. That does not make the fix below less urgent, and here is the part
worth reading twice: in the example workflow, the vulnerable capture step
runs before the uses: step that would have failed. A copied workflow was
therefore exploitable even though the gate itself never ran once.

If you copied the example workflow, you have something to change, and
bumping a version pin is not it. See Action required below.

- uses: wonkwonlee/ChangeSafe@v0.1.1   # or @v0 to track v0.x patches

Action required if you copied the example workflow

Before this release, examples/github-actions/gate-terraform-plan.yml
captured the pull request body like this:

- name: Capture the pull request body
  run: |
    cat <<'CHANGESAFE_PR_BODY' > pr-body.txt
    ${{ github.event.pull_request.body }}
    CHANGESAFE_PR_BODY

A ${{ }} expression is substituted into the script before bash parses it,
so the quoted heredoc does not protect the delimiter. A pull request whose
body contains a line equal to CHANGESAFE_PR_BODY closes the heredoc, and
everything after it runs as shell commands in the runner — in the workspace,
before the gate reads the plan. That is enough to rewrite tfplan.json or
pr-body.txt and get a clean verdict on a change the gate never saw.

Updating the uses: pin does not fix a copy already sitting in your
repository. Replace that step with:

- name: Capture the pull request body
  env:
    CHANGESAFE_PR_BODY: ${{ github.event.pull_request.body }}
  run: printf '%s' "$CHANGESAFE_PR_BODY" > pr-body.txt

The body still reaches the gate as untrusted text that is scanned and never
obeyed. It just stops reaching the shell.

To find affected copies in your own repositories:

grep -rn 'CHANGESAFE_PR_BODY' .github/workflows/

While you are there, the general shape is worth a wider look: any run: block
containing ${{ github.event.… }} has the same problem, whatever tool it was
copied from. The expression is substituted before the shell parses the script,
so no amount of quoting inside the script protects it. Passing the value
through env: does.

Fixed

The Action no longer lets untrusted text act.

  • The example workflow passes the pull request body through the environment
    instead of interpolating it into a script (above).
  • action.yml does the same with its own inputs, which were interpolated
    into the gate script and would have carried the same power for any input
    assembled from event data.
  • The pull request comment can no longer be forged. UNTRUSTED_INSTRUCTION
    quotes the matched excerpt of the body verbatim, and a | in that excerpt
    opened new Markdown table cells — enough to render a passing verdict beside
    the real one. Characters that carry table structure are now neutralized.

The gate reaches a verdict instead of falling over.

  • A policy that throws is recorded as BLOCK under its own id rather than
    abandoning the evaluation. "All policies fail closed" now holds for a policy
    with a bug in it, not only for one that reaches a considered verdict.
  • Identifiers from outside are looked up as own properties. constructor is
    valid kebab-case, so /devices/constructor/routes/x used to return
    Object.prototype.constructor, pass the existence check, and raise a
    TypeError from inside a policy — printing a raw JavaScript message and
    exiting 2. It is now the typed "unknown device" refusal, exit 1.
  • Terraform address globs are matched directly instead of compiled to a
    regex. Twelve * against a 400-character address used to run past thirty
    seconds, and a gate that never answers is a CI job that never finishes.

The self-hosted server answers honestly.

  • serve gained --approver <subject> and --approver-claim <name>=<value>.
    Authentication says who someone is; these say which of those people may act
    here. A genuine but unlisted identity gets 403, not 401. With neither flag
    the behavior is unchanged and startup says so out loud.
  • A token that could not be decoded returned 500, an oversized body claimed a
    schema mismatch nobody had checked, and malformed JSON returned 500. They
    now answer 401, 413, and 400.
  • GET /decisions?limit=abc reached SQLite as LIMIT NaN and returned 500.
  • Identity-provider key documents are fetched over https only (loopback
    excepted), under a deadline and a size cap, and filtered to signing keys for
    the token's algorithm.

The ledger orders simultaneous decisions. Two approvers deciding at once
both read the same chain head, and the second collided on the sequence number
the first had taken — a legitimate decision returning an internal error.
Appends now queue. The database constraint stays, because it is what catches a
second process.

Live analyses are capped per client. POST /api/analyze is
unauthenticated by design, so a deployment with a provider key was spending
its owner's credit at whatever rate was asked. Live calls now count against a
window (CHANGESAFE_LIVE_RATE_LIMIT, default 10/hour, 0 disables) and
answer 429 before reaching a provider. Replay stays uncapped — it costs
nothing, and the promise that anyone can drive the whole gate without an
account is the demo. The cap is a speed bump, not a defense, and the README
says so: it counts in one process's memory and trusts a forwarded header.

The CLI stopped announcing an experimental database it never opens. The
bundled CLI loaded node:sqlite on every command, so changesafe gate
printed an ExperimentalWarning to stderr on every CI run. It is now loaded
when a ledger is actually opened.

Verdicts did not change

No policy behavior changed for any input that previously produced a verdict.
All nine scenarios match their expectations unchanged, and the published
v0.1.0 verification snapshot still reproduces byte-for-byte
(npm run verify:v0.1.0). CORE_POLICY_VERSION, NETWORK_POLICY_VERSION,
and TERRAFORM_POLICY_VERSION are therefore unchanged, and receipts issued by
v0.1.0 and v0.1.1 remain comparable.

The fail-closed guard is the one addition that can produce a finding that did
not exist before — but only where the alternative was no findings, no risk,
and no receipt at all.

Version strings deliberately did not move

changesafe-cli-0.1.0 still appears as appVersion in CLI receipts, and
package.json still reads 0.1.0. That is not an oversight:
npm run verify:v0.1.0 re-runs the current CLI against the published bundle
and requires the replayed receipt to be canonically identical to the signed
one, and appVersion is inside that receipt. Moving the string would mean the
v0.1.0 snapshot no longer reproduces from this repository.

The honest resolution is that the snapshot should reproduce from the code it
names rather than from the working tree — from the sourceCommit already
recorded in its own provenance.json, since there will never be a v0.1.0
tag to reproduce from. Tagging that commit would publish the pre-fix workflow
as an installable @v0.1.0, which is the opposite of what this release is
for. That change belongs in v0.2.0, not in a patch: it touches the release
verifier, which is exactly the tool a release should not be quietly modifying.

Nothing is published to npm, so no package version is implied by this. The
git tag names the release.

v0 also exists and tracks the newest v0.x. It is the only tag that moves;
v0.1.1 will always mean this commit.

Upgrading

Your situation Do
You copied the example workflow Fix the capture step in your repository (above). Then pin @v0.1.1 — until now there was no tag to pin
You are adopting the Action for the first time Follow the README; it already pins @v0.1.1
The CLI or library, from source Pull main; no API changed
The self-hosted server No change required. Consider --approver / --approver-claim — without them, every identity your issuer vouches for may approve
The app with live mode public Nothing required. Consider CHANGESAFE_LIVE_RATE_LIMIT, and read the README note on what the cap is not

Verification

Checked at the tagged commit, not only on a branch:

git checkout v0.1.1
npm ci
npm run verify:v0.1.0   # the published v0.1.0 snapshot still reproduces
npm test                # 440 passing, 2 skipped
npm run build:cli && node packages/cli/dist/changesafe.js scenario check   # 9/9

The red-team corpus still refuses every proposal it is supposed to refuse.
That is the release gate this project actually cares about, and no change in
this release moved it.