Repository navigation
Satchel v0.2.0
Pre-release
Pre-release
Satchel 0.2.0
Eval suites can now run on a schedule, and a GitHub Action previews context repo pull requests,
runs the publish gate on them and publishes on merge.
Status: pre-alpha. Expect breaking changes between 0.x releases.
Install
uv tool install satchel-server # or: uv tool upgrade satchel-server
satchel init --example jaffle
satchel upContainer image and Helm chart:
docker pull ghcr.io/getsatchel/satchel:0.2.0
helm install satchel oci://ghcr.io/getsatchel/charts/satchel --version 0.2.0 -n satchel --create-namespace -f my-values.yamlThis release ships with satchel-adapter-sdk 1.2.1 (IR 1.0.0).
New
- Scheduled eval runs. Add
schedule: { interval_hours: N }to a suite andsatchel worker
runs it against the latest published release every N hours, on its own thread
(SATCHEL_EVAL_TICK_MIN, default 15;0turns it off). The Suites tab shows the schedule and
the next run, Run history labels scheduled and gate runs, and a scheduled run links
Compare with previous. A suite that needs an LLM is skipped while no model is configured.
See Evals. - GitHub Action (
getsatchel/satchel/action@v0.2.0). On a pull request it uploads the context
folder, gets a preview, runs the gate's suites, comments the comparison and fails the check on
a regression. On a push to the default branch it imports and publishes. See
GitHub Action. - Context repo uploads.
POST /v1/sync/preview(curator) builds a pull-request preview without changing drafts or
what is served. Such previews are published by merging, not from the UI.POST /v1/sync/import(admin) issatchel sync importover HTTP. Sources declared in the
config are not synced from an upload.- Both calls are audited as
sync.previewandsync.import.
- Evals documentation. Evals covers suites, graders, runs, the
publish gate and schedules.
Changes
- Eval suite files whose names start with
.(for example macOS._*metadata) are ignored on
import. - New context versions are numbered after the highest existing one, so they never collide with
versions a pull-request preview created. - CI runs only the jobs a pull request can affect (contributors).
Upgrading from 0.1.1
uv tool upgrade satchel-server
satchel db upgradeThe migration 0013 adds release.ref. Run satchel db upgrade (or let the Helm chart's
migration job run) before starting the new version. The worker starts the eval schedule thread
by default; suites run on a schedule only once you add schedule: to them, so nothing changes
until then.