Warning
Cadence is alpha. The manifest format is versioned (cadence/v1alpha2), and
older versions are converted when read, but expect breaking changes before
a stable release.
Evolve code you can measure. Mark the part of a program a model may rewrite, name the command that scores it, and Cadence searches for a better version inside your sandbox.
# solve.py -- only the marked region is ever rewritten
# CADENCE:BEGIN
def solve(problem: list[int]) -> int:
return sum(problem)
# CADENCE:END# .cadence
api_version: cadence/v1alpha2
program: solve.py # the file with the marked region
run: python score.py # scores a candidate; prints `score: <number>`
metrics:
score: maximize
budget:
trials: 20
model:
gemini: {}cadence check # what will run, how it is scored, what it will cost; spends nothing
cadence run # propose, measure, keep what improved, repeatWhy it exists, and what happened when it was pointed at problems with published answers: The Harness Was the Experiment.
Every trial goes around the same loop:
propose apply measure select
program ──▶ model ──▶ unified diff ──▶ your command ──▶ objective ──▶ next parent
You bring three things, and Cadence never reaches past them:
| You bring | What it is | Why it is separate |
|---|---|---|
| A marked program | The file, with CADENCE:BEGIN / CADENCE:END around the part that may change |
Everything outside the markers stays fixed, so every trial is comparable |
| A scoring command | Anything that runs a candidate and prints numbers | It lives outside the region, so the model cannot edit it. A scorer that imports the candidate shares its process, and the candidate can still print a score: run it as a separate process |
| A manifest | .cadence: metrics, budget, model, sandbox limits |
One file states the whole experiment, and its hash is recorded with every result |
What comes back from each trial is a verdict, not just a number. A candidate that crashed, timed out, ran out of memory, or printed no metric is recorded as that, with the reason. A score is a score, and a failure is never a zero.
| You want to... | Cadence gives you |
|---|---|
| Know what a run will do before paying for it | cadence check: the region, the baseline score, whether it repeats, the sandbox, the cost |
| Keep the model away from how it is judged | The region markers, and a scoring command outside them |
| Run candidates with limits | A sandbox per trial: wall clock, memory, output size, a clean environment |
| Use your own model, or a hosted one | Gemini, OpenAI, Anthropic or Ollama, set in .cadence. The key comes from the environment, never from a file |
| See what happened, step by step | A JSON log of every step in cadence-runs/, with or without a database |
| Read a run without reading the database | cadence dashboard: every trial, what it changed, and whether the search is still improving |
| Stop today and carry on tomorrow | Recorded runs resume from the database, and calls already paid for are replayed, not repeated |
| Keep the winner | cadence apply writes a run's best program over yours |
uv tool install git+https://github.com/yash-srivastava19/cadenceThis puts cadence on your PATH, so it works from any directory. It needs
uv. If your system Python is newer than
Cadence supports, uv fetches a compatible one.
To keep runs in a database (step 6), you will also need a clone of this repository. The database migrations live there.
In an empty directory:
cadence init wrote .cadence
wrote solve.py
wrote score.py
wrote IMPROVE.md
Now run `cadence check .`.
| File | What it is |
|---|---|
solve.py |
The program. Only the code between the markers changes |
score.py |
Runs solve and prints score: <number>. The model never sees it as editable |
.cadence |
The manifest |
IMPROVE.md |
Sent with every prompt: what the code does, what may change, what may not |
To use Cadence on your own code, replace solve.py's marked region with
yours and make score.py measure it. See Bring your own program.
init writes model: {gemini: {}}. Set its key in the environment:
export GEMINI_API_KEY=... # or GOOGLE_API_KEYTo keep everything on your machine, use Ollama instead:
model:
ollama:
model: <a model you have pulled>See Models for the others.
cadence check manifest cadence/v1alpha2, 7 defaults applied
region solve.py lines 9-17 (9 lines the model may rewrite)
scoring reported from outside the region
method evolution built with size=8, tournament=3
model gemini, model gemini-3.6-flash
objective weighted_sum over score to maximize
baseline `python score.py` exited 0 in 25ms
metric score = 31, and maximize is better
sandbox 3 seeds per trial, 10s and 256MB each
repeatable scored the same twice, and nothing declares it -- set verifier.tolerance to let cadence reuse a score
cost 20 trials x 3 seeds x 25ms is about 1s of scoring
guidance IMPROVE.md will be sent with every prompt
recording nothing will be recorded -- DATABASE_URL is not set
ready. `cadence run` will spend up to 20 trials.
check runs your baseline once and calls no model. If something would stop a
run, like a missing key or a metric that is never printed, it says so and
does not say ready.
cadence runProgress goes to stderr, one line per trial. The result goes to stdout: the
winner, its metrics, and what the run spent. Pipe it (cadence run > result.json) or read it on the terminal.
Without a database, that output is the only copy of the winner. Keep it, or set up recording first.
Every run also writes a log of each step to cadence-runs/, with or without
a database. See Logs.
Cadence includes a small Claude Code plugin that teaches an agent how to design the editable region, scorer, manifest, and validation strategy before it spends model calls. It does not start runs or apply candidates without confirmation.
From Claude Code:
/plugin marketplace add yash-srivastava19/cadence
/plugin install cadence@cadence
/reload-plugins
Then ask Claude Code to help use Cadence on your problem. The plugin reads the
current documentation, and its source is in
plugins/cadence. The docs also publish llms.txt,
a full Markdown corpus, and a machine-readable skills catalog for other agent
clients.
Recording needs a Postgres. Point cadence at any one you have, and let it create its own schema:
export DATABASE_URL=postgresql://user:password@host:5432/cadence
cadence db status # what is there, and what a run would make of it
cadence db upgrade # create the schema, and update it after upgrading cadenceIf you have no Postgres to hand, one container is enough:
docker run -d --name cadence-pg -p 5433:5432 \
-e POSTGRES_USER=cadence -e POSTGRES_PASSWORD=cadence -e POSTGRES_DB=cadence \
postgres:18-alpine
export DATABASE_URL=postgresql://cadence:cadence@localhost:5433/cadence
cadence db upgradeThen, from your project, with DATABASE_URL still set:
cadence check # now says the run will be recorded
cadence run # prints a run id
cadence runs list # every recorded run, newest first
cadence trials list RUN_ID # the trials of one run
cadence run --resume RUN_ID # carry on a run that stopped
cadence apply RUN_ID # write the best program over yours
cadence dashboard # read all of it in a browserapply overwrites your program and keeps no backup. Commit first.
Where the schema is owned by one role and runs connect as another, give the
migration its own URL and leave DATABASE_URL alone:
cadence db upgrade --url postgresql://owner:.... cadence db rollback
undoes a migration, and asks first because it can drop tables.
Cadence also reads a .env file in the current directory. Variables that
are already set in the environment take precedence.
Mark the region. Put the markers around the body of what should change, and keep the signature the rest of the program calls:
def pack(items, capacity):
# CADENCE:BEGIN
return []
# CADENCE:ENDPrint the metrics. The scoring command prints numbers on stdout, with
names that match metrics: in .cadence. Any of these is read:
{"value": 45, "weight": 18}
value: 45
value = 45
Score outside the region. If the region prints its own score, the model
can rewrite that line. check warns when it sees this.
Make a wrong answer score badly, not crash. A crash tells the run the code is broken. A low score tells it the answer was poor. They are different lessons.
Tell the model what matters in IMPROVE.md: constraints, what may not
change, and ideas worth trying.
Then run cadence check until it says ready.
init writes the short form. Every field, with its default where it has one:
api_version: cadence/v1alpha2
program: solve.py # the file with the marked region
run: python {program} # the scoring command; {program} is replaced
metrics: # required: name -> maximize | minimize
score: maximize
experiment: my-question # unset by default; groups runs in a shared database
guidance: IMPROVE.md # sent with every prompt
markers:
begin: "CADENCE:BEGIN"
end: "CADENCE:END"
method: # how the next parent is chosen
evolution: {size: 8, tournament: 3}
pick: # unset by default: the method's best is the winner
confirm: {seeds: [100, 101], top: 3} # score the top 3 again on new seeds
judge: # unset by default: score each candidate on every seed
cascade: {seeds: [0], above: 0.5} # score on one seed first; the rest only if promising
objective: # unset by default: a weighted sum over metrics
weighted_sum: {score: 1.0}
model: # scripted | gemini | openai | anthropic | ollama
gemini: {}
prompt:
template: region # region | rewrite | improve
hints: [try a different strategy entirely] # unset by default: the built-in rotation
budget:
trials: 20
usd: 5.0 # unset by default; enforced only where a price is declared
sandbox:
seconds: 10 # wall clock per run of the scoring command
memory_mb: 256
seeds: [0, 1, 2] # each trial is scored once per seed
verifier:
tolerance: 0.0 # unset by default; declares the score repeatable, so it can be reusedcadence schema prints the JSON Schema. Point your editor at it for
completion and validation.
Each trial copies the project into a temporary directory, writes the
candidate over program, and runs run there with the seed in
CADENCE_SEED. The command gets a clean environment: your keys are not
passed to it.
| Name | Key from | Notes |
|---|---|---|
gemini |
GEMINI_API_KEY or GOOGLE_API_KEY |
Default model gemini-3.6-flash |
openai |
OPENAI_API_KEY |
Name a model: openai: {model: ...} |
anthropic |
ANTHROPIC_API_KEY |
Name a model: anthropic: {model: ...} |
ollama |
none | Local. OLLAMA_HOST overrides the address |
scripted |
none | Canned answers, for tests. It cannot run from the CLI |
Providers are defined in
providers.yml. Every provider
speaks the OpenAI dialect, so adding one is a row. Put prices and local
overrides in providers.local.yml, which is gitignored and read from the
directory you run cadence in. With a price declared, a run reports what it
cost.
A provider cadence does not ship, added in providers.local.yml:
providers:
groq:
base_url: https://api.groq.com/openai/v1
key_from: [GROQ_API_KEY] # read from the environment, never a file
model: openai/gpt-oss-120b
attempts: 3 # tries per call
backoff: 30.0 # seconds, times the attempt number
request: {max_tokens: 4096, reasoning_effort: medium}request adds fields to every request body, for what one provider needs and
the dialect has no word for. It cannot replace model, messages or
temperature, which cadence sends itself.
A rate limit or an outage is retried. A daily quota that is used up, or a
request too large for the model, is not: the run stops and says so, and
cadence run --resume RUN_ID carries on once the quota resets.
cadence init [DIR] # write a project that check already passes
cadence check [DIR] # everything about a run, without running it
cadence run [DIR] # run the search
cadence run --config FILE # use FILE instead of DIR/.cadence
cadence run --resume RUN_ID # carry on a recorded run
cadence runs list # recorded runs
cadence runs show RUN_ID # one run in full
cadence trials list RUN_ID # the trials of a run
cadence trials show TRIAL_ID # one trial in full, and what it changed
cadence apply RUN_ID [DIR] # write the winner over your program
cadence schema # the manifest's JSON Schema
cadence db status # the database, and whether its schema is current
cadence db upgrade # apply the migrations this cadence needs
cadence db rollback --to -1 # undo the last one; asks first
cadence dashboard [--port N] # the recorded runs, in a browsercheck, run, runs and trials take --json. It is the default when
output is not a terminal.
cadence dashboard # http://127.0.0.1:8787Reads the same database as cadence runs list. Read-only, and bound to
localhost: there is no authentication, and every manifest and candidate
program is readable through it.
It works out what a table of numbers leaves to you.
| It shows | So you do not have to |
|---|---|
| The direction the manifest declared | Remember whether 8.26 beats 9.11 |
| The best score and the trial it arrived at | Scan the column |
| The difference from the baseline, with a percentage | Subtract |
| Scored trials since the last new best | Judge whether the search is still finding anything |
| Each trial against the trial it was patched from | Trace the tree by hand |
| Whether the search ended or the provider stopped it | Read a 429 as a bad result |
Every cadence run writes one file, one JSON object per line, one line per
step:
cadence-runs/
├── .gitignore # "*": the folder ignores itself, so logs are never committed
└── development/ # the environment, from CADENCE_ENV (default: development)
└── <run-id>.jsonl
The run prints the path when it ends. A line looks like this:
{"ts":"2026-09-21T12:50:41.380Z","level":"info","run":"20260921-124926-4dfc96","trial":"20260921-124926-4dfc96/0","span":"model_call","phase":"end","status":"ok","message":"model replied: gemma3:4b, 850 in / 49 out, 74.3s","id":"20260921-124926-4dfc96/0/call-1","parent":"20260921-124926-4dfc96/0","trace_id":"…","span_id":"…","parent_span_id":"…","duration_ms":74341.1,"attrs":{"tokens_in":850,"tokens_out":49,"response_hash":"00c824a847b25b65"}}| Field | What it is |
|---|---|
level |
info; warn when something cost a call and bought nothing (an unusable reply, a rejected patch); error when the run cannot go on |
span |
The step: run, baseline, trial, model_call, read_reply, apply_patch, measure or remeasure (a program scored again) |
phase |
start and end of a step, or event for a single moment |
status |
How the step ended: ok, scored, failed:crashed, unusable, abandoned… |
message |
The line as a sentence |
trace_id, span_id, parent_span_id |
OTEL-shaped IDs: one trace per trial, one span per step. They're derived from the run and trial IDs, so they stay the same across a resume |
attrs |
The step's own numbers and identifiers |
Your code, the model's replies and the prompts are never written to the log, only their hashes. The text of a crash (the last lines of stderr) is kept, because the file stays on your machine.
L=$(ls -t cadence-runs/development/*.jsonl | head -1)
jq -r '.message' $L # the run as sentences
tail -f $L | jq . # follow a run live
grep '"level":"warn"' $L # what cost something and bought nothing
jq -r 'select(.span=="trial" and .phase=="end") | "\(.trial) \(.status) \(.duration_ms)ms"' $L
jq -s 'map(select(.span=="model_call" and .phase=="end")) | map(.attrs.tokens_in + .attrs.tokens_out) | add' $LTo keep runs from different setups apart, set the environment:
CADENCE_ENV=staging cadence run writes to cadence-runs/staging/.
The repository includes a small knapsack lab and a script that drives it with canned model answers:
git clone https://github.com/yash-srivastava19/cadence && cd cadence
uv run python examples/lab/demo.pyThe first answer improves the score. The later ones are malformed on purpose, so you can watch Cadence retry the model, reject what it cannot use, and carry on.
- Reasoning models served through Ollama's OpenAI endpoint can return an empty reply, because the answer arrives in a separate reasoning field.
- A model that repeats the
CADENCE:BEGIN/CADENCE:ENDlines in its answer ends the run instead of failing one trial. - The sandbox limits time, memory and output, and passes a clean environment. It does not restrict network access, or files outside the working copy. Run it where that is acceptable.
The design, and why it is shaped this way: Cadence.
cadence/
core/ DTOs, verdicts, identities, ports
lifecycle/ state machines
observe/ run signals
parsing/ metric reading
control/ choose parents, call models, parse replies, apply patches, record
execution/ run candidates in a sandbox and return what happened
delivery/ how results are presented
web/ the dashboard: read-only, GET only, localhost
commands/ the CLI
| Plane | Owns | Does not know |
|---|---|---|
control |
search, prompts, model calls, patches, objectives, the record | how a candidate is run |
execution |
sandboxes, limits, verdicts, metric collection | why a candidate was chosen |
delivery |
how results are shown | how search or sandboxing works |
web |
serving the record to a browser | anything control has not already worked out |
web is a sibling of commands, not a layer under it. Both open a session,
ask control what happened, and hand the answer to delivery, so the
terminal and the browser cannot disagree about whether a trial improved.
The layers are enforced by import-linter in CI. To add a search method,
implement the Method port. To change how candidates are scored, implement
Judge; to change how the winner is chosen, Pick. To add a sandbox,
implement Sandbox. To watch a run, subscribe to its signals.
git clone https://github.com/yash-srivastava19/cadence && cd cadence
make install # uv sync with the dev and test tools
make test # pytest; the database tests need TEST_DATABASE_URL
make lint # mypy, ruff, import-linter
make fmtSee CONTRIBUTING.md.
MIT. See LICENSE.
