Skip to content

keel v0.5.2

Choose a tag to compare

@github-actions github-actions released this 07 Aug 05:09
· 307 commits to main since this release
79f35b9

Built from 79f35b9. Version binds to this hash:
keel --version reports keel 0.5.2+79f35b9e73d5 [release].

Install

Download all wheels from this release into one directory, then install the
keel_trader wheel by path:

pip install --find-links . ./keel_trader-0.5.2-py3-none-any.whl
keel --version

⚠️ Never install by bare name. The distribution is keel-trader; the name
keel on PyPI belongs to an unrelated project, so pip install keel fetches
someone else's package. A build reporting DIRTY or [checkout] is not this
release and must not be run against live funds.

Configure

config.yaml is attached to this release: the production config, in
auto_trade.mode: confirm — keel previews every order and waits for your
approval. Drop it beside the install (or run keel init-config --live), put
your CDP key in a git-ignored .env, then:

keel migrate     # existing database: apply schema migrations
keel init        # fresh deployment: write config + seed candidate rules

Seeded rules start as candidate and trade nothing until you promote them.

Other changes

docs: correct the run script and plist claiming a headless cycle places nothing (#171)

Comments only. Both keel-live-run.sh and com.keel.live.plist documented a safeguard that is not in force on the live deployment.

What was wrong

Both files assert the scheduled cycle cannot place an order:

keel-live-run.sh: "Because confirm mode FAILS CLOSED with no TTY … this NEVER places an order and moves no money"
com.keel.live.plist: "The detector still places nothing: confirm mode fails closed with no TTY"

That holds only with autonomy OFF. The live deployment has been running with it ON.

agent._effective_mode returns "autonomous" when config.auto_trade.mode == "confirm" and the profile row is autonomous — read fresh from the database every cycle, so it is not a config or deploy-time property at all. In that mode the confirm gate is skipped and the order is placed headlessly. keel-live.log has been recording agent.mode_resolved mode=autonomous on every cycle.

This is the kind of comment that costs something: it is exactly what an operator (or an agent reading the repo) consults to answer "can this thing trade without me?", and the answer it gave was wrong.

What the comments say now

  • Autonomy is named as the thing that decides, with both branches spelled out.
  • Where to read the setting actually in force: tui --once prints the autonomy line, and each cycle logs agent.mode_resolved.
  • keel autonomy off applies on the next cycle; keel kill is what stops trading immediately.
  • guards.check runs first and is un-overridable under both settings — autonomy changes who is asked, never what is allowed.
  • The old wording is called out explicitly, so it is not restored by someone who remembers the file rather than the profile row.

Verification

  • 0 non-comment lines changed (checked mechanically against the diff).
  • plutil -lint passes; the plist still parses to the same twelve :05 triggers.
  • bash -n keel-live-run.sh passes.
  • 1902 tests pass, ruff clean.
  • paperforward-run.sh's "places NOTHING real" is deliberately left alone — it is accurate, since paper routes upstream and never reaches an executor mode.

Anchor the live detector's schedule to UTC (~13h of lag removed) (#172)

What changed

The supervised-live detector now runs on the UTC clock. It evaluates the freshly-closed daily bar at 01:20 UTC instead of 09:05 America/New_York, cutting the delay between a daily candle closing and the agent acting on it from ~13 hours to ~1h20m.

No Python behaviour changed. This is the schedule (keel-live-run.sh, com.keel.live.plist), plus tests that pin what the schedule is actually for.

Why 01:20, and not midnight

Daily candles close at 00:00 UTC, but the agent cannot see the closed bar at that moment. turtle_breakout._completed_days withholds the newest daily bar until the 00:00–01:00 UTC hourly bar has closed — a guard that exists to stop the account simulator consuming a still-forming day as if it were complete. So 01:00 UTC is the earliest instant a cycle sees fresh data, and SCHED_HOUR=1 is exactly that boundary.

The extra 20 minutes is margin for Coinbase to publish that hourly candle and for data.market_feed to persist it. A trigger at exactly 01:00 UTC would race the very bar it exists to consume.

The day-stamp is a correctness mechanism, not notification hygiene

The runner's day-stamp file is the only thing preventing the live money path from entering the same daily signal twice. Its previous comment described it as avoiding a duplicate notification, which was wrong and made it look optional. Nothing downstream dedupes an entry:

  • get_open_positions gates exits, reconciliation and status — never entry.
  • The signals table is written but never read back.
  • client_order_id is a fresh uuid4() on every call, so the exchange cannot dedupe either.
  • The rails in execution/guards.py are dollar caps and account state, not per-day entry counters — a second entry inside the caps passes all of them.

The paper path does gate (strategy/paper.py refuses a second entry while a product is open). The live path does not. Two cycles in one UTC day means two entries off one daily bar.

Corollary worth knowing: a manual keel agent run bypasses the script, and therefore the stamp, entirely.

The invariant

For any UTC date X, the newest visible daily bar is constant — it is X-1 — across the entire eligible window [01:00 UTC, 24:00 UTC). So whichever eligible trigger fires first on UTC date X evaluates bar X-1 and stamps X; every later trigger that UTC day is a no-op. Every daily bar is evaluated exactly once: no missed day, no double day.

That only holds because the stamp and the gate are now on the same clock. A local date straddles two UTC dates, so the old local-anchored gate could run twice within one UTC day on the catch-up path (machine powered off until late in the local day, so the first available trigger is already the next UTC date, and the following local morning runs again on that same UTC date). That is pinned as a regression test.

Why 24 triggers is not a cadence change

The plist goes from 12 triggers to 24 (hourly at :20). launchd re-runs a missed StartCalendarInterval on wake from sleep, but not when the trigger passed while the machine was powered off — which silently cost a full day on 2026-07-28. The only defence is another trigger later. The trigger count is catch-up breadth, not cadence; the stamp is what keeps 24 triggers to one cycle. Catch-up window: 12h → 23h.

Because these are local hours and the gate is on UTC, every UTC date gets a trigger in every UTC hour, so an eligible one always exists — including across both DST transitions.

Incidental fix: the plist was not well-formed XML

This repo separates asides with a double hyphen. XML forbids one inside a comment. plutil -lint reported OK because Apple's CoreFoundation parser is lenient and launchd never complained, but plistlib/expat rejected the file outright — so any ordinary XML tooling could not read it. The comment is rewritten without one, and a test keeps it parseable.

DCA budget: rule, config and simulator now agree

The live executor sizes DCA from config.dca.budget_usd and ignores the rule row's own budget_usd. The live config said 50, the rule row said 25 — so 50 was what actually moved, while the account simulator (which does read the rule's value) modelled 25.

50 is the intended budget. deploy/live-rules.json now records 50, matching both the config and the Dca constructor default. The divergence is documented at the point where it bites, and the manifest test now asserts rule-vs-config agreement rather than pinning a literal — a hardcoded number there would just re-create the drift one deploy later.

Operator action still required. This does not change the live database. scripts/rule_manifest.py apply deliberately never rewrites an existing rule's params; it reports drift and exits 1. The rule row must be updated by hand:

sqlite3 ~/keel/keel-live.db \
  "UPDATE rules SET params = json_set(params, '\$.budget_usd', '50')
   WHERE kind='dca' AND json_extract(params,'\$.product_id')='BTC-USD';"

Note the quoted '50'. Decimal params are stored as JSON strings; json_set(..., 50) writes a number and the manifest still reports drift. Verify with:

python scripts/rule_manifest.py apply --db keel-live.db --manifest deploy/live-rules.json
# -> in sync: 6 rule(s) match deploy/live-rules.json

Tests

New tests/test_schedule.py (20 tests) and two characterization tests in tests/test_agent.py:

  • test_effective_bar_does_not_advance_until_0100_utc — parametrised over 00:05, 00:59, 01:00, 01:20, 02:20 and 13:05 UTC; pins the 01:00 boundary the whole schedule is built on.
  • test_exactly_one_run_per_utc_day_over_a_full_year — pure simulation of the gate over 13 months of real triggers spanning both DST transitions. Exactly one run per UTC date, every run in [01:00, 24:00), no date skipped. Run against America/New_York plus a southern-hemisphere zone and a half-hour-offset zone, to show the property comes from the schedule's structure rather than a lucky offset.
  • test_spring_forward_... / test_fall_back_... — the two days most likely to be wrong. Spring-forward loses a trigger (23 that day); fall-back repeats local 01:20 at two distinct UTC instants on the same UTC date (25 that day), which is precisely why the stamp must be UTC-keyed.
  • test_the_old_local_date_gate_could_double_run_within_one_utc_day — the regression the change exists to prevent.
  • test_two_cycles_in_one_utc_day_place_two_dca_orders and ..._reenter_the_same_turtle_breakout — these assert the duplicate, deliberately. They document the hazard the stamp guards. If a real live-side entry dedupe is ever added they should go red, and the right response is to rewrite them to assert a single entry, not delete them.
  • Shell-level tests run the real script in a sandbox (its deployment root rewritten, with an assertion that no reference to the live path survives) to check the stamp is written on a clean exit and not written on a nonzero exit.

The plist's trigger list is parsed, never hardcoded, so the tests and the plist cannot drift.

Suite: 1902 → 1924 passing. ruff and mypy clean.

Not in scope

Build only — nothing is deployed. paperforward-run.sh is deliberately left local-anchored: the paper path already refuses a second entry while a product is open, so a duplicate cycle there is inert.

test(manifest): keep the rule-manifest test able to catch a keel init reseed (#173)

What this is now

This PR started life as the deploy/live-rules.json DCA budget_usd 25 -> 50 change, split out of #172. #172 was merged and carried that change in, so the value edit, the executor.py comment and the prose are all on main already. Rebased onto origin/main, what remains is the one thing #172 did not land: repairing the test it silently weakened.

The problem

#172 replaced this assertion in test_committed_manifest_is_valid:

assert dca[0]["params"]["budget_usd"] == "25", "DCA budget must not revert to the default 50"

with a check that the manifest's budget equals config.dca.budget_usd. Checking agreement rather than a literal is the right instinct — a hardcoded number there just re-creates the drift it is meant to catch, one deploy later.

But it lost something. The original assertion guarded a specific, real failure: keel init on a fresh box seeds every rule at candidate from each rule kind's constructor defaults, silently discarding any parameter an operator tuned by hand, on a box that otherwise looks correctly provisioned. Nothing errors.

50 is also Dca.__init__'s constructor default. And deploy/live-rules.json's DCA params are now exactly the four constructor defaults:

param manifest Dca.__init__ default
cadence_days 7 7
budget_usd "50" Decimal("50")
dip_bonus_pct "0" Decimal("0")
lookback_days 90 90

So "manifest agrees with config" is satisfied both by a correctly-provisioned deployment and by one that keel init quietly reseeded. The values cannot tell those two states apart any more.

Why I did not do what was asked literally

The instruction was to assert "the value is not merely the constructor default". That is impossible to satisfy honestly — the operator's intended budget is the constructor default. Any assertion of that form would either be false or would require moving Dca's default away from 50 to suit a test, which changes library semantics for a test's convenience. So the guard is restored a different way.

What the test asserts now

  1. Agreement — manifest budget_usd == config.live-sandbox.yaml's dca.budget_usd, parsed from the file, not hardcoded. This is the value the executor actually spends.
  2. Not seed-shaped — the DCA rule's status is live, and no rule in the manifest is candidate. This is the only discriminator assertion (1) leaves standing: keel init always seeds at candidate regardless of what the params say.
  3. The coincidence is pinned, not assumed — the manifest's DCA params are asserted equal to Dca(product_id=...).describe()["params"]. If the operator ever moves the budget off the default, this is what fails, and its message explains that the failure is good news: assertion (1) has regained the ability to catch a reseed on its own, and (2) is no longer the last line of defence.

Verification

Each assertion was confirmed to fail for the right reason, by mutating the committed manifest and reverting:

  • DCA status -> candidate: AssertionError: the live DCA rule is not status=live -- if this is a candidate, keel init likely reseeded it from Dca's constructor defaults...
  • budget_usd -> "25": AssertionError: the live DCA rule's budget_usd must match config.dca.budget_usd... assert Decimal('25') == Decimal('50')
  • cadence_days -> 14: trips assertion (3) with the "reasoning behind assertion (2) no longer applies" message.

ruff, pytest and mypy clean.

Make the 01:20 UTC detector safe: gate entries on a confirmed bar, and stop losing the day-stamp (#174)

Follow-up to #172, from an adversarial review of it. #172 verified the clock-side invariant correctly — 9624 simulated firings, 402 runs on 402 distinct UTC dates, both DST transitions — but it traded a ~13-hour data-publication margin for 20 minutes, and nothing measured whether the data had actually arrived. This PR closes that, plus three ways the day-stamp could be lost.

Nothing is deployed. ~/keel still runs the old local-anchored script, so there is no live exposure today — but a release cut from main as it stands would create it.


Finding 1 (HIGH) — a late candle re-evaluates yesterday's bar, placing a duplicate order

turtle_breakout._completed_days withholds the just-closed daily bar until the 00:00–01:00 UTC hourly bar closes. Verified against the real function, at 01:20 UTC on UTC date X:

feed state effective newest daily bar
both series current X-1 — correct
ONE_HOUR one bar late X-2 — regressed
ONE_DAY not yet stored X-2 — regressed
(at the old 13:05 UTC slot, even 12 bars of hourly lag) X-1 — harmless

X-2 is the bar yesterday's 01:20 UTC cycle already evaluated. Nothing on the live path dedupes an entry — get_open_positions gates exits/reconcile/status but never entry, the signals table is never read back, client_order_id is a fresh uuid4, and the rails are dollar caps, not per-day counters. So that re-evaluation is a duplicate real-money order, not a delayed one.

Why nothing caught it: market_feed.poll_once writes nothing and raises nothing when the venue has no new bar; run_once set last_feed_ts = now_ts unconditionally, so rail 12 — which measures whether the poller ran, not whether data arrived — was blind; and market_feed.is_fresh is measured against the finest configured granularity (FIFTEEN_MINUTE live), never ONE_DAY. The cycle exited 0, the runner stamped the day, and there was no retry.

The fix

data/freshness.py already had expected_last_ts and the bars_behind arithmetic and zero references from agent.py, execution/ or strategy/. New entry_bar_ready() uses them: the gated series must be bars_behind == 0, and every finer configured series must have advanced past that bar's close.

That second condition is _completed_days's own condition stated generically, so the gate cannot read "ready" while _completed_days would still withhold the bar. It is also no stricter than it must be — it only asks that the finer series crossed the boundary, not that it is at its own newest bar, so an hourly series five bars late at 14:20 UTC still confirms fine. A bars_behind == 0 requirement on FIFTEEN_MINUTE would have left only 5 minutes of margin at the 01:20 trigger and blocked entries routinely.

Deliberately not reusing assess()/DEFAULT_TOLERANCE_BARS: that 2-bar tolerance exists so an operator-facing staleness alert does not fire on the normal forming-bar lag. A one-bar-late hourly series is exactly what duplicates an order, and the alert tolerance would wave it through.

Where the gate lives, and why not rail 12

In run_once, not as a rail. Two reasons:

  1. A rail cannot fail the cycle. Rails yield placed=False and a zero exit. The runner would still stamp the day, and the fresh bar would never be evaluated at all — trading a duplicate order for a silently missed day. The gate has to fail the cycle so the runner declines to stamp.
  2. A rail only runs when a signal fires. On the ~99% of days no signal fires, a stale bar would still be stamped as "done", and the real bar would go unevaluated.

Fixing rail 12 instead would be actively dangerous. Its threshold is interval_sec * 3 and it answers "did the poller run". Making last_feed_ts advance only on new data would, in a once-daily deployment, leave it ~24h stale every cycle and veto everything.

Semantics
  • Entries only. Exits still run for every rule, unfiltered — an open position's rule-driven channel exit runs in-process (the protective stop rests at the broker, the channel exit does not), and holding a losing position an extra cycle is strictly worse than a delayed entry.
  • Gated on the granularity the rule trades on. _entry_gate_granularity falls back to the coarsest configured granularity, diverging from engine._trading_granularity's finest-fallback on purpose: Dca declares no timeframe yet reads candles_by_tf[ONE_DAY] directly and keys its cadence off that bar's timestamp, so gating it on FIFTEEN_MINUTE would miss the same hazard entirely.
  • Nonzero exit. Single-cycle keel agent exits DATA_NOT_READY_EXIT (4), so the runner declines to stamp and one of the remaining 23 triggers retries an hour later — duplicate order → ≤60 minutes of delay. --loop is unchanged: it skips the cycle and tries again next interval.
  • Structured log. agent.entry_bar_not_ready names the granularity, expected ts, stored ts, bars_behind, reason and blocking series.

Findings 2/3/4 — three ways the day-stamp could be lost

Finding 2 (HIGH) — a failed stamp write was swallowed. Reproduced: with a read-only logs dir the cycle ran, the stamp was absent, rc was 0, and the next trigger ran a second full cycle. Two layers now:

  • Pre-flight, before keel is invoked at all — probe that the stamp is persistable, and refuse to run a cycle if not. This is the layer that matters. Exiting nonzero after a cycle has run does not prevent the duplicate: the order is already placed. The only way to turn "duplicate real order" into "no trading plus a loud alert" is to refuse to trade when we cannot record that we traded.
  • Atomic write + read-back afterwards, as belt and braces. > "$STAMP" truncates first, so a torn write left an empty stamp — which reads as "never ran" and re-runs the day.

Finding 3 (MED) — empty date -u output disabled the detector permanently. On this hardware $((10#$(date -u '+%H'))) on empty input evaluates to 0, not an error, so an empty clock gave TODAY="", HOUR=0; a missing stamp also reads "", so [ "" = "" ] was true — "already ran", forever, with no alert. The unmodified script literally printed detector already ran this UTC day () -- skipping. Now validated before use (exit 64, nothing run, stamp untouched).

Finding 4 (MED) — the stamp compare was =, so a clock rollback re-evaluated a bar. A Mac booting with a bad RTC before NTP settles (RunAtLoad fires immediately) reads a past date, which != treats as "not today". Now strictly-less-than, making the stamp monotonic. A malformed stamp is refused (exit 65) rather than compared — "garbage" < "2026-08-06" is false and would read as "already ran" forever.

Also: the "N signal(s) PENDING — run the agent interactively" notification now fires only on a clean cycle. Per the script's own COROLLARY, running the agent by hand bypasses the stamp, so prompting for it off a failed cycle's partially-parsed output pointed the operator straight at a duplicate entry. An ordinary nonzero keel exit deliberately does not notify — it is expected and self-healing, and alerting 23 times a day would train the operator to ignore the alerts that do need a human. The policy is written into the header, along with exit codes 64/65/66.

Finding 7 — "no missed day" was stated flatly in the header and plist; it is conditional on the machine being powered on for at least one eligible trigger that UTC day. Both now say so.

Finding 10 — the PENDLOG line logged local, unlabelled time next to lines that are all UTC.

Finding 8 (flock) — documented, not fixed. flock is not installed on macOS (verified), launchd will not start a job already running, and the race that actually matters is a manual run, which bypasses the stamp regardless of any lock.


Finding 6 — the tests were testing a model, not the artifact

_run_gate re-implemented the shell gate in Python, so the year-long simulation proved nothing about the shipped script. Worse, _skip_before_sched_hour made the only two real-shell tests silently skip whenever CI ran between 00:00 and 01:00 UTC — on the sole barrier to duplicate orders.

  • The skip is gone. Every real-script test now shims date on PATH and injects its own instant.
  • The model is pinned to the artifact. Driving the real script over all 9624 triggers would add ~2.5 minutes to a 10-second suite, so the model is kept — but test_the_simulated_gate_matches_the_real_script replays the real script over a curated adversarial sequence (normal days, the sub-SCHED_HOUR trigger, both DST transitions including the twice-fired local 01:20, and the boot-after-outage catch-up) and requires it to agree with the model at every trigger.
  • _stored_series no longer hardcodes instantaneous, never-failing publication — the assumption that made this whole bug class invisible. It takes hourly_lag_bars/daily_lag_bars, and a new test pins the premise against the real _completed_days.

Verification

Every new test was confirmed to fail for the right reason before the fix.

  • Neutering only the entry gate (engine.evaluate(ready_rules, …)product_rules) makes all 5 new agent/CLI tests fail, with the turtle rule emitting a real ENTER Signal on ts=777600 — bar X-2, the already-traded bar.
  • Running the new schedule suite against the unmodified script: 10 failed, 21 passed, including rc=0 despite a "Permission denied" stamp write, and detector already ran this UTC day () -- skipping on an empty clock.

Independently re-verified end-to-end in a sandboxed copy of the script (live path asserted absent):

scenario cycles run rc notified
empty date -u 0 64 yes
read-only logs dir 0 (was 2, rc=0) 66 yes
happy path 1 0
second trigger, same UTC day 1 0
clock rolled back 200 days 1 0
clock corrected 1 0
next UTC day 2 0
corrupt stamp 0 65 yes
failed cycle printing signals=2 1 7 no PENDING prompt, no stamp

Stamp never truncated: forced write failure left yesterday's 2026-06-14 intact.

Gate: ruff clean · mypy clean (94 files) · 1953 passed (from 1924 on main).

Suite time went 10.3s → 19.5s. ~2.4s of that is the differential test; the rest is the 11 per-property real-script tests. That is the honest price of Finding 6 — the alternative is deleting artifact-level coverage of the only barrier against duplicate real-money orders.

What this does not do

It is not an entry dedupe. The day-stamp remains the only thing preventing two cycles in one UTC day from entering twice, and the two characterization tests in tests/test_agent.py still pin that hazard (they are unaffected — their config declares only ONE_DAY, so there is no finer series to confirm against). This PR removes the specific way a late candle could make the stamp's "one run per UTC day" guarantee re-evaluate an already-traded bar.

fix(tests): unbreak CI on Linux, and bump to 0.5.2 (#175)

Two commits: a CI fix that main needs right now, and the 0.5.2 version bump that depends on it.

1. main is red — the real-script tests are macOS-only

#174 added tests that execute the shipped keel-live-run.sh under /usr/bin/sandbox-exec, so a test run can never fire a real notification on a machine that also trades real money. The harness also uses BSD date -r and chflags uchg. None of those exist on Linux, so all 45 invocations died with FileNotFoundError: /usr/bin/sandbox-exec on the ubuntu runner, and main's push CI has been failing since #174 merged.

This is not a coverage question worth solving portably. The artifact under test is a macOS deployment — launchd plus osascript — and the script only ever runs on the box that owns the launchd job. So on non-Darwin the real-script tests skip, while the schedule invariant (exactly one cycle per UTC date, across both DST transitions) stays covered on every platform, because that is proven by the pure-Python model tests.

One runtime guard in _run_script covers all 45 call sites, since every invocation already funnels through it. The five tests that reach for chflags before running the script carry an explicit @_macos_only mark, where the runtime guard would come too late.

This blocks everything else: release.yml refuses to publish unless tests pass, so 0.5.2 could not be cut with main in this state.

2. Bump to 0.5.2

Patch — no schema change, no new capability. Ships #169, #172 and #174.

#174 is why this release exists rather than #172 alone. #172 retimed the live detector to 01:20 UTC, cutting ~12h of detection lag, but traded a 13-hour data-publication margin for 20 minutes — and nothing on the live entry path dedupes an order. #174's freshness gate in run_once withholds all entries whenever any bar is unconfirmed, and it must ship in the same wheel as the schedule it protects.

That makes the deployment order strict: install this wheel first, then copy keel-live-run.sh / com.keel.live.plist. The shell files alone are the unsafe half. The plist also goes from 12 triggers to 24, so it needs a launchctl reload, unlike the comment-only change in #171.

uv.lock relocked in the same commit, verified with uv sync --frozen.

Gate: 1976 tests pass locally, ruff clean, mypy clean.