Skip to content

fix(experiments): resolve the sweep harnesses' paths, so a recorded run can be re-run - #265

Merged
eaitbrahim merged 1 commit into
mainfrom
chore/experiment-reproducibility
Aug 13, 2026
Merged

fix(experiments): resolve the sweep harnesses' paths, so a recorded run can be re-run#265
eaitbrahim merged 1 commit into
mainfrom
chore/experiment-reproducibility

Conversation

@eaitbrahim

Copy link
Copy Markdown
Contributor

What

Four sweep harnesses in docs/experiments/ hardcoded two absolute paths — the deployment DB under one laptop's home, and an output dir inside a per-session temp directory that is collected when that session ends. Both now resolve, and both are overridable:

DB      = $KEEL_EXPERIMENT_DB  or ~/keel/keel.db
OUT_DIR = $KEEL_EXPERIMENT_OUT or <script dir>/_out

The default DB resolves to the same physical file it named before, so nothing changes on the machine these were written on — it just stops being the only machine where they work.

Why it matters more than it looks

A recorded experiment that cannot be re-run is the one thing a recorded experiment must not be, and the output path had already gone stale.

_out/ is gitignored but deliberately not temporary: these sweeps are resumable, reading their own append-only JSONL back to skip completed cells, so that file has to outlive the run. A per-session scratch dir is exactly what broke that.

It is also the handoff point between scripts — each *-rsi-scale run reads the JSONL its matching *-intersection run writes, as the ANCHOR supplying its oversold=20 rows. Both chains verified by name, and the two studies verified not to collide (intersection.jsonl vs intersection_257.jsonl).

Only half of this was lintable

Ruff saw two of the four files. The other two spelled the dead path as a multi-line SCRATCH constant — under the line limit, so flagged by nothing. They were found by grepping for the literal rather than trusting the linter's list. The hardcoded DB was invisible to ruff in all four.

Lint cleanup

Also clears the 13 remaining ruff findings under docs/: 6 import blocks (autofix) and 5 long lines rewrapped without changing any output; the other 2 were the path literals above.

ruff check . now passes repo-wide — previously only the CI-scoped ruff check keel tests packages did.

Verification

  • Candles load through the resolved DB path: 44,407 bars for BTC-USD, matching the ~44k those docstrings cite. Run against a copy of the live DB, not the deployment file.
  • Defaults and both env overrides assert-checked, including that the output dir gets created.
  • ANCHOR chain confirmed intact for both study pairs.
  • _out/ confirmed gitignored and absent from git status.
  • mypy clean; pytest 2717 passed, 1 skipped.

🤖 Generated with Claude Code

…un can be re-run

Four harnesses hardcoded two absolute paths: the deployment DB under one laptop's
home, and an output directory inside a per-session temp dir that is collected when
that session ends. A recorded experiment that cannot be re-run is the one thing a
recorded experiment must not be, and the output path had already gone stale.

Both now resolve, and both are overridable:

    DB      = $KEEL_EXPERIMENT_DB  or ~/keel/keel.db
    OUT_DIR = $KEEL_EXPERIMENT_OUT or <script dir>/_out

The default DB resolves to the same physical file it named before, so nothing
changes on the machine these were written on -- it just stops being the only
machine where they work.

`_out/` is gitignored but deliberately NOT temporary: these sweeps are resumable,
reading their own append-only JSONL back to skip completed cells, so that file has
to outlive the run. A per-session scratch dir is precisely what broke that, and it
is also the shared handoff point -- each `*-rsi-scale` run reads the JSONL its
matching `*-intersection` run writes as the ANCHOR supplying its oversold=20 rows.
Both chains were checked by name, and the two studies were checked not to collide
(`intersection.jsonl` vs `intersection_257.jsonl`).

Only two of the four were visible to ruff. The other two spelled the dead path as
a multi-line `SCRATCH` constant, which is under the line limit and so was flagged
by nothing; they were found by grepping for the literal rather than by trusting the
linter's list.

Also clears the 13 remaining ruff findings in `docs/` -- 6 import blocks (autofix)
and 5 long lines rewrapped without changing any output, the other 2 being the path
literals above. `ruff check .` now passes repo-wide; previously only the CI-scoped
`ruff check keel tests packages` did.

Verified: candles load through the resolved DB path (44,407 bars for BTC-USD, the
~44k those docstrings cite); defaults and both env overrides assert-checked,
including that the output dir is created; `_out/` confirmed ignored; mypy clean;
pytest 2717 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@eaitbrahim
eaitbrahim merged commit 15dc41e into main Aug 13, 2026
1 check passed
@eaitbrahim
eaitbrahim deleted the chore/experiment-reproducibility branch August 13, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant