Epic/04 advanced runner#4
Merged
diegoparrilla merged 10 commits intomainfrom May 2, 2026
Merged
Conversation
…ding
The runner blob now relocates itself out of cartridge ROM into RAM
at runner_entry, mirroring the pattern gemdrive_init already uses.
Target address is gemdrive_reloc - $1000 (4 KB below GEMDRIVE's
relocated copy with $400 slack between them). _memtop drops to the
new low watermark so TOS' allocator doesn't walk over either blob.
The whole module was already PC-relative so labels resolve
correctly against the relocated location; the cartridge-ROM source
is unused after the copy.
runner_post_reloc is the old runner_entry body — Dsetdrv,
Dsetpath, banner, poll loop. After the cwd setup but before the
HELLO send_sync, it now installs a VBL hook at $70 in supervisor
mode (or.w #$0700, sr to mask interrupts during the swap), saving
the old vector into the relocated blob's old_vbl cell. The HELLO
payload extends to 4 bytes carrying d3 = "advanced installed"
flag.
The VBL handler stub (adv_vbl_handler) saves d0/d1/a0, reads the
sentinel, masks to the $06xx range (APP_RUNNER_VBL = $0600),
dispatches inside that range (no commands wired yet — S2), and
chains to the saved $70 handler regardless. XBRA marker
'XBRA''SDRA' so debuggers can identify our hook in the chain.
RP side wires up the HELLO payload read into emul.c's
runnerAdvancedInstalled flag, plus a new
GET /api/v1/runner/adv endpoint that surfaces it via
{ok:true, active, installed}. CLI exposes
`sidecart runner adv status` (human form + --json passthrough);
3 new RunnerAdvStatusTests.
Cartridge size: 9184 / 10240 bytes (1056 free).
The m68k VBL handler (installed at $70 by S1) now dispatches RUNNER_ADV_CMD_RESET ($0601) into adv_force_reset, which mirrors the foreground runner_reset body — clr.l of the three memvalid cookies ($420 / $43A / $51A) followed by a jmp through the reset vector at $4.w. No register restore (the cold reset wipes them), no rte (control transfers permanently to TOS' init; SR's IPL is reset by TOS early enough that entering from inside an interrupt is fine). The point of this path is escape-from-wedged-foreground: when a Pexec'd program has the cartridge bus tied up in an infinite loop, sidecart runner reset never reaches the m68k poll loop because the poll loop isn't running. The VBL keeps firing regardless, so the adv reset always works. RP-side handle_runner_adv_reset deliberately doesn't gate on emul_isRunnerBusy() — the busy lock is exactly what we're escaping. Otherwise mirrors the foreground reset's bookkeeping: records RUNNER_LAST_RESET, fires RUNNER_ADV_CMD_RESET on the sentinel, schedules the +3 s relaunch ticker so the runner sticks across the cold reset. CLI exposes "sidecart runner adv reset". 2 new RunnerAdvResetTests (happy path / 409 runner_inactive). Cartridge size: 9212 / 10240.
… etv_timer)
A program that does
move.l #my_handler, $70.w
without saving the previous value destroys our hook completely —
nothing we install at $70 survives. Tested case: a game whose
runtime fully replaces the VBL vector and never chains. Defeats
the S2 path entirely.
Workaround: hook a different vector that hostile programs are far
less likely to wipe. TOS' etv_timer ETV vector at $400 fires every
200 Hz from the MFP timer-C handler and is much harder for a
program to replace wholesale because TOS' keyboard / mouse / sound
subsystems all chain through it — wholesale replacement breaks the
program before it can lock anything else.
New macro ADV_HOOK_VECTOR in runner.s selects the vector at build
time. Two predefined values (ADV_HOOK_VECTOR_VBL = $70,
ADV_HOOK_VECTOR_ETV_TIMER = $400). Default flipped to etv_timer.
Same handler body services both — the chain pattern
"move.l old(pc), -(sp); rts" works for both an autovectored
interrupt (rts -> chained handler -> rte) and an ETV-style
subroutine (rts -> chained handler -> rts back to TOS' MFP ISR
which then rte's). The ETV register-scratch convention (d0/d1/a0/a1
free, others preserved) is a strict superset of what we save.
Renamed adv_vbl_handler -> adv_hook_handler and old_vbl ->
old_adv_hook to reflect that the hook can be either vector. XBRA
marker 'XBRA''SDRA' retained — debuggers walking either chain see
our hook. Cartridge size: 9212 / 10240.
The build-time ADV_HOOK_VECTOR macro from S3 is now an aconfig
setting (ACONFIG_PARAM_ADV_HOOK_VECTOR, default "etv_timer")
editable from the main setup menu via a new [V] toggle that cycles
between "vbl" and "etv_timer". RP-side gemdrive_command_cb HELLO
handler resolves the string to the vector address ($70 or $400)
and publishes it into shared-var slot 16
(RUNNER_SVAR_ADV_HOOK_VECTOR). m68k's runner_post_reloc reads slot
16 indirectly into a2 at install time and writes the relocated
adv_hook_handler address there; the build-time ADV_HOOK_VECTOR
macro stays as a fallback for older RP firmware that doesn't
publish the slot.
The HELLO payload extends from "1 bit installed" to a packed byte
layout: bit 0 = installed, bits 8..15 = hook_vector_id (0=vbl,
1=etv_timer, 0xFF=unknown). m68k populates the byte from the
adv_active_vec cell it stashed at install time. RP-side records
both fields via emul_recordRunnerAdvancedInstalled +
emul_recordRunnerAdvHookVector; emul_resetRunnerSession clears
both back to "false / unknown".
GET /api/v1/runner/adv JSON gains a "hook_vector" field
("vbl"/"etv_timer"/"unknown"). CLI prints "hook vector :
installed (etv_timer @ $400)" or the equivalent VBL variant. One
new RunnerAdvStatusTests case covers the VBL variant; existing
cases updated for the new field.
Cartridge size: 9276 / 10240 bytes.
…L path The foreground RUNNER_CMD_RESET dispatch in runner_poll_loop and the runner_reset body it pointed at are gone. Cold reset is now exclusively the Advanced Runner VBL handler's job (RUNNER_ADV_CMD_RESET, $0601). Eliminates the duplicated memvalid-clear-and-jmp-through-$4.w sequence and ~94 bytes of cartridge code. POST /api/v1/runner/reset now writes RUNNER_ADV_CMD_RESET to the sentinel instead of the retired RUNNER_CMD_RESET. The CLI verb "sidecart runner reset" therefore picks up the wedge-recovery property the dedicated "adv reset" had — works even when a program has the foreground poll loop wedged. Intentionally no busy-lock gate; escaping the busy state is the whole point. handle_runner_adv_reset, the /api/v1/runner/adv/reset route, cmd_runner_adv_reset, the "runner adv reset" CLI subcommand, and the corresponding RunnerAdvResetTests (2 cases) are all removed. "sidecart runner reset" is now the One Way to reset; "sidecart runner adv status" stays as the diagnostic for confirming which hook vector is active. RUNNER_CMD_RESET constant removed from runner.h with a comment reserving APP_RUNNER + 0x01 so future stories don't reuse the slot. Cartridge size: 9182 / 10240 bytes. 71 CLI tests still pass.
Same RP-visible payload as the foreground meminfo (24-byte struct shipped via RUNNER_CMD_DONE_MEMINFO, recorded by the existing runner_command_cb chandler), but the read happens inside the m68k VBL ISR rather than the foreground poll loop — so it works against wedged programs that hold runnerBusy set forever. m68k: new RUNNER_ADV_CMD_MEMINFO ($0603) dispatched in adv_hook_handler; local .adv_chain promoted to global adv_chain_to_old so the new ISR handler can branch into the chain; new adv_meminfo_isr saves d2-d7/a1-a4 on top of the ISR prologue, builds the 24-byte struct on the stack, reads $432/$436/$42E/$44E/$4F2 and the $FFFF8001 MMU nibble (same cascade as runner_meminfo), ships via send_write_sync, restores extras, and falls through to adv_chain_to_old so TOS' VBL chain keeps firing. No Super() toggle (already supervisor in autovector entry), no Cconws trace (GEMDOS-from-ISR is a hazard). RP: new RUNNER_ADV_CMD_MEMINFO mirror; RUNNER_MEMINFO_TIMEOUT_US hoisted above its first user; new handle_runner_adv_meminfo — synchronous spin (≤ 1 s timeout), same JSON envelope as the foreground variant, intentionally no busy-lock gate. Route adv/meminfo wired. CLI: sidecart runner adv meminfo prints the same human form as runner meminfo. 3 new RunnerAdvMeminfoTests. Cartridge size: 9382 / 10240 bytes.
Patch the m68k VBL ISR's saved PC so the eventual rte resumes at
a user-supplied address. Fire-and-forget recovery / debugging
primitive — the dev can hand the CPU off to any 24-bit even
address (cartridge runner_entry, TOS' init vector, a hand-loaded
PRG entry, etc.) without going through a cold boot.
m68k:
- New RUNNER_ADV_CMD_JUMP ($0604) dispatched in adv_hook_handler.
- New ack RUNNER_ADV_CMD_DONE_JUMP at the start of the new
m68k -> RP report range APP_RUNNER_VBL_DONE ($0680). Disjoint
from the foreground RUNNER_CMD_DONE_* range so the receiver path
is never ambiguous.
- New shared-var slot SHARED_VAR_ADV_JUMP_ADDR (17). RP writes the
resolved u32 there before firing the sentinel; the m68k reads it
indirectly into d0.
- adv_jump_isr body:
- VBL-only guard via adv_active_vec; etv_timer chains via
adv_chain_to_old (no-op).
- movem.l d2-d7/a1-a4, -(sp) saves the 10 extras send_sync may
clobber.
- Stack layout while in the patch:
sp + 0..39 extras (10 regs * 4 = 40 B)
sp + 40..51 ISR prologue (d0-d1/a0 = 12 B)
sp + 52..53 trap-frame SR (word)
sp + 54..57 trap-frame PC (long) <- patched
(initial draft had this off-by-4 — 50(sp) clobbered the saved
a0 + SR while leaving the PC intact, so the rte never actually
jumped. Fixed.)
- send_sync RUNNER_ADV_CMD_DONE_JUMP, 0 — round-trips through the
random token; the chandler clears the sentinel so subsequent
VBLs see NOP and just chain.
- movem restores extras + ISR prologue, rte pops the patched PC
and the user's code resumes at the target.
- Hook stays installed (B2); status keeps reporting installed: true
(F1) until the next runner restart.
RP:
- Mirrored RUNNER_ADV_CMD_JUMP / APP_RUNNER_VBL_DONE /
RUNNER_ADV_CMD_DONE_JUMP / RUNNER_LAST_JUMP (= 6).
- runner_command_cb adds a case for RUNNER_ADV_CMD_DONE_JUMP that
just SEND_COMMAND_TO_DISPLAY(0).
- runner_last_command_str extended with "JUMP".
- New handle_runner_adv_jump for POST /api/v1/runner/adv/jump:
- 409 runner_inactive when not active.
- 409 wrong_hook when ADV_HOOK_VECTOR != vbl.
- 411/415 standard JSON-body precondition checks.
- JSON {"address":"<int>"} extracted as string;
strtoul(s, NULL, 0) accepts decimal and 0x-hex.
- Validation: address <= 0xFFFFFF AND even, otherwise 400
bad_request with a specific message.
- SET_SHARED_VAR(17, addr, ...) writes slot 17 before firing
RUNNER_ADV_CMD_JUMP on the sentinel.
- emul_recordRunnerCommand(RUNNER_LAST_JUMP, ...).
- Route adv/jump wired. memfunc.h include added so SET_SHARED_VAR
resolves.
CLI:
- _parse_adv_jump_address(raw) accepts decimal / $hex / 0xhex,
rejects empty / out-of-24-bit / odd at parse time.
- cmd_runner_adv_jump posts {"address":"0xHEX"} (always
normalised to the modern form) and prints
"ok ADV JUMP 0xHHHHHH sent (VBL ISR rte)".
- New `runner adv jump <ADDRESS>` subparser.
- 7 new RunnerAdvJumpTests: decimal / $hex / 0xhex / reject-odd /
reject-out-of-range / 409 wrong_hook / 409 runner_inactive.
Cartridge size: 9462 / 10240 bytes. 81 CLI tests pass.
…[SIZE]) Stream a workstation file straight into m68k RAM through the VBL ISR — no GEMDRIVE round-trip, no Pexec, no FatFs path on the m68k. Wire shape: POST /api/v1/runner/adv/load?address=<int>[&size=<int>] with the file as the request body. The HTTP handler drains pbuf segments byte-pair-swapped into APP_FREE at RUNNER_ADV_LOAD_BUF (8 KB chunk staging at offset \$4000). When a chunk fills, the handler publishes target/length in shared-var slots 18/19 and fires RUNNER_ADV_CMD_LOAD_CHUNK (\$0605); the m68k's adv_hook_handler matches the new code, copies the chunk to the destination, and acks via RUNNER_ADV_CMD_DONE_LOAD_CHUNK (\$0681). The handler spins chandler_loop with a 1 s per-chunk timeout, advances the cursor, and pumps the next chunk. The final partial chunk is flushed at end-of-body. Validation order: 409 runner_inactive → 409 wrong_hook (VBL only; etv_timer's outer trap-frame layout is past TOS' MFP scratch) → 411/413 body precondition → query parse (address required, size optional, both decimal/\$hex/0xhex) → 400 bad_request for odd or out-of-24-bit address → lazy meminfo fetch → 400 ram_overflow if [start, start+len) escapes [membottom, phystop). Cap-and-truncate: size argument shrinks Content-Length, never grows it. No busy-lock gate — chunked + ISR-driven means there is no foreground operation to gate on, and the whole point of the Advanced surface is to keep working when foreground is wedged. CLI: sidecart runner adv load <local-file> <address> [size] prints \`ok ADV LOAD <file> -> 0x<addr> (<n> bytes)\` on success. Address parser is the S7 helper; the optional size accepts the same three formats. Workstation-side cap-and-truncate keeps the wire body honest when size < file_size. Hard limits + reservations: - Cartridge code now 9554 / 10240 bytes. - New shared-var slots: 18 (load target), 19 (load length). - New runner_last_command_t enum value RUNNER_LAST_LOAD = 7. - Reuses APP_FREE 8 KB buffer carved out at offset \$4000.
…ump+load CLI User hit `python cli/sidecart.py runner adv load FONT.PI1 \$78000` → "argument required: address". Cause: bash/zsh expand bare `\$78000` as a variable reference (unset → empty → silently dropped from argv), not the CLI rejecting the input. The `_parse_adv_jump_address` helper already accepts `\$hex`; the user just needs to single-quote it. Documented in three places that surface to a user hitting this: - `_parse_adv_jump_address` docstring (canonical address parser; any future address-accepting command inherits this warning) - argparse `help=` strings for `runner adv jump`'s `address`, `runner adv load`'s `address`, and `runner adv load`'s `size` (all three accept the `\$hex` form) — visible in `--help` - `docs/epics/04-advanced-runner.md` S7 + S8 rows (design-doc record; file is gitignored locally so the change doesn't reach this commit but lives alongside the rest of the epic spec) Recommendation in the help text: prefer `0xhex` in scripts. No code behaviour change.
…README) Final story of the Advanced-Runner epic. No code change — the audit of every runner write_error envelope (10 endpoints, ~50 call sites) confirmed the validation order and code vocabulary are already consistent across the foreground + advanced surfaces. The polish that remained was documentation drift. docs/api.md - Error code vocabulary: add `wrong_hook` and `ram_overflow`; rewrite the Runner-specific paragraph as a bullet list that distinguishes which codes apply where. - Runner-mode preamble: replace the inaccurate "All Runner endpoints return 409 busy" claim. Only the four foreground gating endpoints (run/cd/res/meminfo) gate on busy; reset and the entire Advanced surface intentionally skip the busy gate, because escaping wedged state is exactly what they exist to do. Add a forward pointer to the new Advanced Runner section, and a \$hex shell-quoting callout. - GET /api/v1/runner: extend `last_command` enum to `JUMP` and `LOAD` (added by S7 + S8 — the schema field was already correct in the C source but the doc was stale). - run/cd/res Common error codes: 409 busy → 503 busy (was a status- code typo, the code+phrase emitted is `503 busy`). - New Advanced Runner section after the foreground meminfo block: `GET /api/v1/runner/adv`, `POST /api/v1/runner/adv/meminfo`, `POST /api/v1/runner/adv/jump`, `POST /api/v1/runner/adv/load`. Curl + sidecart for each; explicit VBL-only / wrong_hook notes on jump and load; cap-and-truncate semantics + ram_overflow / 504 semantics on load. README.md - Add an "Advanced Runner" subsection under Runner mode listing the four sidecart commands and noting the VBL-hook requirement + \$hex shell-quoting rule. Pointer back to docs/api.md for the full reference. Smoke test (acceptance criterion): every command form documented in docs/api.md and the README parses cleanly on a fresh shell, builds the right URL, and reaches the network — verified by running each against an unreachable host and confirming the failure was at the network layer, not the parser. Wire-level coverage is in cli/test_sidecart.py (87/87 tests green). Closes Epic 04.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.