Skip to content

fix(gpu,sdk): governed terminate + content-aware upload skip + export/CLI flag fixes - #654

Merged
jqueguiner merged 3 commits into
mainfrom
fix/gpu-terminate-governed-and-sdk
Jul 29, 2026
Merged

fix(gpu,sdk): governed terminate + content-aware upload skip + export/CLI flag fixes#654
jqueguiner merged 3 commits into
mainfrom
fix/gpu-terminate-governed-and-sdk

Conversation

@jqueguiner

Copy link
Copy Markdown
Owner

Four independent SDK/CLI fixes surfaced during a live GPU session.

1. gpu.terminate — governed server-side kill (cost-safety)

openrunner gpu terminate resolved the provider key client-side, so a platform-stored (governed) key never reached it → No API key found for provider 'runpod', and the pod had to be killed by hand in the RunPod console. Worse, the old code PATCHed the backend status to terminated regardless of whether the provider kill succeeded — stranding a billing pod as falsely-dead (every instance read terminated while one was still alive).

Now: when no explicit api_key is passed, terminate routes through the governed DELETE /gpu/instances/{id} — the API decrypts the org's stored key (never leaves the server), kills the pod, verifies it's gone, and only then marks it terminated. The API already implemented this; the SDK just wasn't calling it. The BYO-key fallback marks terminated only on a confirmed kill.

2. upload_folder(skip_existing=) — compare content, not path

skip_existing compared only the file path against the remote manifest, so a modified file was silently skipped: a dataset update printed Uploaded 0 file(s) and reported success while sending nothing. Now it compares sha256 — changed files re-upload, unchanged ones skip.

3. update_dataset not exported at top level

set_dataset_card was re-exported but update_dataset was not, forcing from openrunner.datasets import update_dataset. Added to the package top-level imports.

4. download-artifact flags

Only --root/-r existed; --project/--output errored (and when masked by || true in a launch script, left an idle pod). Added --output/-o (alias for the download dir) and --project/-p (scope the lookup) to both artifact get and the download-artifact alias.

Test

py_compile clean on all four files; click option aliases verified with CliRunner (-o/--output/-r/--root all map to root, -p/--project parsed). Governed-terminate path verified live: api_client.terminate_instance + get_instance confirm status before returning.

🤖 Generated with Claude Code

jqueguiner and others added 3 commits July 29, 2026 14:24
…/CLI flag fixes

gpu.terminate: terminate through the governed server-side path (DELETE
/gpu/instances/{id}) whenever no explicit api_key is passed. The platform
decrypts the org's stored provider key (which never leaves the API), kills
the pod, verifies it is gone, and only then marks it terminated. The old
client-side path could not see a platform-stored key and failed with
"No API key found for provider 'runpod'", and — worse — it PATCHed the
backend status to "terminated" even when the provider kill failed, stranding
a billing pod as falsely-dead. The BYO-key fallback now marks terminated
only when the provider kill actually succeeded.

datasets.upload_folder: skip_existing now compares content (sha256), not
just path. A pure path match silently dropped modified files, so a dataset
update printed "Uploaded 0 file(s)" and reported success without sending the
changed bytes. Changed files are re-uploaded; unchanged ones are skipped.

sdk: re-export update_dataset at the package top level (parity with
set_dataset_card) so `from openrunner import update_dataset` works.

cli download-artifact / artifact get: accept --output/-o as an alias for the
download dir and add --project/-p to scope the lookup. Previously only
--root/-r existed, so `--project`/`--output` errored — and when masked by a
`|| true` in a launch script that left an idle pod running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
upload_folder now skips only when path AND sha256 match. Update
test_upload_folder_all_skipped to supply the matching sha256, and add
coverage for changed content and missing-remote-sha (both re-upload
rather than silently skip).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
terminate() now routes through the governed DELETE when no explicit api_key
is passed, and never marks a row terminated unless the kill is confirmed.
Rewrite the terminate facade tests accordingly: governed path (confirmed /
not-confirmed / delete-failed), BYO-key client path, and assert no
false-"terminated" PATCH when the provider kill fails.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jqueguiner
jqueguiner merged commit 56252a6 into main Jul 29, 2026
10 checks passed
@jqueguiner
jqueguiner deleted the fix/gpu-terminate-governed-and-sdk branch July 29, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant