You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#50 made the scheduler layer classify an affirmative job-id rejection as JobIdRejectedError (a SchedulerError subtype), and status now reports it distinctly: an unknown id gets the Run not found report, and a known run whose recorded job id is rejected prints Job <job_id>: the scheduler rejected this job id (run <run_id> metadata may be stale or corrupt) with exit 1.
The other commands that query the scheduler with a recorded job id still conflate the deterministic rejection with transient states:
job-output --follow: the pre-tail status check swallows the rejection via its generic except SSHError and falls through to tail -F on an output path that can never exist, hanging indefinitely.
These paths are reachable only with stale or corrupt run metadata (recorded job ids come from scheduler acceptance, so a live submission never produces them), so #50 was deliberately scoped to status, the one command that also accepts raw user-supplied ids; this issue extends the distinct report to the remaining commands.
Proposal
Let wait / submit --wait distinguish a rejection: either surface JobIdRejectedError from wait_for_job to the CLI, or return a distinct terminal marker, and print the same rejected-id diagnosis status gives (exit non-zero, without clobbering run.status).
Make job-output --follow's pre-tail status check treat JobIdRejectedError as fatal instead of falling through to tail -F.
Reuse the recorded rejection fixtures in tests/rejection_fixtures.py for the new tests, mocking at the SSHManager layer.
#50 made the scheduler layer classify an affirmative job-id rejection as
JobIdRejectedError(aSchedulerErrorsubtype), andstatusnow reports it distinctly: an unknown id gets theRun not foundreport, and a known run whose recorded job id is rejected printsJob <job_id>: the scheduler rejected this job id (run <run_id> metadata may be stale or corrupt)with exit 1.The other commands that query the scheduler with a recorded job id still conflate the deterministic rejection with transient states:
waitandsubmit --wait: since status: run-id-shaped argument from the wrong project root raises SSHError instead of the Run not found report on Slurm #50,wait_for_jobreturnsUNKNOWNimmediately on the first poll for a rejection, so the wait loop terminates rather than hanging — but the CLI printsfinal state unknown (scheduler stopped returning data), a transient-sounding message that misdirects the user into retrying, and overwritesrun.statuswithunknown.job-output --follow: the pre-tail status check swallows the rejection via its genericexcept SSHErrorand falls through totail -Fon an output path that can never exist, hanging indefinitely.These paths are reachable only with stale or corrupt run metadata (recorded job ids come from scheduler acceptance, so a live submission never produces them), so #50 was deliberately scoped to
status, the one command that also accepts raw user-supplied ids; this issue extends the distinct report to the remaining commands.Proposal
wait/submit --waitdistinguish a rejection: either surfaceJobIdRejectedErrorfromwait_for_jobto the CLI, or return a distinct terminal marker, and print the same rejected-id diagnosisstatusgives (exit non-zero, without clobberingrun.status).job-output --follow's pre-tail status check treatJobIdRejectedErroras fatal instead of falling through totail -F.tests/rejection_fixtures.pyfor the new tests, mocking at theSSHManagerlayer.