Skip to content

v0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 21 Jul 01:32
· 7 commits to main since this release
v0.4.0
661b9ee
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Reliability and UX release focused on HPC deployments backed by networked filesystems (NFS/Lustre), plus a complete cleanup/archive story and SLURM-aware queue views.

Cleanup & archiving

  • batchq cleanup --delete | --archive [name] — permanently remove, or move completed jobs into a new read-only archive SQLite DB (per run), then remove them from the live DB. --older-than age filter; automatic --vacuum afterward (--no-vacuum to skip); --vacuum alone just compacts.
  • Server-side bulk cleanup/archive that scales to a large backlog (hours → minutes on a slow FS), with live streamed progress and a concurrency guard against overlapping runs.
  • status / details / search --archives fall back to the archive DBs for jobs no longer in the live DB.

Reliability on NFS/Lustre

  • Idempotent submit (client-assigned job id) — a filesystem latency spike that times a request out is now retried transparently instead of failing the job.
  • Resilient SLURM handoff — MarkJobProxied / EndProxiedJob are idempotent and retried through transients, so a stalled DB write after an (irreversible) sbatch no longer loses the SLURM id and orphans the job.
  • Every generated sbatch script stamps #SBATCH --comment="{\"batchq_id\":\"…\"}" onto the SLURM job record, so a rescue can map a SLURM job back to batchq.
  • Fixed an HTTP WriteTimeout that cut off long streaming responses mid-flight.

SLURM scheduling

  • Generic per-job scheduler passthrough (--custom key=value, e.g. account/partition) and resource.* → --gres / --constraint / --nodelist sbatch directives.

Queue UX

  • batchq queue --state/-s RUNNING,QUEUED — filter to specific states (case-insensitive, comma-separated).
  • Job arrays collapse into a single summary row by default (--expand-arrays, or --array-id to drill in), with an array <done>/<total> · <status counts> breakdown.
  • The queue shows the effective SLURM status (PENDING/RUNNING/…) for proxied jobs and array tasks — not just PROXYQUEUED — plus SLURM array ids.

Operational notes

  • On a networked-filesystem deployment, keep [server] read_pool_size = 1 (the single-writer-connection invariant that prevents self-SQLITE_BUSY holds only at 1).
  • First server start after upgrading builds a new (status, end_time) index on an existing large DB — a one-time cost.

Full changelog: v0.3.8...v0.4.0

Assets 6
Loading