Skip to content

Releases: compgenlab/batchq

v0.4.1

Choose a tag to compare

@github-actions github-actions released this 24 Jul 21:17
38e910a
v0.4.1 — SLURM hostless orphaned-claim recovery (--slurm-recover-host…

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 21 Jul 01:32
661b9ee

Reliability and UX release focused on HPC deployments backed by networked filesystems (NFS/Lustre), plus a complete cleanup/archive story and SLURM-aware queue views.

Cleanup & archiving

  • batchq cleanup --delete | --archive [name] — permanently remove, or move completed jobs into a new read-only archive SQLite DB (per run), then remove them from the live DB. --older-than age filter; automatic --vacuum afterward (--no-vacuum to skip); --vacuum alone just compacts.
  • Server-side bulk cleanup/archive that scales to a large backlog (hours → minutes on a slow FS), with live streamed progress and a concurrency guard against overlapping runs.
  • status / details / search --archives fall back to the archive DBs for jobs no longer in the live DB.

Reliability on NFS/Lustre

  • Idempotent submit (client-assigned job id) — a filesystem latency spike that times a request out is now retried transparently instead of failing the job.
  • Resilient SLURM handoff — MarkJobProxied / EndProxiedJob are idempotent and retried through transients, so a stalled DB write after an (irreversible) sbatch no longer loses the SLURM id and orphans the job.
  • Every generated sbatch script stamps #SBATCH --comment="{\"batchq_id\":\"…\"}" onto the SLURM job record, so a rescue can map a SLURM job back to batchq.
  • Fixed an HTTP WriteTimeout that cut off long streaming responses mid-flight.

SLURM scheduling

  • Generic per-job scheduler passthrough (--custom key=value, e.g. account/partition) and resource.* → --gres / --constraint / --nodelist sbatch directives.

Queue UX

  • batchq queue --state/-s RUNNING,QUEUED — filter to specific states (case-insensitive, comma-separated).
  • Job arrays collapse into a single summary row by default (--expand-arrays, or --array-id to drill in), with an array <done>/<total> · <status counts> breakdown.
  • The queue shows the effective SLURM status (PENDING/RUNNING/…) for proxied jobs and array tasks — not just PROXYQUEUED — plus SLURM array ids.

Operational notes

  • On a networked-filesystem deployment, keep [server] read_pool_size = 1 (the single-writer-connection invariant that prevents self-SQLITE_BUSY holds only at 1).
  • First server start after upgrading builds a new (status, end_time) index on an existing large DB — a one-time cost.

Full changelog: v0.3.8...v0.4.0

v0.3.8

Choose a tag to compare

@github-actions github-actions released this 12 Jul 13:10
a48c459
v0.3.8

v0.3.7

Choose a tag to compare

@github-actions github-actions released this 08 Jul 19:26
de137f4
- feat(priority): apply top/nice across a whole array (#49)

v0.3.6

Choose a tag to compare

@mbreese mbreese released this 08 Jul 16:14
0079fca

Features

  • web: combined UI + REST API gateway (batchq web --api) — the browser UI at / (HTTP Basic, admin + [web] password) and the REST API reverse-proxied under /api/ (bearer token) on one listener; the proxy forwards to the same backend the web process is a client of, preserving the single-DB-owner invariant. Missing gateway credentials are generated (crypto/rand) and persisted to the config file on TCP listeners. (#47)
  • slurm: store the submitted sbatch script on array tasks, so batchq details <array-task-id> shows the exact --array=<indices>%N script the task was submitted with — matching single-job behavior — instead of the raw job body. Correct across drip-fed batches. (#48)

Fixes

  • list: defer hydration on detail-filtered listings. (#46)

v0.3.5

Choose a tag to compare

@github-actions github-actions released this 02 Jul 03:07
cdf3757
v0.3.5

v0.3.4

Choose a tag to compare

@github-actions github-actions released this 01 Jul 13:16
09b3f9a
v0.3.4: add --since/--before submit-time filters to queue

v0.3.3

Choose a tag to compare

@github-actions github-actions released this 11 Jun 17:21
ede9629
v0.3.3: normalize job names so SLURM submission can't break

v0.3.2

Choose a tag to compare

@github-actions github-actions released this 10 Jun 07:59
7181946
v0.3.2 — add `batchq backup` command (VACUUM INTO snapshot)

v0.3.1

Choose a tag to compare

@github-actions github-actions released this 10 Jun 05:06
93cfb6a
- SLURM proxy runner array-batch gates:
  - --slurm-min-array N / [slurm_runner] min_array: defer array batches
    smaller than N (avoid a flurry of tiny sbatch --array submissions);
    the array's final remainder is always submitted so the tail is never
    stranded. Clamped to the effective job cap.
  - --slurm-full-array / [slurm_runner] full_array: all-or-nothing — only
    submit an array that fits one pass entirely; overrides min_array.
- Docs: "Running batchq" section in CLAUDE.md (server, clients, runners)
  and the new [slurm_runner] knobs in docs/configuration.md.