0.1.6

Changelog

0.1.6

Fixed — silent partial-failure of s5cmd staging (rc=0 masking)

s5cmd exits rc=0 even when individual per-file copies fail (errors go only to
stderr), so two real failures went undetected:

• Stage-in: a failed per-task input copy (e.g. IncompleteBody on a large
  reference-DB object) left the input missing, the task ran anyway, and a
  downstream step 404'd. Fix: each stage-in s5cmd cp now captures stderr,
  fails the task (non-zero exit) on ERROR/IncompleteBody, and verifies the
  local target exists — aborting the chained stage-in on the first bad input.
• Push-back: the EXIT-trap recursive s5cmd cp ./ <remote> could drop files
  (e.g. versions.yml) while returning rc=0; the success .exitcode was still
  written, so Nextflow proceeded and 404'd on the missing output. Fix: capture
  the push output, detect rc!=0/ERROR/IncompleteBody, and override
  .exitcode to non-zero so the task is marked failed and retried instead of
  proceeding with incomplete outputs.

Detection-only — the copy mechanism, endpoint, and retry behaviour are unchanged.
Test-first: S5cmdRcSafetySpec (characterization + previously-@PendingFeature
specs, now green). See ISSUE-nf-nomad-s5cmd-large-object-staging.

0.1.5

Fixed — inter-task input-staging regression (introduced in 0.1.4)

• 0.1.4 pushed task outputs with s5cmd cp --exclude ".exitcode" ./ <dest> in the
  worker bootstrap. With a non-wildcard directory source, an s5cmd
  exclude-filter suppresses the recursive upload, so the task's outputs never
  reached S3. Every downstream task that consumed an upstream output then failed
  at stage-in with 404 NoSuchKey, making any multi-step pipeline unrunnable.
  (A/B-verified on dylangrblr/Vinotype: 0.1.4 → 19/19 BWAMEM2 stage-in 404s;
  0.1.2 clean.)
• Fix: push outputs with a plain recursive s5cmd cp ./ <dest> (no
  exclude-filter — the proven pre-0.1.4 form), then write + push .exitcode
  strictly LAST, after the output push. The "exitcode-last" preemption
  safety added in 0.1.4 is preserved — a preemption mid-push leaves no remote
  .exitcode, so synchronizeCompletion() returns null and the task is retried
  — it just no longer depends on the broken exclude-filter. _exit_code is now
  initialised early so the EXIT trap is safe under set -u.

0.1.4

Reliability

• Push the remote .exitcode strictly LAST in the worker bootstrap (separate
  s5cmd cp after the outputs push, which now excludes .exitcode). The
  operator-side handler polls for the remote .exitcode and only then pulls
  outputs, so "exitcode present" now implies "outputs already staged". This
  fixes a class of spot/preemption failures where a node was reclaimed mid
  push-back, landing .exitcode = 0 without the output files — the handler
  trusted exit 0 and the task then failed with "Missing output file(s)". With
  the reorder, an interrupted push leaves no remote .exitcode, so
  synchronizeCompletion() returns null and the task is cleanly retried.

0.1.3

Logging

• Standardised all runtime log prefixes to nf-nomad-s5cmd: (was a mix of
  nf-s5cmd: / nf-nomad-s5cmd:). The on-disk nf-s5cmd-task dir name is
  unchanged.
• Quieter by default so plugin logs don't overshadow Nextflow's own stdout on
  the head job or the task's stdout on compute jobs:
  • Demoted lifecycle messages (plugin started, Found <s5cmd>, session complete, swept staged inputs, bin-dir upload, external-S3 staging) from
    info to debug. The one-time active; endpoint=… banner stays at info;
    warnings/errors are unchanged.
  • Changed the default cp.logLevel for s5cmd staging commands from info to
    error, so s5cmd no longer prints a line per transferred file into task
    output. Raise to info/debug via nomad.s5cmd.cp.logLevel when debugging.