Skip to content

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 13 Nov 08:15
· 7 commits to main since this release
b714311

Per-job environment variables

  • lanq run --env NAME=VALUE (repeatable; single runs, --shards, --dry-run) sets a variable for this job only.
    Shards get it too. The split is at the first =, and a name may be given only once (names are compared ignoring
    case).
  • The developer decides which names jobs may set: machines.yml policy.allowed_env, exact names or prefixes ending in
    * ([CONFIGURATION, TEST_FILTER, "MYAPP_*"]). It is empty by default, so every --env is refused until the
    developer lists names. A name not listed is Rejected (2) with an ask_developer step.
  • A hard deny table applies on top, even to listed names, because these variables can make an allowlisted command run
    other code or move where tools keep their files: the loader and shell startup names machines.yml already refused,
    PATH, Node (NODE_OPTIONS, NODE_PATH), .NET (DOTNET_STARTUP_HOOKS, DOTNET_ROOT*, CORECLR_*, COR_*, ...),
    Java, Python, Perl and Ruby startup and path variables, the Go toolchain and module settings and CC/CXX/CGO_*,
    Rust (RUSTFMT, RUSTUP_TOOLCHAIN, RUSTC, RUSTDOC, RUSTFLAGS, RUSTDOCFLAGS, RUSTC_WRAPPER,
    RUSTC_WORKSPACE_WRAPPER, CARGO_ENCODED_RUSTFLAGS, CARGO_HOME, CARGO_TARGET_*, CARGO_BUILD_*,
    CARGO_REGISTRIES_*, CARGO_SOURCE_*), build tools (JAVA_OPTS, GRADLE_USER_HOME, GRADLE_OPTS, MAVEN_OPTS,
    MAVEN_ARGS,
    MAKEFILES, MAKEFLAGS, DOTNET_HOST_PATH, NUGET_PLUGIN_PATHS, COMPLUS_*, DOTNET_MSBUILD_SDK_RESOLVER_*),
    TLS trust and proxies (NODE_TLS_REJECT_UNAUTHORIZED, NODE_EXTRA_CA_CERTS, SSL_CERT_FILE, SSL_CERT_DIR,
    REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, NO_PROXY), package indexes
    (PIP_*, UV_INDEX_URL, UV_EXTRA_INDEX_URL, UV_INDEX, UV_DEFAULT_INDEX, UV_FIND_LINKS, UV_PYTHON), OpenSSL and glibc module paths, profile and system folders (HOME, USERPROFILE, APPDATA, TEMP,
    TMP, XDG_CONFIG_HOME, ...), IFS, CDPATH, and every name starting GIT_, SSH_, NPM_CONFIG_, YARN_,
    MSBUILD, LD_, DYLD_ or LANQ_. NODE_ENV, GOOS and GOARCH stay allowed. An allowed_env entry the
    table fully covers (GIT_*, HOME, PIP_*, CARGO_BUILD_*, CARGO_REGISTRIES_*, CARGO_SOURCE_*) makes the
    config fail to load; a prefix that only
    partly overlaps (NODE_*, UV_*, CARGO_*) loads, and lanq doctor prints an informational OK line (allowed_env) saying those names stay refused, for
    example policy.allowed_env CARGO_*: names on lanq's deny list (CARGO_ENCODED_RUSTFLAGS, CARGO_HOME, CARGO_TARGET_*, ...) stay refused.
  • A deny-table name on the command line is a fix_command step ("this variable can never be set by a job; drop it or
    ask the developer: ..."); from lanq.yml it is ask_developer ("the developer drops it from that job's env: in
    lanq.yml").
  • Each value must be non-empty and free of control characters, at most 4,096 characters, and 16,384 for all of a job's
    variables together (NAME=VALUE each). Values are never echoed in a refusal.
  • A machine whose own env: in machines.yml sets the same name is not used for the job (env_conflict), and neither
    is a Windows machine on which lanq's largest job script plus the values would pass 8,000 characters
    (windows_command_too_long). Both are permanent and judged from the config. When every capable machine clashes,
    the run ends at once with NoCapacity (75, retryable: false) and an ask_developer step ("ask the developer:
    they can drop --env CONFIGURATION or remove it from the machine's env (build-01, big-01)"); when the length rules
    out the last ones, it is Rejected (2), naming each clashing machine too, with an ask_developer step ("they can
    shorten or drop the --env values, or change machines.yml"; for a plan, "that job's env: values in lanq.yml"). An
    agent stops and reports these; it never edits the config or drops variables to get through.
  • lanq.yml gains env: in a job and in defaults:, a mapping of NAME: value. It is merged by name, the job's value
    winning: the one exception to "a job's value replaces the default". Each entry becomes an --env flag.
  • Variables go after the machine's env: and before lanq's own LANQ_*, quoted as data like the machine's.

Exclusive jobs

  • lanq run --exclusive (lanq.yml exclusive: true) takes every slot of the machine (max_jobs), all or nothing, so
    no other lanq job, from you or anyone else, runs there while it does.
  • It waits while any live slot is in use (busy, retryable: "busy (slots 1/4; --exclusive needs all 4 free)"). There
    is no reservation, so on a machine that is never idle it waits out --wait and ends NoCapacity (75). Use --on with a
    machine that is idle, or run it in quiet hours.
  • --exclusive --shards N puts one shard on each machine; --shards auto counts the idle machines. A fixed N above the
    machines that can take the job by config (capabilities, size:, --env) is refused (2) before any network.
    --mem and Windows arguments are not part of that count, so extra shards wait for a machine that is idle.
  • In a plan, the dry run marks an exclusive job's machine full, so a sibling job waits.
  • Each slot of an exclusive job is an ordinary slot file, so lanq unlock and the stale-slot reclaim handle it like any
    other (slot release itself now checks the holder; see Security).
  • An exclusive claim also backs out when a live slot-N above its own max_jobs exists: another developer's
    machines.yml with a higher max_jobs for the same work_root has a job there. That is one look at claim time, so
    such a developer can still start a job in a higher slot afterwards: everyone sharing a worker should use the same
    max_jobs.
  • To fit the Windows command-line limit, every Windows slot claim (normal and exclusive) carries its fixed PowerShell
    slot functions compressed (deflate, base64) and dot-sources them in memory, as the helper install already does. Some endpoint-protection tools flag that
    pattern; the inflated text is lanq's constant claim code, and AMSI scans it as such.

Baseline rerun: is this failure new?

  • lanq run --baseline (lanq.yml baseline: true): when a job, a shard or a plan job ends Failed (its own non-zero
    exit), lanq runs the same request again at the base commit, a clean tree, and says whether the failure is new.
    Never after TimedOut, WindowClosed, InfraError, NoCapacity, Rejected or Cancelled.
  • The base is the merge-base of HEAD and the default branch (origin/HEAD, else origin/main, origin/master,
    main, master). --baseline-ref REF (lanq.yml baseline_ref:, implies --baseline) uses the merge-base of HEAD
    and REF instead. On the base commit with uncommitted changes, the base is HEAD without them; with none, the baseline
    is skipped. Refused (Rejected, 2) before anything is sent: a REF that is empty, over 256 characters, has control
    characters, starts with - or contains ..; a REF that is not a commit; a repository with no commit; no default
    branch; unrelated history. A shallow clone that lacks the merge-base is not refused: the baseline is inconclusive.
  • lanq.yml baseline_ref: must name a branch or tag (under refs/heads/, refs/remotes/ or refs/tags/): a commit
    id, HEAD/*_HEAD and revision characters are refused ("baseline_ref must name a branch or tag (a commit can be
    given on the command line with --baseline-ref)"), so a repository's file cannot make lanq send any old tree to an
    archive worker. --baseline-ref on the command line still takes any commit. A short name must exist under exactly
    one of those (baseline_ref NAME is both a branch and a tag; write refs/heads/NAME or refs/tags/NAME, baseline_ref NAME names more than one ref (refs/heads/NAME, refs/remotes/NAME); write one of them); a full refs/heads/NAME or
    refs/tags/NAME is accepted; git's short forms heads/main, tags/v1.0 and remotes/origin/main, which git would
    accept, are refused with the full ref to write (baseline_ref heads/main: write refs/heads/main). The baseline's ref and label show the name as written.
  • How the keys combine: a job's own baseline: false with its own baseline_ref: is an error; its own baseline: false
    drops an inherited ref quietly; its own baseline_ref: turns the baseline on even when defaults: say false; its
    own baseline: true under defaults: with baseline: false and a ref gives a plain --baseline.
  • The verdict is new (the base completed), preexisting (the base failed too, any exit code), inconclusive
    (anything else: timed out, no machine, the base could not be read or sent, Ctrl+C) or skipped. It is information
    only: the run's status and exit code never change.
  • Shards: only the failed shards rerun, with the same LANQ_SHARD/LANQ_SHARDS. Plans: only the failed jobs that
    asked for it, after the whole plan has ended. Each rerun gets its own --wait.
  • A rerun goes to the machine the job failed on: while that machine could take it but is busy, it waits up to 2 looks
    (about 15 s each, never past what is left of its --wait) for a slot there, then takes any eligible machine; with
    --wait 0 (the default) it does not wait for it. A held or ruled-out machine is not waited for. On another machine,
    baseline.sameMachine is false and the message and the new reason end (on big-01, not build-01 where it failed: the result may reflect the machine). Rerun lines are prefixed [baseline], [baseline 2/4] or
    [baseline unit 2/4] ([baseline] [lanq] waiting for a slot on build-01, where the job failed, before taking another machine (look 1 of 2), or ... (the other baseline reruns hold some) while sibling reruns hold slots).
  • Ctrl+C: a rerun not yet started is inconclusive: cancelled (Ctrl+C) before it started on every path (single,
    shards, plans), with status null before it began waiting and Cancelled (130) while it waited or claimed a slot;
    a running one is stopped and cleaned up (Cancelled, 130, inconclusive). A rerun that finds no machine is
    NoCapacity (75), inconclusive.
  • A rerun of an older commit leaves the worker's newer stored base (and bases.json) alone, so the next normal job
    still gets a small incremental bundle.
  • Output: a [lanq] baseline: ... line per rerun before the status line, a baseline object in --json (run, each
    shard, each plan job), and the verdict in the fix_code reason. --dry-run prints the base (Baseline: on failure, rerun at 1a2b3c4 (merge-base with origin/main) (a dry run runs nothing); the JSON message has no suffix). When the
    merge-base is the branch tip the base is named by the branch: at 1a2b3c4 (origin/main).
  • The base commit's payload gets the same committed-name and package-config secret scans as any snapshot (a hit:
    inconclusive: the base commit was not sent: Refused: possible secrets in payload (...)). On an archive worker the
    rerun receives the base commit's tree, an older version of the files.
  • A red run costs about twice as long with --baseline.

JSON and records

  • Run, shard and plan job objects (and the dry runs) gain env (the variable names only, never values),
    exclusive: true and baseline ({ref, sha, status, exitCode, machine, jobId, logPath, durationSeconds, verdict, message}, plus sameMachine when the rerun ran somewhere). Each is left out when it does not apply.
  • New permanent reason codes env_conflict and windows_command_too_long.
  • lanq health --json rows gain slotsStale (only when above 0), and freeSlots counts live slots only; the text
    line adds (4 stale: reclaimed by the next claim) and busy reasons , 1 stale: reclaimed by the next claim.
  • jobs.jsonl gains optional envNames, exclusive, and on baseline reruns baselineOf (the failed job's id) and
    commit (the base's short id); v stays 2. lanq stats, --target-minutes and the plan's history order leave
    baseline reruns out.
  • Like the other job options, --env, --exclusive, --baseline and --baseline-ref are refused with --plan
    (set them in lanq.yml).

Security

  • --env is gated by policy.allowed_env (empty by default) and the hard deny table, and each NAME=VALUE goes
    through the secret scan (Rejected 2, value never echoed). A repository's lanq.yml can set only names the
    developer allowed. The table is a guard against the obvious ways a variable can turn an allowlisted command into
    other code; it is not a sandbox.
  • Values are visible to other users of the worker (its process list) and appear in the dry run and in suggested
    commands in nextSteps, which repeat your command line. Never pass secrets with --env.
  • Slot release now checks the holder: cleanup removes each slot (a normal job's single slot too) only while its first
    line is still this job's id, so a late cleanup can no longer delete a slot another job reclaimed meanwhile.
  • A claim that did not report back (timed out, lost connection, unreadable output, Ctrl+C mid-claim) is followed by a
    best-effort release of the slots that hold this job's id (15 seconds, never fails the job).

Fixes

  • --shards with --mem: a shard that started while lanq was probing the machines was counted as running, as if its
    memory showed in the probe's answer, although the machine answered before it ran. The next round could then place
    one shard too many on that machine. Which shards are not running yet is now taken before the probe.
  • Stale slots (a crashed run's, more than 30 minutes past their deadline) no longer make a machine busy. The probe
    reports them (slots_stale=), placement and freeSlots count only live slots, and the next claim reclaims them as
    it passes. Before, one dead job's slot kept a machine busy for every --exclusive job for ever, and a dead
    exclusive job's slots blocked the machine for everyone until something reclaimed them. lanq health shows them as
    (4 stale: reclaimed by the next claim).
  • A stale slot a claim cannot reclaim counts as busy for the rest of the run. The claim reports it
    (RECLAIM=STUCK slot-N) only when the slot's .reclaim lock is older than about a minute (a dead reclaimer's), no lock
    could be made, or the file could not be moved or deleted; a younger lock (a reclaim in progress) or a claim that
    simply lost the slot to another job never marks the machine. lanq prints one line per machine, counting the slots
    the claim named: [lanq] build-01: 1 stale slot could not be reclaimed (a .reclaim lock or file permissions); counting it as busy for this run; lanq unlock build-01 clears it. Busy reasons then read busy (slots 2/2, 1 stale, not reclaimable). On Windows an unreadable slot above max_jobs now counts as live, as on Linux.
  • Linux workers: stopping a stale job from inside the stale-slot reclaim or lanq unlock overwrote the loop's own
    variables, which could miss the reclaim, leave a .reclaim lock behind and report a slot as REMOVED= that was not.

Tests

  • Two tests that failed now and then on GitHub's Windows runner are now deterministic. The --mem placement test's
    fake machine now reports less free memory once a shard's upload is done, as a real job would use it. The newline-free flood test now measures the reader's
    memory on one thread, not process-wide, where work left by earlier tests counted too.
  • The exclusive-claim race tests run the real sh scripts behind a ready/go barrier, including an exclusive claim that
    must back out while two normal claims race it.
  • The plan test for --wait counting only while no plan job holds a slot (WaitCountsOnlyWhileNoPlanJobHoldsASlot)
    no longer depends on timing: it releases the first job only after the second has said it waits.

Dependencies

  • The native binaries carry the .NET 10.0.15 runtime (was 10.0.14).

Docs

Full changelog: https://github.com/jangolath/lanqueue/blob/main/CHANGELOG.md

What's Changed

  • v0.8.1: lanq-makeready 0.3.0, safe .NET SDK suggestions by @jangolath in #14
  • Deflake two CI tests; count not-running shards before the probe by @jangolath in #15
  • Pin the .NET 10.0.15 runtime for the native binaries by @jangolath in #16

Full Changelog: v0.8.1...v0.9.0