This repository was archived by the owner on Sep 1, 2025. It is now read-only.

Description
When running jobs that have huge provenance files, the CLI function _nix_hashes receives too many arguments and breaks due to ARG_MAX limitations (depending on the OS).
Here is where the bug happens
|
def _nix_hashes(*paths: str) -> List[str]: |
|
cmd = [ |
|
f"{__NIX_STABLE__}/bin/nix-store", |
|
"--query", |
|
"--hash", |
|
*paths, |
|
] |
|
out, stdout, _ = _run_outputs(cmd, stderr=None) |
|
if out != 0: |
|
raise SystemExit(out) |
|
|
|
return stdout.decode().splitlines() |
Specifically in the *paths argument provided to the shell (line 334).
Let's make _nix_hashes receive bytes from stdin and then call nix-store using xargs so it does not hit the ARG_MAX limit.
I found this bug by running m gitlab:fluidattacks/universe@trunk /docs/generate/criteria, as it currently depends on several hundred derivations for building the entire criteria section of the website.