Fetching paths from cache.nixos.org gets stuck in certain Docker environments

[nebez]

Describe the bug

When fetching paths from cache.nixos.org, the operation gets stuck depending on the environment. This is a follow-up to my comment in #160289. Given enough time, the operation seems to make progress. For a simple operation of installing a single package from nixpkgs, this takes ~3 minutes. For something slightly more complex (our production app), the operation hasn't yet completed and has been running for 2 hours. For comparison, it used to take ~10 minutes on nixos/nix:2.22.3.

This behaviour was introduced in nixos/nix:2.23.0-arm64. It was last working in nixos/nix:2.22.3-arm64. I've tested every new tag published since (until 2.24.1-arm64) and observed the same behaviour.

What's been difficult about narrowing this bug down has been that it is consistent in its reproduction, but behaves very differently depending on invocation. Using the same Dockerfile on my darwin vs. linux produces different results. On darwin, I don't face this issue. I assume this is because the of docker-desktop virtualization/differences. Secondly, invoking the offending command via docker run works fine!

I will describe below.

Steps To Reproduce

On linux, create Dockerfile and fill contents:

FROM nixos/nix:2.23.0-arm64
RUN nix-env --file https://github.com/NixOS/nixpkgs/archive/47b604b07d1e8146d5398b42d3306fdebd343986.tar.gz --install --attr sops

docker build .

Observe getting stuck at: copying path '/nix/store/y359d58sp2j91i86aclsg16p8h414n16-sops-3.8.1' from 'https://cache.nixos.org'...

Wait for a few minutes, and eventually the process will continue. On an arm64 c7g.2xlarge, this operation takes 227 seconds.

Expected behavior

Change base image in your Dockerfile to nixos/nix:2.22.3-arm64

docker build .

You will not get stuck copying path [...]... step.

Process will complete in a much more reasonable 33 seconds on the same arm64 c7g.2xlarge machine.

Additional context

It seems this only happens in the docker builder. When running these same two commands, you will not see any difference in behaviour. Both operations complete in a reasonable 30s.

time docker run --rm nixos/nix:2.22.3 nix-env --file https://github.com/NixOS/nixpkgs/archive/47b604b07d1e8146d5398b42d3306fdebd343986.tar.gz --install --attr sops
time docker run --rm nixos/nix:2.23.0 nix-env --file https://github.com/NixOS/nixpkgs/archive/47b604b07d1e8146d5398b42d3306fdebd343986.tar.gz --install --attr sops

These commands respectively return:

# nixos/nix:2.22.3
real	0m37.988s

# nixos/nix:2.23.0
real	0m41.800s

Notify maintainers

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ docker run --rm nixos/nix:2.22.3 nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-linux"`
 - host os: `Linux 6.1.90-99.173.amzn2023.aarch64`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.22.3`
 - channels(root): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

[user@system:~]$ docker run --rm nixos/nix:2.23.0 nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-linux"`
 - host os: `Linux 6.1.90-99.173.amzn2023.aarch64`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.23.0`
 - channels(root): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

[user@system:~]$ docker -v
Docker version 25.0.3, build 4debf41

---

Add a 👍 reaction to issues you find important.

[eclairevoyant]

1. The nix repo is in github.com/nixos/nix (this repo is for nixpkgs) and
2. nix-env is atrociously slow and generally discouraged, you might consider using the alternatives mentioned at https://stop-using-nix-env.privatevoid.net/

[nebez]

Thanks for the redirect help @eclairevoyant – I've filed the bug over there instead. Will close this one. NixOS/nix#11258