New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockerTools.buildImage: do not add /nix/store in the tar stream #34939

Merged
merged 1 commit into from Feb 14, 2018

Conversation

Projects
None yet
4 participants
@nlewo
Copy link
Member

nlewo commented Feb 13, 2018

Since the /nix/store directory is not immutable, tar can fails if it
has to push it into the layer archive.

Fixes #34137.

Motivation for this change
Things done

Build and run 20 of my Docker images

  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

dockerTools.buildImage: do not add /nix/store in the tar stream
Since the /nix/store directory is not immutable, tar can fails if it
has to push it into the layer archive.

Fixes #34137.
@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Feb 13, 2018

@Profpatsch

This comment has been minimized.

Copy link
Member

Profpatsch commented Feb 14, 2018

I wrote a simple test to check whether dockerTools image generation works.

@GrahamcOfBorg test docker-tools

@GrahamcOfBorg

This comment has been minimized.

Copy link

GrahamcOfBorg commented Feb 14, 2018

Success on aarch64-linux (full log)

Partial log (click to expand)

@GrahamcOfBorg

This comment has been minimized.

Copy link

GrahamcOfBorg commented Feb 14, 2018

Success on x86_64-linux (full log)

Partial log (click to expand)

docker: exit status 1
syncing
docker: running command: sync
docker: exit status 0
test script finished in 15.52s
cleaning up
killing docker (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/tmp/nix-build-vm-test-run-docker-tools.drv-0/vde1.ctl': Directory not empty
/nix/store/majn1mnidiwjpj3fsg6dxcq1hka3bp77-vm-test-run-docker-tools

@Profpatsch

This comment has been minimized.

Copy link
Member

Profpatsch commented Feb 14, 2018

LGTM

@Profpatsch Profpatsch merged commit ce838e5 into NixOS:master Feb 14, 2018

8 checks passed

grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env --file . --query --available --json
Details
@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Feb 14, 2018

Do you think we can backport this into 17.09?

@Profpatsch

This comment has been minimized.

Copy link
Member

Profpatsch commented Feb 15, 2018

Yeah, why not. I’m not sure how backports work tbh, I think the easiest is if you recreate the PR against the stable branch.

@LnL7

This comment has been minimized.

Copy link
Member

LnL7 commented Apr 13, 2018

I just confirmed that this caused LnL7/nix-docker#14 (comment) and the chmod workaround I used doesn't work with older versions of aufs.

Reverting these changes works everywhere LnL7/nix-docker@aebee91.

@Profpatsch

This comment has been minimized.

Copy link
Member

Profpatsch commented Apr 13, 2018

@LnL7 It’s not clear to me what broke exactly and how to act on it.

@LnL7

This comment has been minimized.

Copy link
Member

LnL7 commented Apr 13, 2018

Because /nix is not in the layer it shows up as drw------- 4 root root 4 Apr 12 22:36 nix. Users in the container can't list/read the store causing errors like binaries failing to find ld-linux-x86-64.so.2.

@Profpatsch

This comment has been minimized.

Copy link
Member

Profpatsch commented Apr 13, 2018

Ah, hm. @nlewo did you originally push this because you wanted to run nix inside the docker container?
Because in that case, why not make /nix into a volume for such use-cases?

Personally I’d say /nix should be rather static inside images.

@LnL7 If you add a (failing) VM test for your use-case, we can use it for regression testing in the future.

@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Apr 14, 2018

I did this because you must not read /nix/store at image creation. Basically, the /nix/store is not immutable and tar fails if /nix/store is modified during taring. It is more detailed in #34137.

We are also facing the user permission problem that is addressed in #38837.

I also think we should add a test regarding user permissions.

@LnL7

This comment has been minimized.

Copy link
Member

LnL7 commented Apr 14, 2018

I posted here because I still don't really understand what this fixes, but since you approved the other pr I will assume it won't reintroduce this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment