`dockerTools.pullImage` fails to pull image when behind corporate proxy #29271

kuznero · 2017-09-12T12:42:52Z

Issue description

Git repository with problem to reproduce is here.
nix-build docker.nix fails to dockerTools.pullImage and gives the following error:

WARN[0013] Error getting v2 registry: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:3128: getsockopt: connection refused
INFO[0013] Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:3128: getsockopt: connection refused
ERRO[0013] Handler for POST /v1.30/images/create returned error: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:3128: getsockopt: connection refused
Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:3128: getsockopt: connection refused
[   29.472194] reboot: Power down
builder for ‘/nix/store/1yzv58ac9cdrc28ic81fj4d34g5v9jjx-docker-image-alpine-3.3.tar.drv’ failed with exit code 1

Technical details

My developer OS setup is behind corporate proxy (I am using Cntlm for that), thus occurencies of 127.0.0.1:3128.

I have recently switched to channel 17.09.

Might be related to recent change done to pull.nix.

System: 17.09beta484.64a563f15b (Hummingbird)
Nix version: nix-env (Nix) 1.11.14
Nixpkgs version: "17.09beta484.64a563f15b"
Sandboxing enabled: build-use-sandbox = false

The text was updated successfully, but these errors were encountered:

grahamc · 2017-09-12T12:46:16Z

The issue appears to be the new tooling starts a QEMU VM and also inherits the impure environment variables. However, the HTTP(S) proxy variables point to 127.0.0.1, which obviously doesn't work when inherited inside the VM, as the VM isn't running a CNTLM proxy at 3128.

grahamc · 2017-09-12T13:30:30Z

cc @matejc what would you think about rewriting image pulling to use Skopeo?

nlewo · 2017-09-12T13:35:54Z

@kuznero can you try with nix-build -E 'with import <nixpkgs> {}; pkgs.runCommand "pull" {} "${pkgs.skopeo}/bin/skopeo copy docker://library/ubuntu docker-archive://$out:ubuntu:latest"' to see if it works well with proxy?

matejc · 2017-09-12T14:46:59Z

Yeah, I am not fond of my qemu solution either and skopeo looks nice. If I knew about it back then, I would probably used it. Project is maintained and works (just tried loading image into docker downloaded from docker registry with skopeo).

matejc · 2017-09-12T14:49:21Z

Also: A test would be nice to have once implemented, I hope I am not asking too much :)

kuznero · 2017-09-12T17:15:42Z

@nlewo It does not pull image with your expression. Here is what I get:

$ nix-build -E 'with import <nixpkgs> {}; pkgs.runCommand "pull" {} "${pkgs.skopeo}/bin/skopeo copy docker://library/ubuntu docker-archive://$out:ubuntu:latest"'
these derivations will be built:
  /nix/store/cg2vjrpylkr21w6d8ifa3cljfg2wwc5q-pull.drv
building path(s) ‘/nix/store/r1i45jskz9izfsfj1r0ppk7lg6cqkxw8-pull’
time="2017-09-12T17:10:36Z" level=fatal msg="Error initializing image from source docker://ubuntu:latest: pinging docker registry returned: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io: no such host"
builder for ‘/nix/store/cg2vjrpylkr21w6d8ifa3cljfg2wwc5q-pull.drv’ failed with exit code 1
error: build of ‘/nix/store/cg2vjrpylkr21w6d8ifa3cljfg2wwc5q-pull.drv’ failed

On the other side pulling an image explicitly works as expected:

$ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
d5c6f90da05d: Pull complete
1300883d87d5: Pull complete
c220aa3cfc1b: Pull complete
2e9398f099dc: Pull complete
dc27a084064f: Pull complete
Digest: sha256:34471448724419596ca4e890496d375801de21b0e67b81a77fd6155ce001edad
Status: Downloaded newer image for ubuntu:latest

And in case it might help here is the output for sudo systemctl status docker:

● docker.service - Docker Application Container Engine
   Loaded: loaded (/nix/store/wna98yzg6h5rdw2mkwmzf2vafb0hrfxs-docker-17.06.2-ce/etc/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /nix/store/wbilykgmh3idnbak620gw2kkdirlwfpn-system-units/docker.service.d
           └─overrides.conf
   Active: active (running) since Tue 2017-09-12 19:05:16 CEST; 8min ago
     Docs: https://docs.docker.com
 Main PID: 826 (dockerd)
    Tasks: 24 (limit: 4915)
   Memory: 220.5M
      CPU: 6.005s
   CGroup: /system.slice/docker.service
           ├─826 /nix/store/wna98yzg6h5rdw2mkwmzf2vafb0hrfxs-docker-17.06.2-ce/libexec/docker/dockerd --group=docker --host=fd:// --log-driver=json-file -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 --insecure-registry test-docker-hub:5000 --insecure-registry test-docker-hub:5043 --log-opt max-size=1M --log-opt max-file=1
           └─950 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc

Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.612257939+02:00" level=info msg="Loading containers: done."
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.700291948+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version 1.0.0-rc3\nspec: 1.0.0-rc5\n"
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.734332234+02:00" level=info msg="Daemon has completed initialization"
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.734360420+02:00" level=info msg="Docker daemon" commit=cec0b72a9940e047e945a09e1febd781e88366d6 graphdriver=overlay2 version=17.06.2-ce
Sep 12 19:05:16 soul systemd[1]: Started Docker Application Container Engine.
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.741744839+02:00" level=info msg="API listen on [::]:2375"
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.741788753+02:00" level=info msg="API listen on /var/run/docker.sock"
Sep 12 19:05:16 soul dockerd[826]: time="2017-09-12T19:05:16.741800813+02:00" level=info msg="API listen on /var/run/docker.sock"
Sep 12 19:09:40 soul dockerd[826]: time="2017-09-12T19:09:40.638731809+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version 1.0.0-rc3\nspec: 1.0.0-rc5\n"
Sep 12 19:11:35 soul dockerd[826]: time="2017-09-12T19:11:35.847510344+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version 1.0.0-rc3\nspec: 1.0.0-rc5\n"

kuznero · 2017-09-12T18:44:44Z

Just went through downgrading from 17.09 (beta) to latest 17.03 and tried again - all works as expected.
nixos-version emits 17.03.1820.a0e6a891ee (Gorilla) this time around. Hope that can help, @grahamc.

nlewo · 2017-09-12T20:47:17Z

@kuznero arf, sorry, I forgot to propagate the http proxy env variable...
I've tryied the following expression behind a http proxy and it works for me:

nix-build -E 'with import <nixpkgs> {}; pkgs.runCommand "pull" {impureEnvVars=pkgs.stdenv.lib.fetchers.proxyImpureEnvVars; outputHashMode="flat"; outputHashAlgo="sha256"; outputHash="01kbwagma1gaplysxdj7r4szgzkqyk03k7ipnxp7027m8bkicyag";} "${pkgs.skopeo}/bin/skopeo copy docker://library/ubuntu docker-archive://$out:ubuntu:latest"'

kuznero · 2017-09-12T21:02:31Z

@nlewo I still have one instance with 17.09 - will try tomorrow and let you know if it works for me.

kuznero · 2017-09-13T07:21:33Z

@nlewo your expression worked on 17.09 with minor change - sha256 needed to be different. Otherwise, here is how it looked:

soul :: ~ // ‹local› » nix-build -E 'with import <nixpkgs> {}; pkgs.runCommand "pull" {impureEnvVars=pkgs.stdenv.lib.fetchers.proxyImpureEnvVars; outputHashMode="flat"; outputHashAlgo="sha256"; outputHash="1ncmnh8w0a35dd8mjlf0piy2sjj8i4gckkwaj4h4g8c5p1mkb2c7";} "${pkgs.skopeo}/bin/skopeo copy docker://library/ubuntu docker-archive://$out:ubuntu:latest"'
these derivations will be built:
  /nix/store/6hqc7w9zamcxmmri1vmvqv1ld67f18px-pull.drv
building path(s) ‘/nix/store/nm2872jzyvccy0j6j54vvzx3bgd439v8-pull’
Getting image source signatures
Copying blob sha256:d5c6f90da05dc7e77d2e5fef63c341ab05ba2a03396ab5ae8f18814a7bbf5265
 44.75 MB / 45.07 MB
Copying blob sha256:bbbe761fcb565a007b458e09e08ecb88947f647f57be819a492a6b23694cefd8
 0 B / 849 B 5.07 MB
Copying blob sha256:7afa5ede606fb1845e42f0d4816d2a7593a2b666ff9ca4722dcd2cff8a541acf
 0 B / 617 B B
Copying blob sha256:f6b7253b56f434d6e4d97d259cb1007481cf670df2e04229e83cf37db33d96eb
 0 B / 854 B B
Copying blob sha256:2b8db33536d447786cf57cca870fdd7d50d55fd67060c06252edc1c8db456a8f
 0 B / 170 B B
Copying config sha256:8b72bba4485f1004e8378bc6bc42775f8d4fb851c750c6c0329d3770b3a09086
 0 B / 3.53 KB
Writing manifest to image destination
Storing signatures
/nix/store/nm2872jzyvccy0j6j54vvzx3bgd439v8-pull

nlewo · 2017-09-13T07:21:43Z

@kuznero You can clone nixpkgs repository and try by using it, such as

git clone https://github.com/NixOS/nixpkgs
# here we can checkout a branch of nixpkgs
nix-build -E 'with import $PWD/nixpkgs {}; pkgs.runCommand "pull" {impureEnvVars=pkgs.stdenv.lib.fetchers.proxyImpureEnvVars; outputHashMode="flat"; outputHashAlgo="sha256"; outputHash="01kbwagma1gaplysxdj7r4szgzkqyk03k7ipnxp7027m8bkicyag";} "${pkgs.skopeo}/bin/skopeo copy docker://library/ubuntu docker-archive://$out:ubuntu:latest"'

nlewo · 2017-09-13T07:24:05Z

REgarding the hash, each time a new ubuntu image is pushed, the sha has to be upgraded. It seems they have pushed a new image this nigth (the yesterday hash doesn't work for me anymore).

kuznero · 2017-09-13T07:26:17Z

@nlewo I have downgraded my main dev server for now and will not be running tests on 17.09 until the fix is available through 17.09. Do you think the fix can be done using your approach with skopeo instead of qemu and do you know who will be likely to do this change?

nlewo · 2017-09-13T07:49:12Z

@kuznero I'm on it.

Before this patch, a VM was used to spawn docker that pulled the VM. Now, the tool Skopeo does this job well so we can simplify our dockerTools since we doesn't need Docker anymore:) This also fixe the regression described in NixOS#29271 : cntlm proxy doesn't work in 17.09 while it worked in 17.03. Note Skopeo doesn't produce the same output than docker pull so, we have to update sha.

Before this patch, a VM was used to spawn docker that pulled the VM. Now, the tool Skopeo does this job well so we can simplify our dockerTools since we doesn't need Docker anymore:) This also fixe the regression described in #29271 : cntlm proxy doesn't work in 17.09 while it worked in 17.03. Note Skopeo doesn't produce the same output than docker pull so, we have to update sha.

Before this patch, a VM was used to spawn docker that pulled the VM. Now, the tool Skopeo does this job well so we can simplify our dockerTools since we doesn't need Docker anymore:) This also fixe the regression described in #29271 : cntlm proxy doesn't work in 17.09 while it worked in 17.03. Note Skopeo doesn't produce the same output than docker pull so, we have to update sha. (cherry picked from commit 01174c5) Signed-off-by: Domen Kožar <domen@dev.si>

kuznero · 2017-09-19T06:38:34Z

@domenkozar Sorry for potentially lame question - is this fix available in 17.09 beta channel by now? And how can I check such things myself?

nlewo · 2017-09-19T07:36:12Z

@kuznero I guess if the commit is in https://github.com/NixOS/nixpkgs-channels/tree/nixos-17.09 , it is then available from the channel.
You can also update the channel and browse $USER/.nix-defexpr/channels.

Maybe there is also another way to do this check:)

kuznero · 2017-09-19T18:54:32Z

@nlewo @domenkozar This is what I get now with the change committed into 17.09:

these derivations will be built:
  /nix/store/d5g19b2mamca2ybiq4914py6fh9mzcsd-monitor-lib-0.0.1.0.drv
  /nix/store/b4n9rkp3d145rnnd7qc01jv2z1xcbkyb-docker-layer-monitor.drv
  /nix/store/639jzjk5k4h2nf3s3f5qxpb2kyhfa5cx-runtime-deps.drv
  /nix/store/az0g57crd3qf4q8a5s5nl44jfvm03bi3-docker-image-monitor.tar.gz.drv
building path(s) ‘/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0’
setupCompilerEnvironmentPhase
Build with /nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1.
unpacking sources
unpacking source archive /nix/store/7xdcavk6fky6d3ld6jw75l1dg151dvb0-monitor
source root is monitor
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-monitor-lib-0.0.1.0.drv-0/package.conf.d -j1 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /tmp/nix-build-monitor-lib-0.0.1.0.drv-0/Main.o )
Linking Setup ...
configuring
configureFlags: --verbose --prefix=/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-monitor-lib-0.0.1.0.drv-0/package.conf.d --ghc-option=-j1 --disable-split-objs --disable-library-profiling --disable-profiling --enable-shared --disable-coverage --enable-library-vanilla --disable-executable-dynamic --disable-tests --ghc-option=-split-sections
Configuring monitor-lib-0.0.1.0...
Dependency aeson >=1.1.2.0 && <1.2: using aeson-1.1.2.0
Dependency base >=4.10 && <5: using base-4.10.0.0
Dependency filemanip >=0.3.6.3 && <0.4: using filemanip-0.3.6.3
Dependency filepath >=1.4.1.2 && <1.5: using filepath-1.4.1.2
Dependency monitor-lib -any: using monitor-lib-0.0.1.0
Dependency optparse-applicative >=0.13.2.0 && <0.14: using
optparse-applicative-0.13.2.0
Dependency process >=1.6.1.0 && <1.7: using process-1.6.1.0
Dependency scotty >=0.11.0 && <0.12: using scotty-0.11.0
Dependency text >=1.2.2.2 && <1.3: using text-1.2.2.2
Dependency time >=1.8.0.2 && <1.9: using time-1.8.0.2
Dependency timeit >=1.0.0.0 && <2.0.0.0: using timeit-1.0.0.0
Dependency wai-cors >=0.2.5 && <0.3: using wai-cors-0.2.5
Dependency wai-extra >=3.0.20.0 && <3.1: using wai-extra-3.0.20.0
Dependency wai-middleware-static >=0.8.1 && <0.9: using
wai-middleware-static-0.8.1
Source component graph:
    component lib
    component exe:monitor dependency lib
Configured component graph:
    component monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        include base-4.10.0.0
        include aeson-1.1.2.0-GbHCdLDahN2Is3ZqiKEEBb
        include filemanip-0.3.6.3-HRazNawHUwUFMqg3mdq5t4
        include filepath-1.4.1.2
        include process-1.6.1.0
        include time-1.8.0.2
        include timeit-1.0.0.0-BEJd2JotuuhJN1dKbREbw8
    component monitor-lib-0.0.1.0-LfWgDkibJes4SOiuD4A9H3-monitor
        include base-4.10.0.0
        include monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        include wai-cors-0.2.5-CLSrMDDe5ft4HahEhYbyKZ
        include wai-extra-3.0.20.0-HVgKg4rgXZkFTIQ2sUfYG4
        include wai-middleware-static-0.8.1-HAzbt6Kj30ECPmHcpfVmc3
        include scotty-0.11.0-DusBPMXinvoCpOMc1Ok0ar
        include text-1.2.2.2-5RuPYjcevF59B6YENNKuiP
        include time-1.8.0.2
        include optparse-applicative-0.13.2.0-TpuPGTDFif6m1fGXMdCbP
Linked component graph:
    unit monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        include base-4.10.0.0
        include aeson-1.1.2.0-GbHCdLDahN2Is3ZqiKEEBb
        include filemanip-0.3.6.3-HRazNawHUwUFMqg3mdq5t4
        include filepath-1.4.1.2
        include process-1.6.1.0
        include time-1.8.0.2
        include timeit-1.0.0.0-BEJd2JotuuhJN1dKbREbw8
        Monitor=monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX:Monitor,Monitor.Engine=monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX:Monitor.Engine,Monitor.Models=monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX:Monitor.Models
    unit monitor-lib-0.0.1.0-LfWgDkibJes4SOiuD4A9H3-monitor
        include base-4.10.0.0
        include monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        include wai-cors-0.2.5-CLSrMDDe5ft4HahEhYbyKZ
        include wai-extra-3.0.20.0-HVgKg4rgXZkFTIQ2sUfYG4
        include wai-middleware-static-0.8.1-HAzbt6Kj30ECPmHcpfVmc3
        include scotty-0.11.0-DusBPMXinvoCpOMc1Ok0ar
        include text-1.2.2.2-5RuPYjcevF59B6YENNKuiP
        include time-1.8.0.2
        include optparse-applicative-0.13.2.0-TpuPGTDFif6m1fGXMdCbP
Ready component graph:
    definite monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        depends base-4.10.0.0
        depends aeson-1.1.2.0-GbHCdLDahN2Is3ZqiKEEBb
        depends filemanip-0.3.6.3-HRazNawHUwUFMqg3mdq5t4
        depends filepath-1.4.1.2
        depends process-1.6.1.0
        depends time-1.8.0.2
        depends timeit-1.0.0.0-BEJd2JotuuhJN1dKbREbw8
    definite monitor-lib-0.0.1.0-LfWgDkibJes4SOiuD4A9H3-monitor
        depends base-4.10.0.0
        depends monitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX
        depends wai-cors-0.2.5-CLSrMDDe5ft4HahEhYbyKZ
        depends wai-extra-3.0.20.0-HVgKg4rgXZkFTIQ2sUfYG4
        depends wai-middleware-static-0.8.1-HAzbt6Kj30ECPmHcpfVmc3
        depends scotty-0.11.0-DusBPMXinvoCpOMc1Ok0ar
        depends text-1.2.2.2-5RuPYjcevF59B6YENNKuiP
        depends time-1.8.0.2
        depends optparse-applicative-0.13.2.0-TpuPGTDFif6m1fGXMdCbP
Using Cabal-2.0.0.2 compiled by ghc-8.2
Using compiler: ghc-8.2.1
Using install prefix:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0
Executables installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/bin
Libraries installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/lib/ghc-8.2.1/monitor-lib-0.0.1.0
Dynamic Libraries installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/lib/ghc-8.2.1/x86_64-linux-ghc-8.2.1
Private executables installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/libexec/x86_64-linux-ghc-8.2.1/monitor-lib-0.0.1.0
Data files installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/share/x86_64-linux-ghc-8.2.1/monitor-lib-0.0.1.0
Documentation installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/share/doc/x86_64-linux-ghc-8.2.1/monitor-lib-0.0.1.0
Configuration files installed in:
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/etc
No alex found
Using ar found on system at:
/nix/store/4vg5q2pdwl0hacsk9q484ri3qj5kx16s-binutils-2.28.1/bin/ar
No c2hs found
No cpphs found
No doctest found
Using gcc version 6.4.0 given by user at:
/nix/store/3cjndrsgy200i6cgpkz02nwk891v48jd-gcc-wrapper-6.4.0/bin/gcc
Using ghc version 8.2.1 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/ghc
Using ghc-pkg version 8.2.1 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.18.1 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.67 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/hpc
Using hsc2hs version 0.68.2 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/hsc2hs
No hscolour found
No jhc found
Using ld found on system at:
/nix/store/3cjndrsgy200i6cgpkz02nwk891v48jd-gcc-wrapper-6.4.0/bin/ld.gold
No lhc found
No lhc-pkg found
No pkg-config found
Using runghc version 8.2.1 found on system at:
/nix/store/2k41x977yhbz6nrwznxr14j1i8v29wxf-ghc-8.2.1/bin/runghc
Using strip version 2.28 found on system at:
/nix/store/4vg5q2pdwl0hacsk9q484ri3qj5kx16s-binutils-2.28.1/bin/strip
Using tar found on system at:
/nix/store/8drd14bc6fnd1snqis13n9gr8wmn32dc-gnutar-1.29/bin/tar
No uhc found
building
Preprocessing library for monitor-lib-0.0.1.0..
Building library for monitor-lib-0.0.1.0..
[1 of 6] Compiling Helpers          ( src/Helpers.hs, dist/build/Helpers.o )
[2 of 6] Compiling Monitor.Models   ( src/Monitor/Models.hs, dist/build/Monitor/Models.o )
[3 of 6] Compiling Monitor.Engine.Models ( src/Monitor/Engine/Models.hs, dist/build/Monitor/Engine/Models.o )
[4 of 6] Compiling Monitor.Engine.Internal ( src/Monitor/Engine/Internal.hs, dist/build/Monitor/Engine/Internal.o )
[5 of 6] Compiling Monitor.Engine   ( src/Monitor/Engine.hs, dist/build/Monitor/Engine.o )
[6 of 6] Compiling Monitor          ( src/Monitor.hs, dist/build/Monitor.o )
Preprocessing executable 'monitor' for monitor-lib-0.0.1.0..
Building executable 'monitor' for monitor-lib-0.0.1.0..
[1 of 2] Compiling EngineOptionsParser ( app/EngineOptionsParser.hs, dist/build/monitor/monitor-tmp/EngineOptionsParser.o )
[2 of 2] Compiling Main             ( app/Main.hs, dist/build/monitor/monitor-tmp/Main.o )
Linking dist/build/monitor/monitor ...
haddockPhase
installing
Installing library in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/lib/ghc-8.2.1/monitor-lib-0.0.1.0
Installing executable monitor in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/bin
Warning: The directory
/nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/bin is not in
the system search path.
Registering library for monitor-lib-0.0.1.0..
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0
shrinking /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/lib/ghc-8.2.1/x86_64-linux-ghc-8.2.1/libHSmonitor-lib-0.0.1.0-7gMm6Ut4YIqBmwzO6Y5xmX-ghc8.2.1.so
shrinking /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/bin/monitor
stripping (with flags -S) in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/lib  /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0/bin 
patching script interpreter paths in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0
checking for references to /tmp/nix-build-monitor-lib-0.0.1.0.drv-0 in /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0...
building path(s) ‘/nix/store/fhmkrmgv4l5dz2dlia26pgqj53zx3g0d-docker-layer-monitor’
Adding contents...
Adding /nix/store/li1r4j6526vy84f8qg1ydfryp55xxxh1-bash-4.4-p12
Adding /nix/store/rr7ng964frgq0qdmrnh59p1vihxkcr8z-checks
Adding /nix/store/g6bzbxan7z1s2rfkhz12fcnfy1dmqmhh-coreutils-8.28
Adding /nix/store/jcgq450ah9rgyv2c717lqs2vl4q68mjl-curl-7.55.1-bin
Adding /nix/store/1pcyvv83gsyjj33npx33llzsv10qgv8f-docker-17.06.2-ce
Adding /nix/store/1mz1bdp2f26bawlldbzaj43frfp7cnxh-gawk-4.1.4
Adding /nix/store/jshi9bff0ha6scxkndhl1i6kbmmfxxxz-gnugrep-3.1
Adding /nix/store/03g1ack3mbfbcjd6szi048p5y4x8zarm-gnused-4.4
Adding /nix/store/zslql7m7z5cc0sdy6x3y11pi2k4r5vfp-jq-1.5
Adding /nix/store/jsz7ch06llxfwrqbbaswd7kmkham8phl-monitor-lib-0.0.1.0
Adding /nix/store/ci8c6vby41cv4khgybwxcixw0j6w1lky-static
Adding /nix/store/4r87994zd5r35ikns40zr6xq9b8idrkg-init
Packing layer...
Computing layer checksum...
Finished building layer 'monitor'
building path(s) ‘/nix/store/9c74ljz2g7s77kawmnf5zjv58xq9wy85-runtime-deps’
building path(s) ‘/nix/store/wy2bz2n00xan2xnc543rf97jba1rxpcd-docker-image-monitor.tar.gz’
Unpacking base image...
/tmp/nix-build-docker-image-monitor.tar.gz.drv-0/.attr-0: line 24: image/repositories: No such file or directory
builder for ‘/nix/store/az0g57crd3qf4q8a5s5nl44jfvm03bi3-docker-image-monitor.tar.gz.drv’ failed with exit code 1
error: build of ‘/nix/store/az0g57crd3qf4q8a5s5nl44jfvm03bi3-docker-image-monitor.tar.gz.drv’ failed

It seems that now it is downloading the image correctly, but fails further with: /tmp/nix-build-docker-image-monitor.tar.gz.drv-0/.attr-0: line 24: image/repositories: No such file or directory. Not sure if it is me doing something wrong or the fix does not completely work.

GitHub repository to reproduce on.

nlewo · 2017-09-20T07:06:52Z

@kuznero I think this issue is not related to the previous one, so it would have been better to open a new one :)

I think the problem comes with your base image but it has been updated. So if you could update it:

nix-build -E 'with import <nixpkgs> {}; pkgs.dockerTools.pullImage { imageName = "alpine"; imageTag = "3.3"; sha256 = "00qw04qkjmbim2ykr032v0rq5j0arggwd04b69ib5pcrcds22rsn";}' --check

And maybe this will solve your issue.

kuznero · 2017-09-20T17:03:04Z

@nlewo it fails with the following trace:

these derivations will be built:
  /nix/store/nrpcz4nik4gzal6rd8rw6bcfmrh5ch3g-docker-image-alpine-3.3.tar.drv
these paths will be fetched (3.27 MiB download, 13.72 MiB unpacked):
  /nix/store/ahcgvijqx4qr0ilr82pmla4qj1f6na4v-ostree-2017.9
  /nix/store/aj3208ld6sizy2djbzmxmxd21pf92c3f-gpgme-1.9.0
  /nix/store/c4v0x041vlil4qjgh34d96n9xp87vibm-skopeo-0.1.22-bin
  /nix/store/f11cnrn28g06fvifmbdw3yvi1jifcxyf-paxctl-0.9
  /nix/store/kzgi9k0869m95wcqrgmm3r362a4sln07-stdenv
  /nix/store/xy2lw4card1wml5s5s6qv1fx7gm702ib-patchelf-0.9
error: some outputs of ‘/nix/store/nrpcz4nik4gzal6rd8rw6bcfmrh5ch3g-docker-image-alpine-3.3.tar.drv’ are not valid, so checking is not possible

Will open a new issue.

kuznero · 2017-09-20T17:10:45Z

@nlewo Opened a new issue #29608

Before this patch, a VM was used to spawn docker that pulled the VM. Now, the tool Skopeo does this job well so we can simplify our dockerTools since we doesn't need Docker anymore:) This also fixe the regression described in NixOS#29271 : cntlm proxy doesn't work in 17.09 while it worked in 17.03.

grahamc added 0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 1.severity: blocker This is preventing another PR or issue from being completed 6.topic: user experience labels Sep 12, 2017

nlewo mentioned this issue Sep 13, 2017

dockerTools.pullImage: use skopeo to pull the image #29302

Merged

8 tasks

domenkozar closed this as completed in #29302 Sep 17, 2017

kuznero mentioned this issue Sep 20, 2017

dockerTools.pullImage fails to unpack base image #29608

Closed

nlewo mentioned this issue Apr 3, 2018

dockerTools.pullImage: Skopeo pulls images by digest #38371

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`dockerTools.pullImage` fails to pull image when behind corporate proxy #29271

`dockerTools.pullImage` fails to pull image when behind corporate proxy #29271

kuznero commented Sep 12, 2017

grahamc commented Sep 12, 2017

grahamc commented Sep 12, 2017

nlewo commented Sep 12, 2017

matejc commented Sep 12, 2017

matejc commented Sep 12, 2017

kuznero commented Sep 12, 2017

kuznero commented Sep 12, 2017 •

edited

Loading

nlewo commented Sep 12, 2017

kuznero commented Sep 12, 2017

kuznero commented Sep 13, 2017

nlewo commented Sep 13, 2017

nlewo commented Sep 13, 2017

kuznero commented Sep 13, 2017

nlewo commented Sep 13, 2017

kuznero commented Sep 19, 2017

nlewo commented Sep 19, 2017

kuznero commented Sep 19, 2017

nlewo commented Sep 20, 2017

kuznero commented Sep 20, 2017

kuznero commented Sep 20, 2017

dockerTools.pullImage fails to pull image when behind corporate proxy #29271

dockerTools.pullImage fails to pull image when behind corporate proxy #29271

Comments

kuznero commented Sep 12, 2017

Issue description

Technical details

grahamc commented Sep 12, 2017

grahamc commented Sep 12, 2017

nlewo commented Sep 12, 2017

matejc commented Sep 12, 2017

matejc commented Sep 12, 2017

kuznero commented Sep 12, 2017

kuznero commented Sep 12, 2017 • edited Loading

nlewo commented Sep 12, 2017

kuznero commented Sep 12, 2017

kuznero commented Sep 13, 2017

nlewo commented Sep 13, 2017

nlewo commented Sep 13, 2017

kuznero commented Sep 13, 2017

nlewo commented Sep 13, 2017

kuznero commented Sep 19, 2017

nlewo commented Sep 19, 2017

kuznero commented Sep 19, 2017

nlewo commented Sep 20, 2017

kuznero commented Sep 20, 2017

kuznero commented Sep 20, 2017

`dockerTools.pullImage` fails to pull image when behind corporate proxy #29271

`dockerTools.pullImage` fails to pull image when behind corporate proxy #29271

kuznero commented Sep 12, 2017 •

edited

Loading