New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker load fails to load xz-compressed image #9327

Closed
jperville opened this Issue Nov 25, 2014 · 19 comments

Comments

Projects
None yet
@jperville
Contributor

jperville commented Nov 25, 2014

The ability to docker load xz-compressed images has been broken with docker 1.3.2 (it was working until docker 1.3.1 included). Loading a gzip-compressed version of the same image still works, so the breakage is xz-specific.

Error message is Error: Untar exit status 1 exec: "xz": executable file not found in $PATH, even if the xz command is installed as /usr/bin/xz.

With docker 1.3.2:

vagrant@docker-ubuntu-1404:~$ file /docker-images/busybox-trusty.tar.xz 
/docker-images/busybox-trusty.tar.xz: XZ compressed data
vagrant@docker-ubuntu-1404:~$ docker --version
Docker version 1.3.2, build 39fa2fa
vagrant@docker-ubuntu-1404:~$ sudo docker load --input=/docker-images/busybox-trusty.tar.xz
2014/11/25 02:13:39 Error: Untar exit status 1 exec: "xz": executable file not found in $PATH
vagrant@docker-ubuntu-1404:~$ which xz
/usr/bin/xz

With docker 1.3.1:

vagrant@client-ingest-ubuntu-1404:~$ file /docker-images/busybox-trusty.tar.xz 
/docker-images/busybox-trusty.tar.xz: XZ compressed data
vagrant@client-ingest-ubuntu-1404:~$ docker --version
Docker version 1.3.1, build 4e9bbfa
vagrant@client-ingest-ubuntu-1404:~$ sudo docker load --input=/docker-images/busybox-trusty.tar.xz
vagrant@client-ingest-ubuntu-1404:~$ which xz
/usr/bin/xz
@ajcrowe

This comment has been minimized.

Show comment
Hide comment
@ajcrowe

ajcrowe Nov 25, 2014

This issue also affects me, I'm guessing it has something to do with the chroot that is now taking place.

ajcrowe commented Nov 25, 2014

This issue also affects me, I'm guessing it has something to do with the chroot that is now taking place.

@ignatenkobrain

This comment has been minimized.

Show comment
Hide comment
@ignatenkobrain

ignatenkobrain Nov 25, 2014

I can confirm.

I can confirm.

@jperville

This comment has been minimized.

Show comment
Hide comment
@jperville

jperville Nov 26, 2014

Contributor

Thanks for the quick spotting everyone, crossing fingers that it get fixed in the next release.

Contributor

jperville commented Nov 26, 2014

Thanks for the quick spotting everyone, crossing fingers that it get fixed in the next release.

@ignatenkobrain

This comment has been minimized.

Show comment
Hide comment
@ignatenkobrain

ignatenkobrain Nov 26, 2014

[a66097f7] +job load()
Untar exit status 1 [debug] archive.go:103 [tar autodetect] n: [253 55 122 88 90 0 0 4 230 214]
exec: "xz": executable file not found in $PATH
[a66097f7] -job load() = ERR (1)
[error] server.go:1207 Handler for POST /images/load returned error: exec: "xz": executable file not found in $PATH
[error] server.go:110 HTTP Error: statusCode=500 exec: "xz": executable file not found in $PATH
[a66097f7] +job load()
Untar exit status 1 [debug] archive.go:103 [tar autodetect] n: [253 55 122 88 90 0 0 4 230 214]
exec: "xz": executable file not found in $PATH
[a66097f7] -job load() = ERR (1)
[error] server.go:1207 Handler for POST /images/load returned error: exec: "xz": executable file not found in $PATH
[error] server.go:110 HTTP Error: statusCode=500 exec: "xz": executable file not found in $PATH
@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Nov 26, 2014

Contributor

this clearly needs an integration test to make sure it never breaks again :)

Contributor

SvenDowideit commented Nov 26, 2014

this clearly needs an integration test to make sure it never breaks again :)

@msis

This comment has been minimized.

Show comment
Hide comment
@msis

msis Nov 30, 2014

Is there any quick fix to this bug or should we move back to 1.3.1?

msis commented Nov 30, 2014

Is there any quick fix to this bug or should we move back to 1.3.1?

@rfc1459

This comment has been minimized.

Show comment
Hide comment
@rfc1459

rfc1459 Nov 30, 2014

This is caused by the fix for CVE-2014-6407: gzip and bzip2 are handled internally using the stdlib-provided codecs, while xz is handled via an external command.

Since the unpacker now seals itself into a chroot before expanding any archive, it won't find the xz executable needed to decompress the archive.
I performed a cursory search for a golang implementation of xz, but I couldn't find any.

@msis: 1.3.1 and previous versions are affected by the aforementioned vulnerability, you should not go back to it. The only viable workaround until the issue is fixed is recompressing archives with gzip or bzip2 before consuming them in a Dockerfile.

rfc1459 commented Nov 30, 2014

This is caused by the fix for CVE-2014-6407: gzip and bzip2 are handled internally using the stdlib-provided codecs, while xz is handled via an external command.

Since the unpacker now seals itself into a chroot before expanding any archive, it won't find the xz executable needed to decompress the archive.
I performed a cursory search for a golang implementation of xz, but I couldn't find any.

@msis: 1.3.1 and previous versions are affected by the aforementioned vulnerability, you should not go back to it. The only viable workaround until the issue is fixed is recompressing archives with gzip or bzip2 before consuming them in a Dockerfile.

@englishm

This comment has been minimized.

Show comment
Hide comment
@englishm

englishm Dec 1, 2014

@rfc1459 how could that workaround be applied when using images from the public repository?

englishm commented Dec 1, 2014

@rfc1459 how could that workaround be applied when using images from the public repository?

@ewindisch

This comment has been minimized.

Show comment
Hide comment
@ewindisch

ewindisch Dec 1, 2014

Contributor

I have a PR incoming for this. I spoke to @jfrazelle and I think we can make it into 1.4.0

Contributor

ewindisch commented Dec 1, 2014

I have a PR incoming for this. I spoke to @jfrazelle and I think we can make it into 1.4.0

@isanych

This comment has been minimized.

Show comment
Hide comment
@isanych

isanych Dec 3, 2014

Is it worth to create pull request with .xz to .bz2 changes to mkimage.sh or 1.4.0 will be released soon enough?

isanych commented Dec 3, 2014

Is it worth to create pull request with .xz to .bz2 changes to mkimage.sh or 1.4.0 will be released soon enough?

BurntSushi added a commit to diffeo/Datawake that referenced this issue Dec 3, 2014

Use the most recent version of jplock/zookeeper.
The maintainer of the zookeeper image does not keep the 'latest' tag up
to date, so we have to manually request it.

See https://registry.hub.docker.com/u/jplock/zookeeper/

The particular reason for using the latest version is to work around
this bug in Docker 1.3: moby/moby#9327
(The bug is triggered with 'jplock/zookeeper:latest' but not with
'jplock/zookeeper:3.4.6'.)
@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Dec 3, 2014

Contributor

1.4.0 will be released by the end of the week, and @ewindisch is making the
patch.

On Wed, Dec 3, 2014 at 3:49 AM, Igor Kostenko notifications@github.com
wrote:

Is it worth to create pull request with .xz to .bz2 changes to mkimage.sh
or 1.4.0 will be released soon enough?


Reply to this email directly or view it on GitHub
#9327 (comment).

Contributor

jessfraz commented Dec 3, 2014

1.4.0 will be released by the end of the week, and @ewindisch is making the
patch.

On Wed, Dec 3, 2014 at 3:49 AM, Igor Kostenko notifications@github.com
wrote:

Is it worth to create pull request with .xz to .bz2 changes to mkimage.sh
or 1.4.0 will be released soon enough?


Reply to this email directly or view it on GitHub
#9327 (comment).

jbergstroem added a commit to jbergstroem/gentoo-bb that referenced this issue Dec 8, 2014

Switch the portage tarball from xz to bz2
Docker 1.3.2 currently has a bug wrt using xz from a sandbox. See
moby/moby#9327

We loose ~13% bandwidth but at least have a working build

moul added a commit to armbuild/docker-brew-ubuntu-debootstrap that referenced this issue Dec 8, 2014

@o10g

This comment has been minimized.

Show comment
Hide comment
@o10g

o10g Dec 10, 2014

Docker version 1.3.2 still has the issue.

o10g commented Dec 10, 2014

Docker version 1.3.2 still has the issue.

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Dec 11, 2014

Contributor

closed, commits in 1.3.3 solved 001ac15

Contributor

jessfraz commented Dec 11, 2014

closed, commits in 1.3.3 solved 001ac15

@EwanValentine

This comment has been minimized.

Show comment
Hide comment
@EwanValentine

EwanValentine Apr 22, 2015

I'm still getting this... 20GB vm with 18GB left free.

Docker version 1.6.0, build 4749651

Boot2Docker-cli version: v1.6.0
Git commit: 9894ae9

I'm still getting this... 20GB vm with 18GB left free.

Docker version 1.6.0, build 4749651

Boot2Docker-cli version: v1.6.0
Git commit: 9894ae9

@murali44

This comment has been minimized.

Show comment
Hide comment
@murali44

murali44 Apr 23, 2015

I'm seeing similar issues when I use docker load. I'm using docker 1.6.0.

Error: Untar exit status 1 no such file or directory

Error: Untar exit status 1 lchown /usr/bin/unzip: no such file or directory

Error: Untar exit status 1 lchown /usr/lib/x86_64-linux-gnu/libgphoto2/2.5.3.1/stv0674.so: no such file or directory

I'm seeing similar issues when I use docker load. I'm using docker 1.6.0.

Error: Untar exit status 1 no such file or directory

Error: Untar exit status 1 lchown /usr/bin/unzip: no such file or directory

Error: Untar exit status 1 lchown /usr/lib/x86_64-linux-gnu/libgphoto2/2.5.3.1/stv0674.so: no such file or directory

@jessfraz jessfraz reopened this Apr 24, 2015

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Apr 24, 2015

Contributor

@murali44 are you on boot2docker as well

Contributor

jessfraz commented Apr 24, 2015

@murali44 are you on boot2docker as well

@murali44

This comment has been minimized.

Show comment
Hide comment
@murali44

murali44 Apr 24, 2015

@jfrazelle no, I'm not. I'm just running docker on an Ubuntu machine. I tried both 1.5 and 1.6.

I'm trying to load multiple docker images from a tar file using the docker load command.

@jfrazelle no, I'm not. I'm just running docker on an Ubuntu machine. I tried both 1.5 and 1.6.

I'm trying to load multiple docker images from a tar file using the docker load command.

@EwanValentine

This comment has been minimized.

Show comment
Hide comment
@EwanValentine

EwanValentine Apr 24, 2015

Re-sizing the docker VM, doesn't seem to re-size /dev/sda1, which in my case was still full. So I just removed looooads of old, un-used containers that were still running on there and re-built the containers I actually wanted to use.

Re-sizing the docker VM, doesn't seem to re-size /dev/sda1, which in my case was still full. So I just removed looooads of old, un-used containers that were still running on there and re-built the containers I actually wanted to use.

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Apr 24, 2015

Contributor

I cannot reproduce locally or on test server, @murali44 can you check if its your disk size too

Contributor

jessfraz commented Apr 24, 2015

I cannot reproduce locally or on test server, @murali44 can you check if its your disk size too

@jessfraz jessfraz closed this Apr 24, 2015

@tiborvass tiborvass removed their assignment Nov 3, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment