New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker compose 1.10.1 creates invalid tar files during build when permissions do not allow read of any files #4437

Closed
akatrevorjay opened this Issue Feb 7, 2017 · 13 comments

Comments

Projects
None yet
5 participants
@akatrevorjay
Copy link

akatrevorjay commented Feb 7, 2017

This is rather problematic with volumes for instance. This is a recent regression since 1.9.x.

One of the big gotchas is it thinks the docker daemon just closes the connection, when in reality it's sending it bad data that it never seems to know about.

Kudos.

@shin-

This comment has been minimized.

Copy link
Contributor

shin- commented Feb 7, 2017

Hi,

Thank you for the report. I'm a bit confused about what the issue actually is here. Do you mind sharing more details / clarifying? Here's some info that would help:

  • OS / version
  • Docker Engine version
  • What do you mean by "when permissions do not allow read of any files"? Is the user executing docker-compose not able to read the files it's supposed to tar?
  • What do you mean by "creates invalid tar files"? Are they corrupted and illegible? Are some files missing?
@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Feb 14, 2017

Hi.

No problem. It's easy to reproduce, just make a volume owned by root (the default) in a compose project then build it.

@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Feb 14, 2017

OS/version: Linux of some kind, but that's not a factor here.
Docker engine: Doesn't matter as this is a compose bug, but 1.13.1

Is the user executing docker-compose not able to read the files it's supposed to tar?

Yes.

What do you mean by "creates invalid tar files"? Are they corrupted and illegible? Are some files missing?

It makes invalid tar streams silently and tries to hand them to the daemon as if they're valid, which it rightfully rejects.

@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Mar 3, 2017

Building web
ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
Mar 02 19:11:17 nsfdeux dockerd[6193]: time="2017-03-02T19:11:17.660152127-08:00" level=error msg="Handler for POST /v1.22/build returned error: Error processing tar file(archive/tar: invalid tar header): "
@shin-

This comment has been minimized.

Copy link
Contributor

shin- commented Mar 3, 2017

I can't seem to reproduce this with Compose 1.11.2:

$ sudo ls -R -l
.:
total 16
drwx------ 3 root    root    4096 Mar  3 12:00 a
drwx------ 3 root    root    4096 Mar  3 12:01 c
-rw-rw-r-- 1 joffrey joffrey   77 Mar  3 12:04 docker-compose.yml
-rw-rw-r-- 1 joffrey joffrey   57 Mar  3 12:04 Dockerfile

./a:
total 4
drwx------ 2 root root 4096 Mar  3 12:00 b

./a/b:
total 4
-rwx------ 1 root root 19 Mar  3 12:00 c.txt

./c:
total 4
drwx------ 2 root root 4096 Mar  3 12:01 d

./c/d:
total 4
-rwx------ 1 root root 41 Mar  3 12:01 xyz.py
$ docker-compose build --no-cache
Building foo
Step 1/3 : FROM alpine:3.4
 ---> 0766572b4bac
Step 2/3 : COPY . /data
 ---> db10f045d62e
Removing intermediate container 1cea5adf291f
Step 3/3 : CMD echo Hello World
 ---> Running in 239216950eba
 ---> 8f41791320f7
Removing intermediate container 239216950eba
Successfully built 8f41791320f7
@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Mar 4, 2017

@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Mar 28, 2017

We can reproduce this reliably, at least on Linux.

@shin-

This comment has been minimized.

Copy link
Contributor

shin- commented Mar 28, 2017

Apologies, I lost track of that ticket.

Yes, tested on Linux (Ubuntu 16.04)

@denysvitali

This comment has been minimized.

Copy link

denysvitali commented May 9, 2017

Experienced the same issue, had to delete my volumes contents to build again. (I think chowning may have worked too)

@akatrevorjay

This comment has been minimized.

Copy link
Author

akatrevorjay commented Jun 4, 2017

AHA! Figured it out. It's due to a socket file. @shin-

@kriestof

This comment has been minimized.

Copy link

kriestof commented Jun 6, 2017

Had same error after upgrading docker(17.05)+docker-compose(1.13).

I had some files with root owner generated by old build on volumes contents. I just had to remove them(this are only some cache and tmp files). I think before I also had to do delete them before build, but the error was more descriptive.

Probably similar to what @denysvitali says.

@HughWarrington

This comment has been minimized.

Copy link

HughWarrington commented Dec 11, 2017

Can confirm on Ubuntu 16.04, docker-compose 1.16.1, Docker 17.09.0-ce.

I touched a file in the build directory, set it owned by root, and set permissions to 600. Then ran (as a non-root user) docker-compose up --build.

The symptoms were exactly as @akatrevorjay described on 3 March.

@shin-

This comment has been minimized.

Copy link
Contributor

shin- commented Feb 13, 2018

Fixed in 1.19.0 (#5496)

@shin- shin- closed this Feb 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment