Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated build fails on 'tar' with: "Directory renamed before its status could be extracted" #727

Open
0o-de-lally opened this issue Jul 1, 2016 · 45 comments

Comments

@0o-de-lally
Copy link

Hello I'm trying to move a repository into Docker Cloud. All my
repositories are failing with a 'tar' command error.

The Error:
tar: .meteor/packages/babel-compiler: Directory renamed before its
status could be extracted
tar: Exiting with failure status due to previous errors

I've searched online and there are no answers. Even on Docker's github
moby/moby#19647

@pierreozoux
Copy link

I have the same here!

Here is my repo https://hub.docker.com/r/pierreozoux/tioapp/builds/bsnpyaasmo46p962p9ikbhu/

I can build fine on my machine, but can't build on the hub.

Did you move to some distributed filesyteme backend?
Or overlayFS?

I see these issues that might be related:
meteor/meteor#5762
meteorhacks/meteord#64

@0o-de-lally
Copy link
Author

@pierreozoux Have you tried it on Quay.io ? It seems that my automated builds work there.

@pkennedyr
Copy link
Contributor

Hi @pierreozoux, @keyscores: Thanks for the feedback. We are indeed on OverlayFS which might be the source of the trouble. We are happy to file an FR for selectable file drivers as an advanced feature if enough people are seeing this issue.

Please +1 if you are interested in this feature.

@0o-de-lally
Copy link
Author

@pkennedyr @borjaburgos I have no need to set file system drivers. I think that this is a docker bug that needs to be fixed. I would recommend fixing that bug rather than adding a feature.

I just need my images to build predictably. For now I will use Quay.io, and I will recommend that others use the Quay automated build service over Docker Cloud for that reason.

@pkennedyr
Copy link
Contributor

@keyscores, We value your feedback and are investigating this issue further to find a more versatile solution.

@0o-de-lally
Copy link
Author

@pkennedyr This issue is still occurring, even though I received an email from support saying it was closed.

The error I receive:
tar: .meteor/packages/coffeescript/.1.2.3.m88qvm++os+web.browser+web.cordova: Directory renamed before its status could be extracted

@karalabe
Copy link

My repos started failing with this error about a month ago, then last week it seem to have worked a few times, then it started failing again. I'm extracting a simple tar file, nothing fancy.

@cioc
Copy link

cioc commented Aug 19, 2016

@mattmcd
Copy link

mattmcd commented Aug 29, 2016

Same issue with tar in my builds: https://hub.docker.com/r/mattmcd/naclbuildbase/builds/blfctp7gaqtlmgkeq3twjse/
Vaguely interesting that the previous unzip step worked fine in this build.

@0o-de-lally
Copy link
Author

0o-de-lally commented Oct 27, 2016

@pchico83 @borjaburgos This is still happening. I got a support message saying this was solved months ago.

Building works fine on quay.io, but dockercloud produces authentication errors when pulling from quay. So I can't use docker cloud.

@0o-de-lally
Copy link
Author

@borjaburgos , I've written multiple times this week, with no response.

Tar files are not opening correctly during builds on certain infrastructure. I think this is an old issue that should have been solved a while ago.

Try with docker hub image: jshimko/meteor-launchpad:latest

Full error:
tar: .meteor/packages/babel-compiler/.6.13.0.1qw8sow++os+web.browser+web.cordova/npm/node_modules/.bin: Directory renamed before its status could be extracted

@0o-de-lally
Copy link
Author

@pkennedyr When I do a 'build on your own infrastructure' option in docker cloud. The issue goes away. I think it has to do with your docker version on certain machines in your build pool. See this issue: moby/moby#19647

@pkennedyr
Copy link
Contributor

@keyscores We apologize for the delay wrt this issue. Indeed, the build failure appears related to moby/moby#19647 and once the upstream overlayfs kernel update is available, Docker Cloud and Docker Hub will apply the the corresponding version update to address this issue.

@0o-de-lally
Copy link
Author

@pkennedyr To be clear: this is intermittent. It seems like you have some servers in the pool with the correct specs, and other which are not. Sometimes I get the error, other times not.

@emilgoldsmith
Copy link

Yeah I'm also experiencing this problem right now, and even rebuilding is not working for me though it used to work and also worked intermittently but my last 15 attempts or so have all failed in a row. Do we have an estimate on when this can be fixed?

@Bregor
Copy link

Bregor commented Jan 25, 2017

Same here.

Kernel:

$ uname -r
4.9.5-040905-generic

Docker:

# docker info
Containers: 46
 Running: 15
 Paused: 0
 Stopped: 31
Images: 17
Server Version: 1.12.6
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.8.17-040817-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.796 GiB
Name: kube04
ID: BSSV:EKQL:B646:2JMV:JEPX:PPVG:GEPF:IOZF:WF3H:QDND:YLHQ:UEYN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Eventually tar returns following:

tar: ./node_modules/webpack/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/webpack/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/webpack: Directory renamed before its status could be extracted
tar: ./node_modules/envify/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/envify/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/envify: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base: Directory renamed before its status could be extracted
...

@jbernard1
Copy link

So to those of you wondering how to fix this. I followed @pommi 's direction of using bsdtar instead of tar. This is what I added to my docker file.

RUN apt-get install -y --no-install-recommends bsdtar
RUN export tar='bsdtar'

andrew-wharton added a commit to andrew-wharton/education-platform-prototyping that referenced this issue May 7, 2017
trajano added a commit to trajano/ci.docker.ibm-http-server that referenced this issue May 24, 2017
Moved the logic of building the images into docker.  Removed using any
absolute paths on the build system (i.e. $(pwd) which does not work
correctly on Windows)

Added .gitattributes to ensure line endings for files being used in the
Docker build containers are correct.

Please compare with whitespace checks turned off.

In order to support builds within the container bsdtar needs to be used
rather than the regular tar due to
docker/hub-feedback#727

The tar server was also removed in favor of copying from the container
to the "im" folder and imported back using Dockerfile for the final
image.

The step where the files are tarred in install_ihs was removed and put
into docker.  This makes it easier to debug issues with tar
trajano added a commit to trajano/ci.docker.ibm-http-server that referenced this issue May 24, 2017
Moved the logic of building the images into docker.  Removed using any
absolute paths on the build system (i.e. $(pwd) which does not work
correctly on Windows)

Added .gitattributes to ensure line endings for files being used in the
Docker build containers are correct.

Please compare with whitespace checks turned off.

In order to support builds within the container bsdtar needs to be used
rather than the regular tar due to
docker/hub-feedback#727

The tar server was also removed in favor of copying from the container
to the "im" folder and imported back using Dockerfile for the final
image.

The step where the files are tarred in install_ihs was removed and put
into docker.  This makes it easier to debug issues with tar
@Dan23
Copy link

Dan23 commented May 29, 2017

Do we know when this issue will be resolved in either the edge or stable release?

agmangas added a commit to agmangas/meteor-android-build that referenced this issue Jun 7, 2017
@markshust
Copy link

overlayfs is the root of all these issues. convert to aufs and it should resolve this.
cae4475a-3b12-11e7-91da-e73a44ad3b7c

@mmuman
Copy link

mmuman commented Feb 11, 2019

Anyone actually tried to fix the cause of this? Seems it's been 2 years and everyone has to workaround with bsdtar or other things…
And, sorry but why is it tagged as "feature-request" while it actually really is a bug???

@beenishkh
Copy link

This is 2019 and the issue is still there, on macOS. And by now, docker desktop doesn't support 'ausf' at all, and I can't use bdstar. Wasted whole day trying to work around it. Team : You had 3 years to solve this problem :). Your work impact people's live, you could've done better here. I mean, 3 years! And still no fix. :( In the meanwhile, how do I get around it ? :'(

@kammoh
Copy link

kammoh commented Mar 1, 2019

I'm surprised this has been tagged as a "feature-request". It looks like a nasty critical bug to me; a serious bug which has been pretty much overlooked for almost 3 years!

@jonathanstiansen
Copy link

Happening for me on Mac Mojave:

$ uname -a
Darwin JonosMaokRetina.hitronhub.home 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

@AntonioSchmidt
Copy link

For those who are having this issue with meteor, I was able to make it work following this script here:
https://github.com/jshimko/meteor-launchpad/blob/master/scripts/install-meteor.sh

I basically added these lines to my Dockerfile:

RUN apt-get update
RUN apt-get install -y --no-install-recommends bsdtar
RUN curl -v https://install.meteor.com -o /tmp/install_meteor.sh
RUN sed -i.bak "s/tar -xzf.*/bsdtar -xf \"\$TARBALL_FILE\" -C \"\$INSTALL_TMPDIR\"/g" /tmp/install_meteor.sh
RUN sh /tmp/install_meteor.sh

I hope this helps someone.

@rpavlovicz
Copy link

Either rebuilding my images or setting "RUN export tar='bsdtar'" in the Dockerfile previously worked for me when i had this problem (i was trying too many things at once to know what the fix was), but now the tar issue has presented itself again. Maybe due to an OSX update? Either way, nothing is helping now.

@MrCavallo
Copy link

MrCavallo commented Apr 1, 2019

I'm relatively new to Docker, running Docker Desktop on a Mac, and with my very first project, I hit this bug. Tar is completely useless, bsdtar will not work in my case, and aufs is no longer an option. What this means is that for me, Docker is completely useless.

Given that this has been an extant bug for literally years with no solution, I think I'll be deleting Docker as a purely practical matter. If the devs don't have time to deal with this, then I certainly don't.

@ahomatthew
Copy link

ahomatthew commented Apr 4, 2019

For those who are having this issue with meteor, I was able to make it work following this script here:
https://github.com/jshimko/meteor-launchpad/blob/master/scripts/install-meteor.sh

I basically added these lines to my Dockerfile:

RUN apt-get update
RUN apt-get install -y --no-install-recommends bsdtar
RUN curl -v https://install.meteor.com -o /tmp/install_meteor.sh
RUN sed -i.bak "s/tar -xzf.*/bsdtar -xf \"\$TARBALL_FILE\" -C \"\$INSTALL_TMPDIR\"/g" /tmp/install_meteor.sh
RUN sh /tmp/install_meteor.sh

I hope this helps someone.

@AntonioSchmidt you did help someone

@Heermosi
Copy link

This is 2019 and the issue is still there, on macOS. And by now, docker desktop doesn't support 'ausf' at all, and I can't use bdstar. Wasted whole day trying to work around it. Team : You had 3 years to solve this problem :). Your work impact people's live, you could've done better here. I mean, 3 years! And still no fix. :( In the meanwhile, how do I get around it ? :'(

As a Chinese, I simply agree you on this......
If such things happen in China, the group would be fired if they cannot find a way to resolve it in weeks.
I'm sure, 3 years is enough to fire 70+ groups of maintainers.

I got stuck on this issue for weeks, carefully checked all the limits. Fun, I've learnt a lot of things about linux(or damm it, why should people wasting time searching for a reason?) , and my boss shall send some one to assist me(or replace me?).

@darran-kelinske-fivestars

What is a practical fix for this in 2019 on macOS running Docker Desktop

@beenishkh
Copy link

Run linux on mac via a vagrant VM and run your code there.

@mg262
Copy link

mg262 commented Jul 5, 2019

Working on a Mac, I found this issue but resolved it by upgrading my Dockerfile base image from Ubuntu 16.04 to 18.04. (See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728489) . I'm a bit confused because I thought it was the overlayfs in the host OS that caused the issue, not the guest OS, but hey, it works.

Edit: it makes the error less frequent, but it still happens. I think the reason is that Docker for Mac is on kernel 4.9, and the issue is fixed in kernel 4.11. I recommend downgrading tothe 25 July 2018 stable release and then using aufs instead of overlayfs. [In Preferences/Daemon/Advanced, replace the contents with {"storage-driver" : "aufs"}.]

@supershabam
Copy link

I'm running into this issue running latest stable docker for mac. Hoping somebody gets assigned to this bug.

@ishandrov
Copy link

@keyscores We apologize for the delay wrt this issue. Indeed, the build failure appears related to moby/moby#19647 and once the upstream overlayfs kernel update is available, Docker Cloud and Docker Hub will apply the the corresponding version update to address this issue.

I'm still getting the bug on Mac. When are you going to fix it?! I sense it won't be fixed at all, right?!

@ryanmickler
Copy link

supposedly fixed in ubuntu xenial:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728489

@geissonator
Copy link

Didn't fix it for me on my mac with any of the recent ubuntu's (eoan, bionic, xenial).

@leafbot
Copy link

leafbot commented Oct 11, 2019

This solved it:

WORKDIR /
RUN curl https://install.meteor.com/ | /bin/sh
WORKDIR /dir/to/app
COPY . .
RUN meteor npm install

or.. its just that I increased memory and swap in docker settings.

@pruthvi6767
Copy link

On mac : used gtar 
On RHEL: installed libarchive and export tar=bsdtar

@gaebor
Copy link

gaebor commented Nov 8, 2019

Same error :( on Windows Docker Desktop

FROM ubuntu:18.04

RUN apt update && apt -y upgrade
RUN apt -y install python libpng-dev wget
RUN cd /opt \
&& wget -O - -o /dev/null ftp://ftp.fu-berlin.de/unix/misc/sage/linux/64bit/sage-8.9-Ubuntu_18.04-x86_64.tar.bz2 | tar -xjf -

bsdtar works though

@huguesbrunelle
Copy link

As @leafbot said : Increased memory and swap in docker settings worked for me.

@gaebor
Copy link

gaebor commented Dec 16, 2019

@huguesbrunelle I tried extracting a 1.7GB tar file into ~6GB data, with 4.5GB ram and 1GB swap still the same error. Unfortunately, I cannot give more memory to the VM :(

@abhinavsingi
Copy link

I deleted disk image from - Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw
and then restarted Docker - it fixed the issue for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests