exec: "/bin/sh": stat /bin/sh: no such file or directory

[ConradIrwin]

Since upgrading to 0.10 I see this error during build processes.

I'm not sure exactly what causes it, it always happens at the same point in a given Dockerfile, but I can't see the correlation between the failures in different Dockerfiles.

It may be to do with the depth of the aufs tree, as it only seems to happen on long dockerfiles.

[LK4D4]

What base image in this Dockerfile?

[ConradIrwin]

FROM phusion/passenger-minimal:latest

[emilf]

I had the same issue with phusion/baseimage:latest
/bin/sh exists in the image and exporting the image, extracting the tar and running sh works fine.

[emilf]

NB. running this:
docker run -i -t ubuntu /bin/bash

Results in this:

Unable to find image 'ubuntu' locally
Pulling repository ubuntu
c0fe63f9a4c1: Download complete 
e20bcab99567: Download complete 
25593492b938: Download complete 
ab4344e23e3a: Download complete 
f697cdc2ef19: Download complete 
511136ea3c5a: Download complete 
9a8b9d29f4fc: Download complete 
6170bb7b0ad1: Download complete 
79fdb1362c84: Download complete 
01bdd5cc09de: Download complete 
1c7f181e78b9: Download complete 
f323cf34fd77: Download complete 
d0732e6ce563: Download complete 
9109d385566e: Download complete 
13dec63ebd94: Download complete 
2014/04/10 22:50:16 no such file or directory

Running docker version returns this:

Client version: 0.10.0
Client API version: 1.10
Go version (client): go1.2.1
Git commit (client): dc9c28f
Server version: 0.10.0
Server API version: 1.10
Git commit (server): dc9c28f
Go version (server): go1.2.1
Last stable version: 0.10.0

uname -a && cat /etc/lsb-release 

Linux emil-lap 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=16
DISTRIB_CODENAME=petra
DISTRIB_DESCRIPTION="Linux Mint 16 Petra"

[afsheenb]

I've triggered this issue with docker 0.9 and 0.10.0. Setting '-s devicemapper' as one of my DOCKER_OPTS in /etc/default/docker fixed this. I'm assuming it's an AUFS related issue, like the 'too many layers' bug, like #1171.

[emilf]

Perhaps this is an aufs issue. The same dockerfile ran fine on a test VM running btrfs, instead of aufs. I'm not sure though. I'll try it with devicemapper when I get a chance.

[ConradIrwin]

I was using aufs. Looking at the tree it failed at around 42 layers, but I
could have been overfitting (as I read about the 42 layer limit being
fixed). This was on ubuntu 12.04.

Conrad

On Friday, April 11, 2014, emilf notifications@github.com wrote:

Perhaps this is an aufs issue. The same dockerfile ran fine on a test VM
running btrfs, instead of aufs. I'm not sure though. I'll try it with
devicemapper when I get a chance.

Reply to this email directly or view it on GitHubhttps://github.com//issues/5135#issuecomment-40270477
.

[harto]

Not sure if this is related, but I'm able to reproduce this issue when Docker is configured with -g to write to Amazon EC2 ephemeral storage (e.g. /mnt/docker).

Script to reproduce is here: https://gist.github.com/harto/10610004

Cross-posted to #4338, in case it's related.

[cordoval]

for me it fails on the very last part of the script, even though it uses it before for other commands, any hint on why? I am trying to do the EXPOSE lines after i do call further apt-get commands for cleaning it up

[GavinYin]

the same question when run docker run -i -t ubuntu /bin/bash then directly shows “exec: "/bin/bash": stat /bin/bash: no such file or directory”

[creack]

I think it is linked to apparmor. Can you try with master to see if it fixes the issue?

[flaub]

Just upgraded to 0.11 and am now seeing this issue.

[peikk0]

I've the same issue here with Docker 0.11. It happens right after the 42th layer too. Apparmor is disabled on the host.

[mitar]

Same happening for me for 29th layer.

[mitar]

This is image I was building. I am using Debian testing Docker package.

[mitar]

My docker version:

Client version: 1.0.0
Client API version: 1.12
Go version (client): go1.2.1
Git commit (client): 63fe64c
Server version: 1.0.0
Server API version: 1.12
Go version (server): go1.2.1
Git commit (server): 63fe64c

uname -a

Linux server2 3.14-0.bpo.1-amd64 #1 SMP Debian 3.14.7-1~bpo70+1 (2014-06-21) x86_64 GNU/Linux

[mitar]

It seems it is around 42 layer. If I pull now image built on the hub and I try to run it, it also ails with exit 1.

[bcicen]

Received the same titular error when trying to build exactly the 43rd layer with and without apparmor.

Linux stage 3.8.0-42-generic #63~precise1-Ubuntu SMP Fri Jul 4 21:16:26 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070

[cpuguy83]

Hmmm, custom kernels + aufs graph drivers?

[cpuguy83]

FYI, aufs can handle as many layers as it's compiled with. We originally had it limited at 42 within Docker because it was thought that was the lowest common denominator for everyone, later decided to bump that to 127 since that seemed to be for real the lowest common denominator.

But there is nothing keeping people from building a kernel with support for fewer layers, I think.

[Tranquility]

I have that problem too. I get this error in layer 40. If i download an older version of the 14.04 image from my server everything is fine.

I am using aufs with docker version 1.1.2 and kernel 3.14.x.

This is the error I get when the docker build command tries to run anything in layer 40:
Error response from daemon: Cannot start container 2b667fa3d31d8369550216db9b5df1f1f424b3622ef802b9c7184af4b4775395: exec: "bash": executable file not found in $PATH

[cpuguy83]

This seems like an issue with using a custom kernel.

[mitar]

With this custom kernel being Debian official kernel image?

[cpuguy83]

When I say custom kernel I mean one compiled yourself, or at least non-official (ie, not from Debian).

[mitar]

So, I have the issue of 42 layers on official Debian kernel (although from backports).

[sametmax]

Same.

Le 06/02/2015 16:43, Brian Goff a écrit :

@sametmax https://github.com/sametmax |/bin/bash|

—
Reply to this email directly or view it on GitHub
#5135 (comment).

[sametmax]

Eventually deleted all images and container, and now it works with a
fresh new one. It's on my dev machine, so it's ok. But for people with
containers in prod...

Le 06/02/2015 16:44, Sam & Max a écrit :

Same.

Le 06/02/2015 16:43, Brian Goff a écrit :

@sametmax https://github.com/sametmax |/bin/bash|

—
Reply to this email directly or view it on GitHub
#5135 (comment).

[mipearson]

Just got bitten by this. "Docker will fail if you do too many things with docker" is not expected behaviour, is not documented except in this github issue.

What are the changes of getting docker to warn if you start adding too many layers?

[madduci]

I'm getting the same error with docker 1.6 and Ubuntu-Core armhf 15.04 image imported from a tar.gz file. Any kind of command fails with "no such file or directory" even if the files are all there

[ghost]

We are seeing these symptoms too. I've tried reducing the number of layers but it doesn't solve the issue.
We're using the overlay driver on CoreOS fwiw.

[crosbymichael]

I'm going to close this issue but please open a new one if you hit this issue with steps to reproduce and information. Just saying that you encountered this is not going to help anyone fixing the problem. We need to have the image/Dockerfile, etc, and your system information.

This issue has become a catchall and there are not really any useful information in here to solve the problem. So if you still have this problem, please open a new issue with all the information that you can. if you cannot share the information like the image publicly you can email me personally and I can take a look at it.

[math3usmartins]

Dockerfile

FROM node:0.12.7
EXPOSE 8080 443
RUN apt-get update
RUN apt-get install -y net-tools
RUN apt-get install -y git
RUN npm install -g pm2
CMD ["/bin/bash"]

Save and import image works, but docker run fails:

[ok] docker build myimage
[ok] docker save --output="myimage" myimage
[ok] docker rmi myimage
[ok] cat myimage.tar | docker import - myimage
[fail] docker run -tid --name myimagecontainer myimage 
exec: "/bin/sh": stat /bin/sh: no such file or directory
Error response from daemon: Cannot start container 017cec2fb...
[8] System error: exec: "/bin/sh": stat /bin/sh: no such file or directory

I've also tried docker import --change "CMD ..." - myimage and docker run -ti (without -d). No success.

PS. I've adjusted the container ID in the error message above just to improve readability; the error message contains the full ID.

[thaJeztah]

@math3usmartins you're combining docker save with docker import, which doesn't work. Either use docker save and docker load or docker export and docker import, depending what your goal is. See http://docs.docker.com/reference/commandline/save/ and http://docs.docker.com/reference/commandline/export/

[math3usmartins]

thanks a lot @thaJeztah

[prashantabkari]

I am facing the same problem when i try to create a base image from scratch
Following is the dockerfile
FROM scratch
ADD a.out /home/abkari/docker_file/server_docker/server.o
CMD ["/bin/bash"]

On running the container with the above image i get the following error
Error response from daemon: Cannot start container 29891a11a5915a3c81d9a3c4800c8a1b33ea32fad00da2335ece69e3dd856458: [8] System error: exec: "/bin/bash": stat /bin/bash: no such file or directory

[tcurdt]

@prashantabkari it's not a real problem - read the above comments.
With that dockerfile that's expected - there is no shell in that empty image.

[colemickens]

Is it expected that:

CMD /static-binary in my Dockerfile results in CMD ["/bin/sh", "-c", "/static-binary"] in the actual image?

In case anyone else is having issues, try specifically specifying CMD ["/static-binary"] in a scratch image.

Is this behavior documented?

[thaJeztah]

@colemickens yes, see https://docs.docker.com/engine/reference/builder/#/exec-form-entrypoint-example

[david-vega]

I just had this issue with the latest Docker version on a clean install of ubuntu-gnome 16.10, the reason was that docker-compose didn't like the volumes: definition on the docker-compose.yml file

volumes:
   - .:/data/app

My workaround was to delete that, as a side note I use the same files on my mac an they work perfectly fine. anyways leaving this comment if someone faces the same issue.

[fiantsev]

guys, this error may be related to corrupted images (in my case there was shutdown of a build machine during build process). Subsequent attempt to build lead to this problem. So i fixed this by removing all images (including hidden ones) because i cant rely of them anymore. My save-day command was:

docker rmi -f $(docker images -aq)

!!! be careful - it will remove all your images !!!

[cowlinator]

I am getting this error with the unmodified "hello-world" image:

$ docker run -it hello-world /bin/sh
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
d1725b59e92d: Pull complete
Digest: sha256:0add3ace90ecb4adbf7777e9aacf18357296e799f81cabc9fde470971e499788
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown.

Furthermore, docker run -it hello-world runs fine, and docker inspect hello-world reveals that the "Cmd" is indeed /bin/sh.

[thaJeztah]

@cowlinator the hello-world image is an image that's created "from scratch"; there's nothing in that image other than a single binary, so no /bin/sh. https://github.com/docker-library/hello-world/blob/master/amd64/hello-world/Dockerfile

and docker inspect hello-world reveals that the "Cmd" is indeed /bin/sh.

I think that may be a presentation issue; the image uses the "exec-form" for the Cmd;

CMD ["/hello"]

But the docker inspect may prepend the default entry point (/bin/sh) even though it's not used f the Cmd is using the exec-form 🤔

Might be worth opening an new issue for that, so that we can discuss if that presentation can be changed