Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exec user process caused "exec format error" when run container with CMD on RHEL #475

Closed
gouyang opened this issue Feb 13, 2018 · 34 comments
Assignees

Comments

@gouyang
Copy link

@gouyang gouyang commented Feb 13, 2018

Download below two files to a directory and build an image from it.

https://github.com/docker-library/hello-world/tree/master/amd64/hello-world
https://github.com/docker-library/hello-world/blob/master/amd64/hello-world/hello

on RHEL 7.4, it throws an error as below. No this error on Fedora 27.

# buildah bud -t testing/hello .
STEP 1: FROM scratch
STEP 2: COPY hello /
STEP 3: CMD ["/hello"]
STEP 4: COMMIT containers-storage:[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.override_kernel_check=true]docker.io/testing/hello:latest

# buildah images 
IMAGE ID             IMAGE NAME                                               CREATED AT             SIZE
f4ff539da64e         docker.io/testing/hello:latest                           Feb 13, 2018 08:27     3.699 KB
# buildah from docker.io/testing/hello
hello-working-container
# buildah run hello-working-container
standard_init_linux.go:190: exec user process caused "exec format error"
@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Feb 13, 2018

@gouyang Thanks for the issue report, we'll take a look at it.

@rhatdan

This comment has been minimized.

Copy link
Member

@rhatdan rhatdan commented Feb 27, 2018

@gouyang Could you try to build buildah from master and see if you get this issue. I just built buildah-0.15 for RHEL, but it will take a few weeks to get released.

@gouyang

This comment has been minimized.

Copy link
Author

@gouyang gouyang commented Feb 28, 2018

The issue is still existing by building buildah from master.

# ./buildah run hello-working-container
standard_init_linux.go:190: exec user process caused "exec format error"
# ./buildah version
Version:       0.15
Go Version:    go1.8.1
Image Spec:    1.0.0
Runtime Spec:  1.0.0
Git Commit:    d1330a5
Built:         Wed Feb 28 19:59:37 2018
OS/Arch:       linux/amd64
@rhatdan

This comment has been minimized.

Copy link
Member

@rhatdan rhatdan commented Feb 28, 2018

THis could be a runc version error.
runc version

@TomSweeneyRedHat TomSweeneyRedHat self-assigned this Feb 28, 2018
@gouyang

This comment has been minimized.

Copy link
Author

@gouyang gouyang commented Mar 1, 2018

It's the same result with runc build from master.

# runc --version
runc version 1.0.0-rc5+dev
commit: aada2af1b2ffb3538d967878ad55f9365c77a0a8
spec: 1.0.0
@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Mar 2, 2018

I finally had some time to play and wasn't able to duplicate this. I tried both Buildah 0.12 and Buildah 0.15 and everything worked for me. Our runc versions are different, I just used what ever was installed with RHEL rather than updating that.

# buildah bud -t testing/hello .
STEP 1: FROM scratch
STEP 2: COPY hello /
STEP 3: CMD ["/hello"]
STEP 4: COMMIT containers-storage:[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.override_kernel_check=true]docker.io/testing/hello:latest
# buildah --version
buildah version 0.15 (image-spec 1.0.0, runtime-spec 1.0.0)
[root@localhost tmp]# cat /etc/*release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="OpenShift Enterprise"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"
Red Hat Enterprise Linux Server release 7.4 (Maipo)
Red Hat Enterprise Linux Server release 7.4 (Maipo)
[root@localhost tmp]# runc --version
runc version spec: 1.0.0

I then upgraded runc to the latest version and had the same result:

# buildah bud -t testing/hello .
STEP 1: FROM scratch
STEP 2: COPY hello /
STEP 3: CMD ["/hello"]
STEP 4: COMMIT containers-storage:[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.override_kernel_check=true]docker.io/testing/hello:latest
[root@localhost tmp]# runc version
No help topic for 'version'
[root@localhost tmp]# runc --version
runc version 1.0.0-rc5+dev
commit: ce80fa0a64803d52883955cb77b2708b438a0b28
spec: 1.0.0

If possible, could you retest on a new VM with RHEL @gouyang ?

@gouyang

This comment has been minimized.

Copy link
Author

@gouyang gouyang commented Mar 3, 2018

It's not reproduced on a new VM RHEL guest. It's strange it 's always reproduced on one of my RHEL system no matter reboot the system or reinstall buildah+runc on it.

@gouyang

This comment has been minimized.

Copy link
Author

@gouyang gouyang commented Mar 3, 2018

It seems the problem of the hello file, built a binary locally works fine.

# cat hello.go 
package main

import "fmt"

func main() {
	fmt.Println("hello world")
}
# go build -o hello .
# ./hello 
hello world
# buildah bud -t hello2 .
STEP 1: FROM scratch
STEP 2: COPY hello /
STEP 3: CMD ["/hello"]
STEP 4: COMMIT containers-storage:[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.override_kernel_check=true]docker.io/library/hello2:latest
# buildah from hello2
hello2-working-container
# buildah run hello2-working-container
hello world
@gouyang gouyang closed this Mar 3, 2018
@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Mar 3, 2018

@gouyang thanks for the follow up. That's strange that the binary was problematic on just that one machine. If you run into that again, please reopen or open a new issue.

@momocow

This comment has been minimized.

Copy link

@momocow momocow commented Apr 17, 2018

@gouyang Make sure if there is a shebang in hello.

Though I'm not familiar with go, I happen to find here since I met with the same problem (standard_init_linux.go:190: exec user process caused "exec format error").

I think it is because the system does not know how to execute the file.
FYI: What's the appropriate Go shebang line?

Also be aware of the difference between the shell form and exec form of CMD in Dockerfile.

@khalidumarr

This comment has been minimized.

Copy link

@khalidumarr khalidumarr commented Jun 17, 2018

just adding #!/bin/bash to my entry point file fixed the issue.

@pixdrift

This comment has been minimized.

Copy link
Collaborator

@pixdrift pixdrift commented Jun 17, 2018

@khalidumarr, which version of buildah are you using? Did you execute with buildah run and you received this error?

@khalidumarr

This comment has been minimized.

Copy link

@khalidumarr khalidumarr commented Jun 17, 2018

i was using docker run directly.

@ealexhaywood

This comment has been minimized.

Copy link

@ealexhaywood ealexhaywood commented Jul 16, 2018

Why should there be a shebang in hello? It's a compiled binary

@TrayserCassa

This comment has been minimized.

Copy link

@TrayserCassa TrayserCassa commented Jul 23, 2018

We build with an wrong architecture...
CGO_ENABLED=0
GOOS=linux
GOARCH=amd64

and then:
go build -o bin/serviceXYZ_linux_amd64

Our Image is for godeployment and we get this error. Maybe it will help somebody..

@rhatdan

This comment has been minimized.

Copy link
Member

@rhatdan rhatdan commented Jul 23, 2018

@TrayserCassa What error?

@TrayserCassa

This comment has been minimized.

Copy link

@TrayserCassa TrayserCassa commented Sep 4, 2018

@rhatdan
Sorry for the late answering...
Like in the title in this issue: 'exec format error'

@rhatdan

This comment has been minimized.

Copy link
Member

@rhatdan rhatdan commented Sep 4, 2018

There is probably a good change the runc version is out of date.
rpm -q runc

@adambant

This comment has been minimized.

Copy link

@adambant adambant commented Sep 14, 2018

just adding #!/bin/bash to my entry point file fixed the issue.

Turns out the #!/bin/bash was in my entry point file, but since I did a copy and paste into that file, the first line was a newline, not the #!/bin/bash, effectively ignoring it. If this helps anyone as well. Deleted the empty line and all worked fine.

@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Sep 15, 2018

@adambant TYVM for the follow up and info on your work around.

@Kiran01bm

This comment has been minimized.

Copy link

@Kiran01bm Kiran01bm commented Sep 18, 2018

@gouyang Make sure if there is a shebang in hello.

Though I'm not familiar with go, I happen to find here since I met with the same problem (standard_init_linux.go:190: exec user process caused "exec format error").

I think it is because the system does not know how to execute the file.
FYI: What's the appropriate Go shebang line?

Also be aware of the difference between the shell form and exec form of CMD in Dockerfile.

FWIW - I had the same error (copied below) with a ruby container and the problem was due to a missing shebang in my ruby program. I was using CMD instruction in a exec form (CMD instruction copied below)

Error

standard_init_linux.go:190: exec user process caused "exec format error"

CMD:

CMD ["somefile.rb"]

@k4korea

This comment has been minimized.

Copy link

@k4korea k4korea commented Nov 22, 2018

good. best quality .. answer very very good

@jomach

This comment has been minimized.

Copy link

@jomach jomach commented Dec 4, 2018

in my case the entrypoint.sh had the wrong shebang

@fractos

This comment has been minimized.

Copy link

@fractos fractos commented Jan 19, 2019

In case anyone finds this useful, you can get this issue if your shell script uses CRLF for line endings and/or UTF-8 with BOM (e.g. if you created a shell script file in Visual Studio). Changing to LF only and straight UTF-8 fixed it for me.

@limup

This comment has been minimized.

Copy link

@limup limup commented Jan 26, 2019

just adding #!/bin/bash to my entry point file fixed the issue.

Thanks soooooo much!!!!

Now... I have a new error:

/bin/bash: /docker-entrypoint.sh: Permission denied

=( =(

@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Jan 26, 2019

@limup can you open a new issue please and send us details if you've not figured out the issue yet?

@limup

This comment has been minimized.

Copy link

@limup limup commented Jan 26, 2019

@limup can you open a new issue please and send us details if you've not figured out the issue yet?

Well, I fixed the problem. I added the command in the Dockerfile and execute:

`ENTRYPOINT ["/docker-entrypoint.sh"]
RUN chmod +x /docker-entrypoint.sh`

[~See ya]

@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Jan 27, 2019

Thanks for the follow up and solution @limup !

@rhatdan

This comment has been minimized.

Copy link
Member

@rhatdan rhatdan commented Jan 28, 2019

@limup I would rather you called it container-entrypoint, and ran it with podman :^)

@limup

This comment has been minimized.

Copy link

@limup limup commented Jan 29, 2019

@limup I would rather you called it container-entrypoint, and ran it with podman :^)

@rhatdan I'm curious with the podman... what's it? And what is it useful for?

@TomSweeneyRedHat

This comment has been minimized.

Copy link
Collaborator

@TomSweeneyRedHat TomSweeneyRedHat commented Jan 29, 2019

@limup Podman is a related project to Buildah. It lives under the libpod project at: https://github.com/containers/libpod and has it's own website https://podman.io. In short, it's a OCI compliant container system. buildah run emulates what happens with the RUN command in a Dockerfile. podman run is the equivalent of the cli docker run command. In fact you can alias podman for docker and it should be gtg on your system. For more details, please see this blog: https://developers.redhat.com/blog/2018/11/20/buildah-podman-containers-without-daemons/

@limup

This comment has been minimized.

Copy link

@limup limup commented Jan 29, 2019

Thanks for sharing it.

@akshayeshenoi

This comment has been minimized.

Copy link

@akshayeshenoi akshayeshenoi commented Apr 17, 2019

just adding #!/bin/bash to my entry point file fixed the issue.

For anyone who got a standard_init_linux.go:190: exec user process caused "no such file or directory" error after applying this fix, you're probably on an alpine base image which does not come with bash.

Replacing #!/bin/bash with #!/bin/sh will do the trick!

@koursoh

This comment has been minimized.

Copy link

@koursoh koursoh commented Sep 9, 2019

just adding #!/bin/bash to my entry point file fixed the issue.

By entry point, it means to the top of the .sh file itself. For some people, it might not be clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.