Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider not messing with namespaces and clone(2) in 'version' #17657

Closed
debarshiray opened this issue Feb 28, 2023 · 4 comments · Fixed by #17684
Closed

Consider not messing with namespaces and clone(2) in 'version' #17657

debarshiray opened this issue Feb 28, 2023 · 4 comments · Fixed by #17684
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@debarshiray
Copy link
Member

debarshiray commented Feb 28, 2023

Issue Description

podman version ends up doing various things with namespaces (at least user) and the clone(2) system call that sometimes end up with an EPERM in various build environments like Fedora's. It might be worth short circuiting those because ultimately it's just returning some relatively simple metadata.

The background is that Toolbx recently switched to using Cobra to generate the shell completions instead of static hand-written ones. This means that toolbox completion ... is run during the Toolbx build, and for various not-so-important reasons the start-up code in toolbox(1) calls podman version.

Steps to reproduce the issue

Try to build Toolbx 0.0.99.4 for Fedora 36 or 37 in the Fedora build system

Describe the results you received

When building Toolbx 0.0.99.4 in the Fedora build system, the podman version invocation hits an EPERM on both Fedora 36 and 37 (but works fine on F38 and F39/Rawhide). Here is a scratch build with toolbox --verbose completion ... and strace --follow-forks podman version sprinkled in the right places. Note that there are three toolbox completion ... invocations there, so everything is repeated three times.

Looking closely at the strace output in the build log, you will see:

[pid 1066178] geteuid()                 = 1000
[pid 1066178] getegid()                 = 425
[pid 1066178] clone(child_stack=NULL, flags=CLONE_NEWNS|CLONE_NEWUSER|SIGCHLD) = -1 EPERM (Operation not permitted)
[pid 1066178] write(2, "cannot clone: Operation not perm"..., 38cannot clone: Operation not permitted
...
...
[pid 1066178] write(2, "Error: cannot re-exec process\n", 30Error: cannot re-exec process
) = 30

My understanding is that this is coming from becomeRootInUserNS() in pkg/rootless/rootless.go, which calls reexec_in_user_namespace in pkg/rootless/rootless_linux.c, where the clone(2) is.

So far, I see that the Fedora 36 and 37 build environment is inside a chroot(2), while on Fedora 38 and 39 it's inside systemd-nspawn(1), and glibc-2.36-9.fc37 versus glibc-2.37-1.fc38.x86_64. I don't see any differences in libseccomp.

Describe the results you expected

podman version shouldn't encounter an EPERM and return the Podman version.

podman info output

One would have to run podman info on the Fedora build system, so I am skipping it.

podman-4.4.1-3.fc37; Fedora 37 on aarch64, ppc64le, s390x and x86_64

Similarly, on Fedora 36, which also includes armv7hl.

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

A chroot(2) created by Mock 3.5 as part of the Fedora 36 and 37 build environment.

Additional information

I will try to work around this by avoiding the podman version call in Toolbx when generating the shell completions.

@debarshiray debarshiray added the kind/bug Categorizes issue or PR as related to a bug. label Feb 28, 2023
@Luap99
Copy link
Member

Luap99 commented Feb 28, 2023

Is there a reason why this isn't run as root? AFAIK as root we do not need to reexec at all.

That said it should be possible to make version not require the reexec.

@debarshiray
Copy link
Member Author

Is there a reason why this isn't run as root?

What do you mean by this? People, including distribution build systems, usually don't do builds as root, because why use it if it's not needed? :)

Even though an RPM might eventually place files in system locations like /usr on the end user's system, that doesn't happen when the RPM is created by the build system. So, there's really no need for root.

debarshiray added a commit to debarshiray/toolbox that referenced this issue Feb 28, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence invoking 'podman version' to decide
if 'podman system migrate' is needed or not.

Unfortunately, some build environments, like Fedora's, are set up inside
a chroot(2) or systemd-nspawn(1) or similar, where 'podman version' may
not work because it does various things with namespaces(7) and clone(2)
that can, under certain circumstances, encounter an EPERM.

Therefore, it's better to avoid using podman(1) when generating the
shell completions, especially, since they are generated by Cobra itself
and podman(1) is not involved at all.

Note that podman(1) is needed when the generated shell completions are
actually used in interactive command line environments.  The shell
completions invoke the hidden '__complete' command to get the results
that are presented to the user, and, if needed, 'podman system migrate'
will continue to be run as part of that.

This reverts commit f3e005d.

containers/podman#17657
debarshiray added a commit to debarshiray/toolbox that referenced this issue Feb 28, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence invoking 'podman version' to decide
if 'podman system migrate' is needed or not.

Unfortunately, some build environments, like Fedora's, are set up inside
a chroot(2) or systemd-nspawn(1) or similar, where 'podman version' may
not work because it does various things with namespaces(7) and clone(2)
that can, under certain circumstances, encounter an EPERM.

Therefore, it's better to avoid using podman(1) when generating the
shell completions, especially, since they are generated by Cobra itself
and podman(1) is not involved at all.

Note that podman(1) is needed when the generated shell completions are
actually used in interactive command line environments.  The shell
completions invoke the hidden '__complete' command to get the results
that are presented to the user, and, if needed, 'podman system migrate'
will continue to be run as part of that.

This partially reverts commit f3e005d
because podman(1) is now only an optional runtime dependency for the
system tests.

containers/podman#17657
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.  eg., the inability to use podman(1) in the
case of Fedora or not having subordinate user and group ID ranges in the
case of openSUSE.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.  eg., the inability to use podman(1) in the
case of Fedora or not having subordinate user and group ID ranges in the
case of openSUSE.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 2, 2023
Closes: containers#17657

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 2, 2023
Closes: containers#17657

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
Copy link
Member

giuseppe commented Mar 2, 2023

I've opened a PR to not join the rootless userns with "podman version": #17684

although I am not sure how useful is that, if you cannot join a user namespace you cannot use podman rootless

giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 3, 2023
Closes: containers#17657

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 3, 2023
Closes: containers#17657

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 3, 2023
Closes: containers#17657

[NO NEW TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
giuseppe added a commit to giuseppe/libpod that referenced this issue Mar 3, 2023
Closes: containers#17657

[NO NEW TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@debarshiray
Copy link
Member Author

I've opened a PR to not join the rootless userns with "podman version": #17684

Thanks, @giuseppe !

although I am not sure how useful is that, if you cannot join a user namespace you cannot use podman rootless

I suppose it's like dnf repolist. One can't really use dnf(8) without root, but some commands short circuit that.

In this particular case, podman version not messing with namespaces and clone(2) would have helped with the Toolbx build, but we already dodged that in Toolbx itself. Maybe it will prove useful at some other point in the future?

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Aug 30, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants