Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v4.0.x: Do not use CMA in user namespaces #6997

Merged
merged 1 commit into from
Sep 22, 2019

Conversation

jsquyres
Copy link
Member

Trying out to run processes via mpirun in Podman containers has shown
that the CMA btl_vader_single_copy_mechanism does not work when user
namespaces are involved.

Creating containers with Podman requires at least user namespaces to be
able to do unprivileged mounts in a container

Even if running the container with user namespace user ID mappings which
result in the same user ID on the inside and outside of all involved
containers, the check in the kernel to allow ptrace (and thus
process_vm_{read,write}v()), fails if the same IDs are not in the same
user namespace.

One workaround is to specify '--mca btl_vader_single_copy_mechanism none'
and this commit adds code to automatically skip CMA if user namespaces
are detected and fall back to MCA_BTL_VADER_EMUL.

Signed-off-by: Adrian Reber areber@redhat.com
(cherry picked from commit fc68d8a)

Trying out to run processes via mpirun in Podman containers has shown
that the CMA btl_vader_single_copy_mechanism does not work when user
namespaces are involved.

Creating containers with Podman requires at least user namespaces to be
able to do unprivileged mounts in a container

Even if running the container with user namespace user ID mappings which
result in the same user ID on the inside and outside of all involved
containers, the check in the kernel to allow ptrace (and thus
process_vm_{read,write}v()), fails if the same IDs are not in the same
user namespace.

One workaround is to specify '--mca btl_vader_single_copy_mechanism none'
and this commit adds code to automatically skip CMA if user namespaces
are detected and fall back to MCA_BTL_VADER_EMUL.

Signed-off-by: Adrian Reber <areber@redhat.com>
(cherry picked from commit fc68d8a)
@jsquyres
Copy link
Member Author

FYI @adrianreber

@jsquyres jsquyres added the NEWS label Sep 21, 2019
@jsquyres
Copy link
Member Author

bot:lanl:retest

Copy link
Member

@gpaulsen gpaulsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very clean code.

@gpaulsen
Copy link
Member

gpaulsen commented Sep 21, 2019

@jsquyres Why the WIP label?

@jsquyres
Copy link
Member Author

jsquyres commented Sep 21, 2019

WIP was a mistake -- meant to put WIP on the corresponding 3.0.x/3.1.x PRs. This PR is good to go for v4.0.x.

@gpaulsen gpaulsen merged commit 2f10132 into open-mpi:v4.0.x Sep 22, 2019
@jsquyres jsquyres deleted the pr/v4.0.x/vader-do-not-use-cma branch September 25, 2019 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants