Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fedora 34: create cluster fails with permission denied error on /dev/dma_heap #2296

Closed
elmiko opened this issue Jun 3, 2021 · 13 comments
Closed
Labels
kind/external upstream bugs

Comments

@elmiko
Copy link
Contributor

elmiko commented Jun 3, 2021

What happened:

After updating my Fedora Linux to the latest kernel (5.12.8-300.fc34.x86_64) i tried to run kind create cluster and hit this error:

Creating cluster "mgmt" ...                                                                                                               
 βœ“ Ensuring node image (kindest/node:v1.21.1) πŸ–Ό                                                                                           
 βœ— Preparing nodes πŸ“¦                                                                                                                     
ERROR: failed to create cluster: docker run error: command "docker run --hostname mgmt-control-plane --name mgmt-control-plane --label io.
x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run 
--volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --device /dev/fuse --detach --tty --label 
io.x-k8s.kind.cluster=mgmt --net kind --restart=on-failure:1 --init=false --volume=/var/run/docker.sock:/var/run/docker.sock --publish=127
.0.0.1:40963:6443/TCP -e KUBECONFIG=/etc/kubernetes/admin.conf kindest/node:v1.21.1@sha256:80773e2069dd4a80a4929fdef050c1e463e1d5578c2cd9a
f3962cfbc230e1500" failed with error: exit status 126                                                                                     
Command Output: 6a36fe59f40c4d4338b75c0b357d0ceb4f789319550fe61196c3f76dbdb8ec3b                                                          
docker: Error response from daemon: open /dev/dma_heap: permission denied.

What you expected to happen:

Cluster to be created as normal.

How to reproduce it (as minimally and precisely as possible):

  1. Install Fedora 34
  2. run dnf --refresh upgrade -y to update to latest kernel
  3. run kind create cluster

Anything else we need to know?:

Environment:

  • kind version: kind v0.12.0-alpha+1188d9bd86afbf go1.16.4 linux/amd64
  • Kubernetes version: 1.21.1
  • Docker version: 20.10.6
  • OS (e.g. from /etc/os-release): Fedora 34
@elmiko elmiko added the kind/bug Categorizes issue or PR as related to a bug. label Jun 3, 2021
@BenTheElder
Copy link
Member

Is this rootless docker by any chance?
https://kind.sigs.k8s.io/docs/user/rootless/

Can you share the rest of docker info?

@BenTheElder
Copy link
Member

I think this is a fedora bug actually, mislabeling the device in selinux https://bugzilla.redhat.com/show_bug.cgi?id=1966158

@elmiko
Copy link
Contributor Author

elmiko commented Jun 3, 2021

I think this is a fedora bug actually, mislabeling the device in selinux https://bugzilla.redhat.com/show_bug.cgi?id=1966158

yeah, that seems correct. i suppose we can close this then as it looks like they are putting out a fix in fedora. thanks for finding that bugzilla!

@elmiko elmiko closed this as completed Jun 3, 2021
@BenTheElder BenTheElder added kind/external upstream bugs and removed kind/bug Categorizes issue or PR as related to a bug. labels Jun 3, 2021
@BenTheElder
Copy link
Member

Thanks, Please let us know if anything changes!

@BenTheElder BenTheElder changed the title create cluster fails with permission denied error on /dev/dma_heap fedora 34: create cluster fails with permission denied error on /dev/dma_heap Jun 3, 2021
@christianh814
Copy link

So I just ran into this and it looks like the policy was fixed. A relabel/restorecon is needed, however.

All steps as root

Update the package

dnf -y update selinux-policy

Then before you relabel/restore; you need to set SELinux to permissive

setenforce 0

Now either relabel, or just restore from the updated policy

restorecon -vR /dev/dma_heap

Set SELinux to enforcing

setenforce 1

@BenTheElder
Copy link
Member

thanks @christianh814! πŸ™

@tao12345666333
Copy link
Member

FYI fedora-selinux/selinux-policy#763

@dlipovetsky
Copy link
Contributor

For anyone on Fedora 33, there is no backport as of June 28, 2021, so putting SELinux in permissive mode (setenforce 0) is the only option I am aware of.

@BenTheElder
Copy link
Member

@dlipovetsky thanks, that may need a known issues entry. @aojea at this point we might need to update the fedora-33 entry to just be more generally a sub-section on fedora known issues ...

@aojea
Copy link
Contributor

aojea commented Jun 28, 2021

@dlipovetsky thanks, that may need a known issues entry. @aojea at this point we might need to update the fedora-33 entry to just be more generally a sub-section on fedora known issues ...

The problem I see is how to keep up with the fedora release schedule

We say maintained for approximately 13 months because the supported period for releases is dependent on the date the release under development goes final. As a result, Release X is supported until one month (4 weeks) after the release of Release X+2.

Fedora Linux 33 EOL auto closure is 11/2021

@dlipovetsky
Copy link
Contributor

I PR'd the known issue. Once the Fedora 33 backport lands, it should be updated (to tell users to update their SELinux policy).

@BenTheElder
Copy link
Member

@aojea fair point, but the other entry is proving to not be version specific and we've had a number of issues tracked that aren't on other common distros. we might still need a page for this or something, we know OOTB there are issues to be aware of.

@aojea
Copy link
Contributor

aojea commented Jun 29, 2021

@aojea fair point, but the other entry is proving to not be version specific and we've had a number of issues tracked that aren't on other common distros. we might still need a page for this or something, we know OOTB there are issues to be aware of.

yep #2341

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/external upstream bugs
Projects
None yet
Development

No branches or pull requests

6 participants