apply_k3s_selinux fails on MicroOS snapshot of 2023-01-23 #528

valkenburg-prevue-ch · 2023-01-24T10:17:05Z

The script apply_k3s_selinux = ["/sbin/semodule -v -i /usr/share/selinux/packages/k3s.pp"] fails as of this morning, with the following output:

k3s-agent-large-hel1-djs:~ # /sbin/semodule -v -i /usr/share/selinux/packages/k3s.pp
Attempting to install module '/usr/share/selinux/packages/k3s.pp':
Ok: return value of 0.
Committing changes:
Conflicting name type transition rules
Binary policy creation failed at /var/lib/selinux/targeted/tmp/modules/400/k3s/cil:135
Failed to generate binary
/sbin/semodule:  Failed!

I have the strong suspicion that something changed in microos, as the image has been updated yesterday:

Does anyone know how to work around this?

The text was updated successfully, but these errors were encountered:

valkenburg-prevue-ch · 2023-01-24T10:30:21Z

If I make this modification in locals.tf,

  # apply_k3s_selinux = ["/sbin/semodule -v -i /usr/share/selinux/packages/k3s.pp"]
  apply_k3s_selinux = ["echo 'Hello unsafe world!'"]

I confirm that the cluster comes up. However, this made me notice an error message which is probably leading closer to the issue:

The following NEW package is going to be installed:
k3s-selinux
...
Conflicting name type transition rules
Binary policy creation failed at /var/lib/selinux/targeted/tmp/modules/200/k3s/cil:135
Failed to generate binary
/usr/sbin/semodule:  Failed!

However, this message does not block the rest of the installation, it just continues.

thebearingedge · 2023-01-24T16:36:11Z

I noticed this just a few minutes ago. I'm searching the repository history but still haven't quite grasped the rationale for skipping the automatic selinux installation that comes with the k3s installer.

I haven't learned the consequences yet, but letting the k3s install script include it produces no error.

Edit: oh maybe it's just to allow modification to the configurations before the k3s service is started?

aleksasiriski · 2023-01-24T19:31:27Z

Same thing here, guess something changed outside of this script (either MicroOS or K3S script) since the latest version worked 2 days ago flawlessly.

mysticaltech · 2023-01-24T19:52:37Z

We probably just need to update the k3s SELinux RPM! On it.

mysticaltech · 2023-01-24T22:45:03Z

@valkenburg-prevue-ch You are right something changed with MicroOS, breaking the k3s-SELinux install.

mysticaltech · 2023-01-25T00:03:30Z

I noticed this just a few minutes ago. I'm searching the repository history but still haven't quite grasped the rationale for skipping the automatic selinux installation that comes with the k3s installer.

I haven't learned the consequences yet, but letting the k3s install script include it produces no error.

Edit: oh maybe it's just to allow modification to the configurations before the k3s service is started?

@thebearingedge In the current state of things, this would have failed too. We just install it manually and apply the policy ourselves because MicroOS installs all new packages to a new Snapshot, and requires a reboot. We remove the need for that extra reboot, but installing the package manually before k3s is installed.

mysticaltech · 2023-01-25T00:05:00Z

If I make this modification in locals.tf,
  # apply_k3s_selinux = ["/sbin/semodule -v -i /usr/share/selinux/packages/k3s.pp"]
  apply_k3s_selinux = ["echo 'Hello unsafe world!'"]
I confirm that the cluster comes up. However, this made me notice an error message which is probably leading closer to the issue:
The following NEW package is going to be installed:
k3s-selinux
...
Conflicting name type transition rules
Binary policy creation failed at /var/lib/selinux/targeted/tmp/modules/200/k3s/cil:135
Failed to generate binary
/usr/sbin/semodule:  Failed!
However, this message does not block the rest of the installation, it just continues.

@valkenburg-prevue-ch This basically disables k3s-selinux. It a neat trick but not ideal.

I have signaled our problem on the k3s-selinux repo. It turns out even folks on Fedora CoreOS are having the exact same error! So not specific to MicroOS.

mysticaltech · 2023-01-25T02:01:23Z

Ok, the solution has been found, thanks to the tips given by the maintainer of k3s-selinux.

It required a downgrade of container-selinux, and he is working on fix to support newer versions! 🤞

valkenburg-prevue-ch · 2023-01-27T09:51:03Z

Oh nooooo! The fix is broken today. The line:

transactional-update shell <<< "zypper install -y --oldpackage https://download.opensuse.org/tumbleweed/repo/oss/noarch/container-selinux-2.188.0-2.1.noarch.rpm && zypper addlock container-selinux"

fails because
https://download.opensuse.org/tumbleweed/repo/oss/noarch/container-selinux-2.188.0-2.1.noarch.rpm no longer exists!

Reopening the issue.

valkenburg-prevue-ch · 2023-01-27T11:24:08Z

Ok, I missed one more commit of the master branch, it seems fixed there already. Thanks @mysticaltech and @aleksasiriski .

mysticaltech · 2023-01-27T14:19:19Z

Apologies folks, I screwed up the link. Thanks for fixing it @aleksasiriski 🙏

mariusleu · 2024-08-17T13:42:54Z

This error hits again on a fresh MicroOS image:

module.kube-hetzner.null_resource.first_control_plane (remote-exec): + curl -sfL https://get.k3s.io
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + INSTALL_K3S_SKIP_START=true
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + INSTALL_K3S_SKIP_SELINUX_RPM=true
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + INSTALL_K3S_CHANNEL=stable
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + INSTALL_K3S_EXEC='server '
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + sh -
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Finding release for channel stable
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Using v1.30.3+k3s1 as release
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/sha256sum-amd64.txt
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/k3s
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Verifying binary download
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Installing k3s to /usr/local/bin/k3s
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Skipping installation of SELinux RPM
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Skipping /usr/local/bin/kubectl symlink to k3s, already exists
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Skipping /usr/local/bin/crictl symlink to k3s, already exists
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Skipping /usr/local/bin/ctr symlink to k3s, already exists
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
module.kube-hetzner.null_resource.first_control_plane (remote-exec): [INFO]  systemd: Enabling k3s unit
module.kube-hetzner.null_resource.first_control_plane (remote-exec): Created symlink '/etc/systemd/system/multi-user.target.wants/k3s.service' → '/etc/systemd/system/k3s.service'.
module.kube-hetzner.null_resource.first_control_plane (remote-exec): + /sbin/semodule -v -i /usr/share/selinux/packages/k3s.pp
module.kube-hetzner.null_resource.first_control_plane (remote-exec): Attempting to install module '/usr/share/selinux/packages/k3s.pp':
module.kube-hetzner.null_resource.first_control_plane (remote-exec): libsemanage.map_compressed_file: Unable to open /usr/share/selinux/packages/k3s.pp
module.kube-hetzner.null_resource.first_control_plane (remote-exec):  (No such file or directory).
module.kube-hetzner.null_resource.first_control_plane (remote-exec): libsemanage.semanage_direct_install_file: Unable to read file /usr/share/selinux/packages/k3s.pp
module.kube-hetzner.null_resource.first_control_plane (remote-exec):  (No such file or directory).
module.kube-hetzner.null_resource.first_control_plane (remote-exec): /sbin/semodule:  Failed on /usr/share/selinux/packages/k3s.pp!
╷
│ Error: remote-exec provisioner error
│ 
│   with module.kube-hetzner.null_resource.first_control_plane,
│   on .terraform/modules/kube-hetzner/init.tf line 70, in resource "null_resource" "first_control_plane":
│   70:   provisioner "remote-exec" {
│ 
│ error executing "/tmp/terraform_102051961.sh": Process exited with status 1

Anyone else experiencing this?

valkenburg-prevue-ch mentioned this issue Jan 24, 2023

Hetzner Cloud Networks #527

Closed

mysticaltech mentioned this issue Jan 24, 2023

Incompatible with container-selinux > 2.188 k3s-io/k3s-selinux#36

Open

mysticaltech mentioned this issue Jan 25, 2023

Emergency downgrade of the container-selinux package + tiny improvements #533

Merged

mysticaltech closed this as completed in #533 Jan 25, 2023

vesurbag mentioned this issue Jan 25, 2023

Error: remote-exec provisioner error #534

Closed

valkenburg-prevue-ch reopened this Jan 27, 2023

valkenburg-prevue-ch closed this as completed Jan 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apply_k3s_selinux fails on MicroOS snapshot of 2023-01-23 #528

apply_k3s_selinux fails on MicroOS snapshot of 2023-01-23 #528

valkenburg-prevue-ch commented Jan 24, 2023

valkenburg-prevue-ch commented Jan 24, 2023 •

edited

Loading

thebearingedge commented Jan 24, 2023 •

edited

Loading

aleksasiriski commented Jan 24, 2023

mysticaltech commented Jan 24, 2023

mysticaltech commented Jan 24, 2023

mysticaltech commented Jan 25, 2023

mysticaltech commented Jan 25, 2023 •

edited

Loading

mysticaltech commented Jan 25, 2023 •

edited

Loading

valkenburg-prevue-ch commented Jan 27, 2023

valkenburg-prevue-ch commented Jan 27, 2023

mysticaltech commented Jan 27, 2023 •

edited

Loading

mariusleu commented Aug 17, 2024

apply_k3s_selinux fails on MicroOS snapshot of 2023-01-23 #528

apply_k3s_selinux fails on MicroOS snapshot of 2023-01-23 #528

Comments

valkenburg-prevue-ch commented Jan 24, 2023

valkenburg-prevue-ch commented Jan 24, 2023 • edited Loading

thebearingedge commented Jan 24, 2023 • edited Loading

aleksasiriski commented Jan 24, 2023

mysticaltech commented Jan 24, 2023

mysticaltech commented Jan 24, 2023

mysticaltech commented Jan 25, 2023

mysticaltech commented Jan 25, 2023 • edited Loading

mysticaltech commented Jan 25, 2023 • edited Loading

valkenburg-prevue-ch commented Jan 27, 2023

valkenburg-prevue-ch commented Jan 27, 2023

mysticaltech commented Jan 27, 2023 • edited Loading

mariusleu commented Aug 17, 2024

valkenburg-prevue-ch commented Jan 24, 2023 •

edited

Loading

thebearingedge commented Jan 24, 2023 •

edited

Loading

mysticaltech commented Jan 25, 2023 •

edited

Loading

mysticaltech commented Jan 25, 2023 •

edited

Loading

mysticaltech commented Jan 27, 2023 •

edited

Loading