RFE: safe SELinux policy updates

[dustymabe]

Right now SELinux stores its policy as binary files in /etc/selinux/ and that's how it's delivered in the images we build/ship. If a user updates the SELinux policy with a local persistent modification (such as running setsebool -P) then the binary files in /etc/selinux/ get overwritten and now they are considered to be "changed" by ostree and are no longer updated when a system upgrades because files in /etc/ are "configuration".

This means if people make a permanent local policy modification they cease to get SELinux policy updates, which is bad.

To workaround this for now people can apply their local policy modifications non persistently on every boot. i.e., something like:

variant: fcos
version: 1.1.0
systemd:
  units:
    - name: setsebool.service
      enabled: true
      contents: |
        [Service]
        Type=oneshot 
        ExecStart=setsebool container_manage_cgroup true
        RemainAfterExit=yes
        [Install]
        WantedBy=multi-user.target

However, we need to solve this problem more generically. Here is a smathering of issues where this issue is touched on:

• SELinux policy modifications override new policies during upgrades rpm-ostree#27
• Handle SELinux policy recompilation ostreedev/ostree#1026
• enable container_manage_cgroup SELinux boolean by default #397
• selinux: provisioning policy customizations #396
• modifying sebooleans via Ignition #368

A while back we had a conversation with the SELinux team about how to solve this problem in the future. I'm planning to take those notes and summarize the conversation here to try to push the conversation forward.

For now if you are trying to figure out if your system will no longer receive policy updates you can run sudo ostree admin config-diff | grep selinux/targeted/policy to see if there are diffs between the local persistent policy and what's delivered by the OSTree. For example:

$ sudo ostree admin config-diff | grep selinux/targeted/policy
M    selinux/targeted/policy/policy.32

In order to recover your system you can switch to dynamically modifying the policy (see the FCCT with the systemd unit above) and also run sudo rsync -rclv /usr/etc/selinux/ /etc/selinux/. You may want to run with --dry-run first.

[core@localhost ~]$ sudo rsync -rclv /usr/etc/selinux/ /etc/selinux/
sending incremental file list
targeted/active/booleans.local
targeted/active/commit_num
targeted/active/policy.kern
targeted/policy/policy.32

sent 16,525,161 bytes  received 534 bytes  11,017,130.00 bytes/sec
total size is 32,526,563  speedup is 1.97

After a reboot you should be back to normal. If you see any extra denials in logs you may want to run restorecon -vr /etc/ /var/ to fix the contexts on any files.

[jlebon]

If you need to install a custom SELinux policy, currently the safest way to do this is to package it in an RPM with a scriptlet to semodule -n -i xxx.pp and package layer it. rpm-ostree knows how to correctly handle these. (Edit: see e.g. the rpm-ostree testsuite: https://github.com/coreos/rpm-ostree/blob/da84ab872cec71c098cddbc3ffe30d4f8921e1ce/tests/common/libtest.sh#L274-L313.)

[YaLTeR]

What's the workaround for this?

sudo semanage port -a -t ssh_port_t -p tcp REDACTED

As in, how to do this non-persistently so I still get selinux policy updates?

[dustymabe]

hey @YaLTeR - this is still something that needs to be addressed by the team. See the discussion in #396 that specifically talks about modifying the SSH port like you are trying to do. There is a hacky workaround in there, but we need to do better.

[WOnder93]

Hi guys,

I'm a member of the Red Hat SELinux team and have been assigned to try to find a solution for this issue. I'm able to reproduce it on a FCOS VM, for example by installing a dummy SELinux module and doing an rpm-ostree rebase ... that updates the selinux-policy package. However, I'm afraid I don't understand the workings of rpm-ostree enough to find a path forward...

Basically, it should be enough to run semodule -B (or semodule -nB if the resulting policy should not be loaded to the kernel immediately) after any change to the SElinux store (/etc/selinux/{targeted,mls,minimum}/active/) by the package manager. This would normally be done by the selinux-policy scriptlets, but IIUC with rpm-ostree these are run only on some base tree without the user modules/config present.

I found some code in rpm-ostree that appears to do something like this, but it doesn't seem to help in this case.

I would welcome any pointers/explanations/documentation links that could help me understand the problem better. Also if you already have any ideas or outlines for a potential solution, please share it here and hopefully we can brainstorm something together. It is our priority to do as much as we can to help solve this, so I'm offering myself as a resource :)

[cgwalters]

I found some code in rpm-ostree that appears to do something like this, but it doesn't seem to help in this case.

That code only runs on the main "base image build" side, not the client side currently.

And the reason for that is what we really want here is conditional logic:

if have_local_policy_modifications:
  run("semodule -nB")

(When generating a base image/ostree-commit, we know we need to compile the policy so it's unconditional)

To say this another way, we don't want everyone without local policy customizations to pay the cost of recompiling policy on update - a core principle here we're aiming for with rpm-ostree is "image based updates" by default where client systems are just replicating an image and not running lots of arbitrary code as root on each machine.

This would normally be done by the selinux-policy scriptlets, but IIUC with rpm-ostree these are run only on some base tree without the user modules/config present.

And yes, this is the other problem. Right now we handle /etc at shutdown time, but that's generally not expected to take a long time or error out. Compiling the policy though is subject to both. We need to handle the case of:

• Admin invokes e.g. setsebool container_manage_cgroup true
• Admin invokes rpm-ostree upgrade (queues an update in the background, staged) but does not reboot
• Admin invokes e.g. setsebool httpd_use_nfs true
• Admin invokes systemctl reboot
• During shutdown, ostree-finalize-staged.service runs and merges current /etc (including selinux boolean changes) with new deployment's /etc

One approach may be to speculatively recompile the policy at rpm-ostree upgrade time using the current /etc, and in this case detect that policy was modified again, and recompile during ostree-finalize-staged.service time.

Or alternatively, detect this during the next boot and reconcile at that time.

[WOnder93]

I found some code in rpm-ostree that appears to do something like this, but it doesn't seem to help in this case.

That code only runs on the main "base image build" side, not the client side currently.

And the reason for that is what we really want here is conditional logic:

if have_local_policy_modifications:
  run("semodule -nB")

(When generating a base image/ostree-commit, we know we need to compile the policy so it's unconditional)

To say this another way, we don't want everyone without local policy customizations to pay the cost of recompiling policy on update - a core principle here we're aiming for with rpm-ostree is "image based updates" by default where client systems are just replicating an image and not running lots of arbitrary code as root on each machine.

Ok, thanks for the explanations, things are starting to click into place.

This would normally be done by the selinux-policy scriptlets, but IIUC with rpm-ostree these are run only on some base tree without the user modules/config present.

And yes, this is the other problem. Right now we handle /etc at shutdown time, but that's generally not expected to take a long time or error out. Compiling the policy though is subject to both. We need to handle the case of:

* Admin invokes e.g. `setsebool container_manage_cgroup true`

* Admin invokes `rpm-ostree upgrade` (queues an update in the background, staged) but _does not_ reboot

* Admin invokes e.g. `setsebool httpd_use_nfs true`

* Admin invokes `systemctl reboot`

* During shutdown, `ostree-finalize-staged.service` runs and merges current `/etc` (including selinux boolean changes) with new deployment's `/etc`

One approach may be to speculatively recompile the policy at rpm-ostree upgrade time using the current /etc, and in this case detect that policy was modified again, and recompile during ostree-finalize-staged.service time.

Oh, so I missed the fact that the policy customizations might be added/changed between staging the upgrade and rebooting... That makes things a bit more tricky...

Or alternatively, detect this during the next boot and reconcile at that time.

This looks like the most appealing option to me. We can't fully avoid delaying the reboot either way and I think delaying boot is a bit better than delaying shutdown. But here the question is when to rebuild the policy. Systemd loads the policy when switching root, so ideally we would rebuild policy before that (e.g. by chrooting into /sysroot and relabeling /etc/selinux), which is tricky as we'd need to add some script/unit to initrd (OTOH, ostree already seems to have some hook(s) there, so perhaps this is not a big deal). If we rebuild policy after switching root, then there will be a window of time when the system would run with the uncustomized policy and I'm not sure if we can easily prevent systemd from starting other services before the new policy is built and loaded...

I took a shot at the rebuild-before-switch-root approach using this very rough PoC:
https://gitlab.com/omos/coreos-selinux-tweak

It is of course nowhere near production ready and doesn't yet have any logic to detect when to rebuild, but it proves that it is feasible to do the rebuild before switching root. One catch though is that rpm-ostree doesn't update the initrd automatically unless the user runs rpm-ostree initramfs --enable. (But perhaps that wouldn't be needed if the logic would be part of some system package as opposed to a locally installed RPM?) It also currently hard-codes the SELINUXTYPE ("targeted") due to the need to use setfiles rather than restorecon, which refuses to do anything on a system without SELinux policy loaded... But I think the whole thing could be done from a dedicated executable using libselinux and libsemanage APIs, in which case there would be no need for hardcoding or chrooting (or so I hope).

Assuming this is an acceptable approach, there is then the question of where and how to best implement this. Maybe it could be just added to ostree-prepare-root? Or should it rather have a dedicated binary + systemd unit + dracut module to install these into the initrd? Would that ideally be a part of ostree, rpm-ostree, or as a subpackage of selinux-policy?

[travier]

I think the most robust approach is to make the RPM case work well and easy to do as explained in comment #701 (comment). This will make this change explicit in rpm-ostree (overlayed package) and keep the policy rebuild happening at an opportune time: as part of the local rpm-ostree compose during updates.

For that to work well, we need SELinux userspace tools to be able to easily produce those RPMs instead of directly doing policy changes, especially in a binary form as it is done right now by semanage. If we can have a declarative, plain text way of configuring booleans this also becomes simpler.

Removing all dependencies to Python for those tools would also be a big plus for us too.

Sorry if this is starting to look like a wish-list.

[travier]

Also related: https://github.com/overdrop/overdrop-sebool

[WOnder93]

I think the most robust approach is to make the RPM case work well and easy to do as explained in comment #701 (comment). This will make this change explicit in rpm-ostree (overlayed package) and keep the policy rebuild happening at an opportune time: as part of the local rpm-ostree compose during updates.

For that to work well, we need SELinux userspace tools to be able to easily produce those RPMs instead of directly doing policy changes, especially in a binary form as it is done right now by semanage.

I had thought of that option (provide tools that generate RPMs for customizations and install/remove/update those), but it seems somewhat overcomplicated to generate a spec file, run rpmbuild over it and then manage the resulting RPM set somehow. Is there a precedence of doing something like this for another system component? Another disadvantage would be that this wouldn't prevent people from trying to apply customizations in the usual way, not realizing that they will break their system in a subtle way by doing that.

Ensuring that whatever is in /etc on boot is reflected in the policy that gets loaded on boot feels to me like the most direct way to address the problem. With classic services, you expect that when they are started, they load the present configuration, refreshing any cached state if needed. This seems to be one of the principles that CoreOS/ostree relies on, so IMHO it makes sense to make SELinux policy management (which you can think of as a special kind of "service" started on boot) more in line with this principle.

If we can have a declarative, plain text way of configuring booleans this also becomes simpler.

Yes, as a follow up it would be great to refactor the SELinux tooling to allow having the policy and various bits configuration split between /usr and /etc, but that on its own doesn't solve the issue with SELinux policy package updates and thus I would consider it a separate RFE.

Removing all dependencies to Python for those tools would also be a big plus for us too.

Sorry if this is starting to look like a wish-list.

Let's focus on one thing at a time, please :) But feel free to reach out internally to our team's PO (Lukas Vrabec, @wrabcak) to ensure that these tasks are properly tracked on our side. (FWIW, we are already tracking a requirement to remove libsemanage -> policycoreutils executable calls, though it's currently parked.)

[jlebon]

Also not a fan of generating RPMs for this. Feels like if we have room for experimentation, we should aim for something much nicer. Though I also agree that ideally we'd do this some time before rebooting, and even more ideally at rpm-ostree upgrade time, because those are better understood failure points for users (e.g. it doesn't break or delay boot). That gets us back to the /etc merge issue, though the speculative recompile approach @cgwalters mentioned would work pretty well in practice IMO.

My strawman would be something like:

1. Enhance semodule -B (or more specifically, whatever underlying API which rebuilds the policy that also backs e.g. setsebool) to know to no-op if no changes are required (e.g. it could hash all the input files it took to generate the policy and store it alongside the binary).
2. During deployment creation, ostree always seeds the new /etc with the base policy. Then, at both rpm-ostree upgrade and ostree admin finalize-staged time, we run semodule -B unconditionally (which now knows to just no-op if inputs didn't change as above).
3. Teach rpm-ostree status to clearly display when a custom policy is present.
4. Optionally add an rpm-ostree reset --selinux which restores the base policy in a new deployment. (This would now also require restoring the input files too.)

So from the SELinux side, the only main work needed would be the no-op detection bit. WDYT?

[jlebon]

That said, overall I'm not opposed to the approach you've taken in your POC to regenerate on boot. As you've said, the conditional rebuild bit would also be needed there, so it sounds like it could be a stepping stone to something like #701 (comment) ?

[WOnder93]

@jlebon Sounds good to me. I personally dislike the idea of preparing for the next boot during shutdown (on a workstation an impatient user might force a power-off when the shutdown takes longer than a few seconds, not to mention losing changes in case of a power outage), but since it's already the status quo, I'm not going to argue against it :)

As you point out, the common first step is to detect and avoid the redundant policy rebuilding. I was hoping I could take a shortcut in the rebuild-needed detection by doing something like ostree admin config-diff to see if the policy has any customizations, but obviously that's not very robust and wouldn't skip the rebuild if there were no changes since the last one. Implementing a robust chnage detection in libsemanage would be more complex, but I agree it would be the right thing to do. I'll look into it as a next step.

I had an idea to implement the rebuild-on-boot functionality behind a new libsemanage function (the rebuild being opt-in via semanage.conf(5)) and to switch systemd to use this new function instead of directly calling selinux_init_load_policy(3). As that would be relatively simple to do and independent of ostree/CoreOS, I'll probably do some experiments with that, too.

[jlebon]

@jlebon Sounds good to me. I personally dislike the idea of preparing for the next boot during shutdown (on a workstation an impatient user might force a power-off when the shutdown takes longer than a few seconds, not to mention losing changes in case of a power outage), but since it's already the status quo, I'm not going to argue against it :)

To be clear, most of the time the work will happen at rpm-ostree upgrade time, which isn't during shutdown. The shutdown bit would just be for the corner-case of changes having occurred after rpm-ostree upgrade but before rebooting.

In general though, the OSTree model is all about preparing the next deployment in the background and atomically switching to it on reboot. So it's very resilient to power outages. :) If e.g. power goes out while the policy is being built for a new deployment, we'll just reboot into the same deployment we were in, without any loss (other than the CPU time we wasted in preparing the lost deployment).

As you point out, the common first step is to detect and avoid the redundant policy rebuilding. I was hoping I could take a shortcut in the rebuild-needed detection by doing something like ostree admin config-diff to see if the policy has any customizations, but obviously that's not very robust and wouldn't skip the rebuild if there were no changes since the last one. Implementing a robust chnage detection in libsemanage would be more complex, but I agree it would be the right thing to do. I'll look into it as a next step.

Cool, SGTM! And of course, this would also be useful for non-OSTree based systems too.

I had an idea to implement the rebuild-on-boot functionality behind a new libsemanage function (the rebuild being opt-in via semanage.conf(5)) and to switch systemd to use this new function instead of directly calling selinux_init_load_policy(3). As that would be relatively simple to do and independent of ostree/CoreOS, I'll probably do some experiments with that, too.

It's an interesting avenue, though my concern there before opting in would still be boot delays and potential failure. Also, AFAICT the store root on non-OSTree systems is still /var/lib/selinux which may lie on a different device not yet mounted at switchroot time.

[cgwalters]

Tangentially related, with https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md we will also want to run the policy build as a finalization step. Also relates to ostreedev/ostree-rs-ext#159

[WOnder93]

Update: I have submitted patches implementing the policy rebuild optimization upstream and I think they are nearing their final form w.r.t. API/CLI:
https://lore.kernel.org/selinux/20220203165327.213601-1-omosnace@redhat.com/T/

Basically, passing --rebuild-if-modules-changed to semodule instead of -B will rebuild the policy only if it has changed since the last transaction. The full rebuild takes about 7 seconds on Fedora x86_64, while when the module checksums match it takes only about a second (overhead from transaction management and reading+decompressing+hashing the modules still remains).

I have built libsemanage and policycoreutils with the proposed patches in COPR in case someone would like to try them out:
https://copr.fedorainfracloud.org/coprs/omos/selinux-testing/

I also briefly looked into what it would take to add the policy rebuild step into ostree admin finalize-staged (which seems to be key missing element). I can see basically two options:

1. Run semodule --rebuild-if-modules-changed -n in a chroot inside the new deployment somewhere in sysroot_finalize_deployment(). This would have to involve checking if the new flag is supported (possibly just by searching the output of semodule --help) or ignoring failures so that it still works with old userspace as well. There doesn't seem to be any code running stuff in a chroot in libostree, so this might be a bit complicated to do.
2. Link libostree against libsemanage and use its API directly to rebuild the policy (explicitly pointing to the policy store inside the deployment's root). This would be simpler, but could be problematic in case a newer libsemanage starts to use a new backwards-incompatible store format - then the store inside the new deployment may have been created by the new libsemanage and the runtime environment's libsemanage may not be able to work with it.

What do you think would be better? (Or can you think of a better option?) Also, would you like me to draft some patches to implement this, or would one of you rather like to do it themselves? (If it should be me, then please give me some hints on how it should be implemented :)

[jlebon]

Update: I have submitted patches implementing the policy rebuild optimization upstream and I think they are nearing their final form w.r.t. API/CLI: lore.kernel.org/selinux/20220203165327.213601-1-omosnace@redhat.com/T

Basically, passing --rebuild-if-modules-changed to semodule instead of -B will rebuild the policy only if it has changed since the last transaction. The full rebuild takes about 7 seconds on Fedora x86_64, while when the module checksums match it takes only about a second (overhead from transaction management and reading+decompressing+hashing the modules still remains).

Awesome! Thanks a lot for working on this.

I have built libsemanage and policycoreutils with the proposed patches in COPR in case someone would like to try them out: copr.fedorainfracloud.org/coprs/omos/selinux-testing

I also briefly looked into what it would take to add the policy rebuild step into ostree admin finalize-staged (which seems to be key missing element). I can see basically two options:

1. Run semodule --rebuild-if-modules-changed -n in a chroot inside the new deployment somewhere in sysroot_finalize_deployment(). This would have to involve checking if the new flag is supported (possibly just by searching the output of semodule --help) or ignoring failures so that it still works with old userspace as well. There doesn't seem to be any code running stuff in a chroot in libostree, so this might be a bit complicated to do.

2. Link libostree against libsemanage and use its API directly to rebuild the policy (explicitly pointing to the policy store inside the deployment's root). This would be simpler, but could be problematic in case a newer libsemanage starts to use a new backwards-incompatible store format - then the store inside the new deployment may have been created by the new libsemanage and the runtime environment's libsemanage may not be able to work with it.

What do you think would be better? (Or can you think of a better option?) Also, would you like me to draft some patches to implement this, or would one of you rather like to do it themselves? (If it should be me, then please give me some hints on how it should be implemented :)

The first option SGTM. I don't see why libostree couldn't learn to use bubblewrap to run things against the pending deployment. We could even then upstream the true sanity-check we do in rpm-ostree into libostree: https://github.com/coreos/rpm-ostree/blob/4a45703f26106e493990573251155cf42743d0f4/src/libpriv/rpmostree-scripts.cxx#L959 (edit: looks like we'd also be able to fix https://github.com/ostreedev/ostree/blob/490f515e189d1da4a0e04cc12b7a5e58e5a924dd/src/libostree/ostree-bootloader-grub2.c#L292).

(As you see there, we have some Rust code in rpm-ostree that wraps bwrap that we could export to its own crate, but we can't easily re-use that in libostree. I guess we could wrap it behind an ostree-rs-ext command, but it doesn't seem worth the complexity, and I think we'll want this for all libostree systems that use SELinux, which may include targets that are still not ready for Rust.)

[cgwalters]

I agree the right long term direction is to drive more of what's in rpm-ostree into ostree, including the containerization bits.

However...at a deeply practical level, the simplest way to go about this right now is probably
to add something like support for "hooks" into ostree-finalize-staged.service. In fact...rpm-ostree could likely do this today by installing a systemd drop-in for the service in /usr which does e.g. ExecStart=rpm-ostree internals finalize-staged which does the selinux policy heavy lifting, using all the bwrap code and in rust etc.

[cgwalters]

(Tangentially related, I've been thinking we should move towards depending on crun/runc and not bwrap)

[jlebon]

Chatted with @cgwalters OOB about this. We think that just g_spawn_sync'ing bwrap in libostree shouldn't be too bad and should be less complex than using hooks to redirect to rpm-ostree while benefiting all libostree users.

@WOnder93 If you're interested in working on this, that'd be great. Feel free to reach out on IRC or directly if you need any help!

[cgwalters]

We had a chat about this, I want to note this whole issue heavily intersects with https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md in that we will definitely want to support people changing SELinux booleans inside their container builds. It's an entirely different fix for this problem (and honestly, I think one that most non-single-system deployments will strongly prefer).

[WOnder93]

@jlebon Ack, I've already started working on it. Hoping to have a PR in a couple of days...

[jlebon]

A somewhat hacky approach (though commonly used pattern in the rpm-ostree testsuite) is to fake it out by layering an RPM which triggers a policy rebuild, and then rebase to that as if it were the new base. We can add as a test fixture an RPM built using https://github.com/coreos/rpm-ostree/blob/730bec87b129354ce2f43569fd5257bf249e63d1/tests/common/libtest.sh#L278. So roughly, it'd be something like:

1. rpm-ostree install fixture.rpm
2. ostree refs CHECKSUM --create base-with-modified-policy (where CHECKSUM is the generated OSTree commit)
3. rpm-ostree cleanup -p
4. Do something that triggers a local policy rebuild, e.g. setsebool container_manage_cgroup true
5. rpm-ostree rebase base-with-modified-policy
6. reboot and verify that both the modifications from the fixture RPM and the setsebool are present

[HuijingHei]

Downstream testing result refer to https://bugzilla.redhat.com/show_bug.cgi?id=2057497#c33
When rebase to 8.6 image, the policy is rebuilt with new policy only if running semodule manually after setsebool(before upgrading)

Is there any plan to do the fix?

If can download old image like 36.20220430.1.0 from https://getfedora.org/en/coreos?stream=next, then I shall do testing to check if can reproduce on fcos

[dustymabe]

A somewhat hacky approach (though commonly used pattern in the rpm-ostree testsuite) is to fake it out by layering an RPM which triggers a policy rebuild, and then rebase to that as if it were the new base.

If we wanted to simplify things we don't have to fully test it on every upgrade test. I think it would suffice to:

1. Do something that triggers a local policy rebuild, e.g. setsebool -P container_manage_cgroup true
2. rpm-ostree upgrade
3. reboot and verify that modifications from new SELinux policy (if there was a new one) exist and also the modifications from the setsebool are present

Asking more questions:

It used to be pretty simple to tell if I (the user) had made custom policy modifications with sudo ostree admin config-diff | grep selinux/targeted/policy. So I either had custom policy modifications (and my SELinux policy may be out of date) or I didn't (and I was in line with latest booted deployment). Now the waters are a bitty muddier.

How can I tell if I've made a local policy modification and it's been rebuilt with the latest policy delivered by the OSTree? sudo ostree admin config-diff | grep selinux/targeted/policy will still tell me if I have local policy modifications or not, but it won't tell me if the policy delivered with the OSTree is merged in or not.

[HuijingHei]

Downstream testing result refer to https://bugzilla.redhat.com/show_bug.cgi?id=2057497#c33 When rebase to 8.6 image, the policy is rebuilt with new policy only if running semodule manually after setsebool(before upgrading)

Is there any plan to do the fix?

Thanks @WOnder93 for the help to create https://bugzilla.redhat.com/show_bug.cgi?id=2089802 to track the problem

[dustymabe]

As identified by the team (see BZ in previous comment) right now there is a gap where sebooleans won't get detected as changed and carried forward. We'll hold marking this as fully fixed until BZ#2089802 gets fixed.

[dustymabe]

Upstream progress: https://lore.kernel.org/selinux/20220608170954.114668-1-omosnace@redhat.com/T/

[markusdd]

Hi, this issue basically blew up our OKD4 cluster upgrade because we have to apply 2 custom .cil selinux modules to our nodes because of some weird 3rd party tools we need to use in our CI containers.
This caused changes to the binary policy file, which in turn never was updated, finally rusulting in a failure because kubelet would not launch anymore.
So we restored policy.33 binary file from /usr/etc and this allowed the system to proceed, but our rules are gone from the policy (obviously).

This is what it looks like after a manual policy build and after restoriung the binary policy file.

Can you please advise how we can apply our 2 custom modules without touching the central policy file permanently?
Unfortunately there is not seboolean for that, the normal execheap one does not cut it.

Can modules be applied non-permanently after boot e.g. through a systemd one-shot file? I know this is possible for booleans when omitting the -P flag.

We urgently need those custom modules, but automatic OKD4 upgrades also must work.

Any help is appreciated.

Edit: Is it e.g. possible to compile our rules into a seperate policy binary which does not touch the delivered ostree one so we don't interfere with it?

[jlebon]

Once OKD rebases to an FCOS that includes the SELinux patches, you should be good to go. Until then, the most supported path would be to package the SELinux modules into an RPM and rpm-ostree install it.

[jlebon]

Thanks to @WOnder93's work, the SELinux policy is now recompiled when necessary. There's more work we could do here, like some of the ideas in #701 (comment), but the main issue is fixed so I'm going to close this. We can track follow-ups in separate tickets. Feel free to reopen if you disagree!

[YaLTeR]

So applying custom SELinux policies will no longer prevent further upstream policy updates from applying on Silverblue? Starting from which versions of the involved packages? Sorry, it's not quite clear to me from the discussion...

[markusdd]

Yeah to be sure if this can be closed I need to simple statements:

• Will custom 'setsebool' modifications be 'merged' into a global ostree policy update?
• Will custom modules applied via 'semodule -i' be taken over and compiled into an ostree policy update?

If those 2 are true, it's fine. If not, we need to reopen.

[jlebon]

So applying custom SELinux policies will no longer prevent further upstream policy updates from applying on Silverblue?

Correct.

Starting from which versions of the involved packages?

The ostree side is in v2022.3, which entered f36 6 months ago. The policycoreutils side was fixed in this update, which shipped 9 months ago.

• Will custom 'setsebool' modifications be 'merged' into a global ostree policy update?
• Will custom modules applied via 'semodule -i' be taken over and compiled into an ostree policy update?

Correct to both. Both actions will trigger policy rebuilds. From that point on, on every update the policy will be recompiled as needed to ensure base policy updates are baked in.

[jlebon]

Wait, I just realized we never added tests for this (conversation about that starts at #701 (comment)). Let's definitely make sure we do that before marking this as done.

[markusdd]

Thanks for the feedback, fantastic, taht is what was needed.

But I agree, tests should be added to ensure this does not break, as disruptions to this usually cause very weird and hard to find dailures on production systems.

[queeup]

Is there any doc or howto for custom selinux policies?

[markusdd]

you just do it based on the official docs of your distro or selinux themselves

it is no different than e.g. doing it on RHEL 8