[xenial] haveged process is unconfined under Xenial

[conorsch]

Description

Running under Xenial, the haveged process is reported as unconfined by aa-status. Should we confine it under Xenial?

Steps to Reproduce

1. make staging-xenial (run make build-debs-xenial first if you haven't recently)
2. Log into app-staging and run sudo aa-status

Expected Behavior

The haveged process is confined by AppArmor.

Actual Behavior

The haveged process is not confined by AppArmor. We have branch logic in the config tests to permit a single unconfined process under Xenial (as opposed to Trusty, which expects zero unconfined processes), to accommodate.

Comments

Do we want to confine haveged? If so, must we write a custom profile, or can we retrieve one via upstream packages present in the Xenial repos? Investigate the status under Trusty, where we do not currently maintain a profile specifically for haveged.

[redshiftzero]

Took a look at this since I had Xenial staging VMs up today. The haveged process has a profile already defined, but for some reason the process is running unconfined. Heading to the AppArmor FAQ, it says:

AppArmor can only track and protect processes that are started after the kernel module has been loaded. After the apparmor packages have been installed, apparmor will be started. But running processes won't be protected by AppArmor. Either restarting the processes or rebooting will fix this.

Following this, I restarted haveged in my Xenial VMs, and AppArmor now reports it's running the haveged profile in enforce mode:

root@app-staging:/home/vagrant# service haveged restart
root@app-staging:/home/vagrant# aa-status
apparmor module is loaded.
20 profiles are loaded.
20 profiles are in enforce mode.
[... snip... ]
   /usr/sbin/haveged
[... snip... ]
0 profiles are in complain mode.
11 processes have profiles defined.
11 processes are in enforce mode.
[... snip... ]
   /usr/sbin/haveged (2187)
[... snip... ]
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Great. But then something happens, and after a few minutes running aa-status again indicates that the profile is running unconfined as reported in the ticket. The same behavior occurs with rebooting.

Investigating further, in /var/log/kern.log we see a bunch of AppArmor denials like the following:

Feb  8 01:54:18 app-staging kernel: [10985.789464] audit: type=1400 audit(1549590858.732:22): apparmor="DENIED" operation="mknod" profile="/usr/sbin/haveged" name="/run/haveged.pid" pid=5569 comm="haveged" requested_mask="c" denied_mask="c" fsuid=0 ouid=0

This is upstream unresolved bug https://bugs.launchpad.net/ubuntu/+source/haveged/+bug/1708674

In that bug, they report that adding /run/haveged.pid rw, to /etc/apparmor.d/usr.sbin.haveged resolves the issue.

Trying this locally, one can verify that this does indeed resolve the issue by first adding that line to the AppArmor profile and then:

root@app-staging:/home/vagrant# service aa-status reload
root@app-staging:/home/vagrant# service apparmor reload
root@app-staging:/home/vagrant# aa-status
apparmor module is loaded.
20 profiles are loaded.
20 profiles are in enforce mode.
[... snip... ]
   /usr/sbin/haveged
[... snip... ]
0 profiles are in complain mode.
11 processes have profiles defined.
11 processes are in enforce mode.
[... snip... ]
   /usr/sbin/haveged (2710)
[... snip... ]
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

However, there does appear to be an issue where upon reboot, haveged will still be running unconfined unless the process is restarted. This could be due to a race condition where haveged starts before AppArmor does...

[redshiftzero]

If I directly edit the init script in /etc/init.d/apparmor to make sure haveged starts after apparmor, then everything works and haveged runs confined upon reboots using the modified AppArmor profile as described above. Indeed, it looks like this is the same or similar bug that was filed and fixed over in Debian, in a haveged version that does not (yet) exist in Ubuntu land.

To recap, we've found that the cause of the failure were the following two problems:

1. Read/write on pid file needed whitelisting (described in my previous comment on this ticket)
2. Race condition due to haveged getting started prior to AppArmor (as described in my paragraph above)

Our choices to resolve are:

1. Fix ourselves
2. Wait for upstream to fix and prod them
3. Leave process unconfined

I propose we do 1 and 2. That is, we temporarily fix ourselves so that haveged is running confined in 0.12.0 on xenial, and then prod upstream ubuntu to resolve these two bugs. What do you think @kushaldas @emkll?

If you're thinking "but what if upstream pushes a new package in the meantime?", well, it turns out there hasn't been a new haveged release since 2015 and the only bug filed in the couple of years in launchpad for haveged is this one. This is the lowest risk option that is available to us on the 0.12.0 timescale (indeed, there's a very good chance that when a new release occurs it will fix the AppArmor profile).

[kushaldas]

I propose we do 1 and 2. That is, we temporarily fix ourselves so that haveged is running confined in 0.12.0 on xenial, and then prod upstream ubuntu to resolve these two bugs. What do you think @kushaldas @emkll?

I am 👍 to both of those points.

As a small note: I could not reproduce this error :( Once again the operating system is playing with us.

[emkll]

I can reproduce this error:

vagrant@app-staging:~$ sudo aa-status
apparmor module is loaded.
20 profiles are loaded.
20 profiles are in enforce mode.
[... snip ...]
1 processes are unconfined but have a profile defined.
   /usr/sbin/haveged (493) 

I agree, let's go for 1 and 2, I've tested the fix from Debian upstream linked above [0], and it works reliably based on local testing. Based on my understanding, prod upstream changes will, at worst, break AppArmor profile for haveged, and we can add an infra test to guard against this regression.

How should we deliver this change? unattended (postinst) or attended (Ansible)? Given that it's and the requirement to run ./securedrop-admin install to apply changes in #4114, and that it's a Xenial-only change, I propose we make these changes in the Ansible install logic.

[0] : https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=824179;filename=haveged-824179.patch;msg=10