Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: apparmor denies /var/lib/dpkg/arch #3137

Closed
panlinux opened this issue May 31, 2024 · 8 comments · Fixed by #3139
Closed

Bug: apparmor denies /var/lib/dpkg/arch #3137

panlinux opened this issue May 31, 2024 · 8 comments · Fixed by #3139
Labels
bug Issue and problem confirmed to fix

Comments

@panlinux
Copy link
Contributor

panlinux commented May 31, 2024 •
edited
Loading

Description of the bug

Seen in the logs from https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2067728:

May 31 18:00:17 loky kernel: audit: type=1400 audit(1717158617.647:261): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8510 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:17 loky kernel: audit: type=1400 audit(1717158617.678:262): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8511 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:17 loky kernel: audit: type=1400 audit(1717158617.680:263): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8512 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:17 loky kernel: audit: type=1400 audit(1717158617.683:264): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8513 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:18 loky kernel: audit: type=1400 audit(1717158618.556:265): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8712 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:18 loky kernel: audit: type=1400 audit(1717158618.601:266): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8714 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:18 loky kernel: audit: type=1400 audit(1717158618.603:267): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8715 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:18 loky kernel: audit: type=1400 audit(1717158618.607:268): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8720 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
May 31 18:00:18 loky kernel: audit: type=1400 audit(1717158618.622:269): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_esm_cache//dpkg" name="/var/lib/dpkg/arch" pid=8725 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

Unsure if that's the cause of the LP bug, though, as the ua logs also show errors contacting the contract server (timeouts).

The journal logs at those times are fine:

May 31 18:00:17.176632 loky systemd[1]: Starting apt-news.service - Update APT News...
May 31 18:00:17.184741 loky systemd[1]: Starting esm-cache.service - Update the local ESM caches...
May 31 18:00:17.370505 loky systemd[1]: apt-news.service: Deactivated successfully.
May 31 18:00:17.370797 loky systemd[1]: Finished apt-news.service - Update APT News.
May 31 18:00:18.673192 loky systemd[1]: esm-cache.service: Deactivated successfully.
May 31 18:00:18.673539 loky systemd[1]: Finished esm-cache.service - Update the local ESM caches.
May 31 18:00:18.673778 loky systemd[1]: esm-cache.service: Consumed 1.385s CPU time.
May 31 18:05:58.552545 loky systemd[1]: Starting ua-timer.service - Ubuntu Pro Timer for running repeated jobs...
May 31 18:05:58.739150 loky systemd[1]: ua-timer.service: Deactivated successfully.
May 31 18:05:58.739422 loky systemd[1]: Finished ua-timer.service - Ubuntu Pro Timer for running repeated jobs.

Incidentally, looks like apport is trying to report the version of the ubuntu-advantage-tools package, but it should also check ubuntu-pro-client.

Expected behavior

No apparmor denied errors.

Current behavior
For some unknown operation, the pro client on that system is triggering dpkg apparmor denied errors. We haven't seen those in our testing.

in uaclient/system.py, we have get_dpkg_arch() which calls dpkg --print-architecture, and that works just fine with the current apparmor profile:

# aa-exec -p ubuntu_pro_esm_cache//dpkg dpkg --print-architecture
amd64

And strace confirms that that command does not touch /var/lib/dpkg:

# strace -f dpkg --print-architecture 2>&1|grep open
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libmd.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/dpkg/dpkg.cfg.d", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/etc/dpkg/dpkg.cfg", O_RDONLY) = 3
openat(AT_FDCWD, "/root/.dpkg.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.utf8/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en/LC_MESSAGES/dpkg.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

To Reproduce
Unknown at the moment.

System information:

  • Ubuntu release: jammy
  • Pro Client version: 32.3

Additional context

Add any other context about the problem here.
@panlinux panlinux added the bug Issue and problem confirmed to fix label May 31, 2024
@panlinux
Copy link
Contributor Author

panlinux commented Jun 3, 2024

Ok, had an excellent troubleshoting session with @renanrodrigo and we found the origin of the problem.

Basically, just the presence of /var/lib/dpkg/arch is enough to trigger the attempted access and subsequent DENIED. Even if the file is empty. But the most common case seems to be systems that have a subarchitecture, like i386, added like dpkg --add-architecture i386. That will create /var/lib/dpkg/arch with amd64 and i386 in it.

We don't know yet the consequence of this DENIED error. It looks like it's triggered by apt-cache policy called by Pro, which ends up calling dpkg --print-foreign-architectures, which is what attempts to read /var/lib/dpkg/arch.

I diffed the output of apt-cache policy with and without the apparmor profile, and even though the run with the apparmor profile had the DENIED log entries, the actual output has no differences:

$ sudo aa-exec -p ubuntu_pro_esm_cache apt-cache policy > denied
$ sudo apt-cache policy > allowed
$ diff -u allowed denied
$

It's the same attached or unattached.

@panlinux
Copy link
Contributor Author

panlinux commented Jun 3, 2024 •
edited
Loading

Further troubleshooting shows that also the apt_methods_gpgv child profile is affected in this situation:

[Mon Jun  3 13:39:19 2024] audit: type=1400 audit(1717421960.564:105): apparmor="DENIED" operation="open" profile="ubuntu_pro_esm_cache//apt_methods_gpgv" name="/var/lib/dpkg/arch" pid=4879 comm="dpkg" requested_mask="r" denied_mask="r" fsuid=105 ouid=0

I'll go over the profiles and make sure /var/lib/dpkg is allowed whenever we are dealing with apt or dpkg.

panlinux added a commit that referenced this issue Jun 3, 2024
@panlinux
apparmor: esm_cache: allow /var/lib/dpkg/**
f559821 
This is needed in systems that have a /var/lib/dpkg/arch file, which can
be triggered by `dpkg --add-architecture i386` on amd64, for example.

It's best to allow reading of all files therein.

LP: #2067810
Fixes: #3137
@panlinux panlinux mentioned this issue Jun 3, 2024
Merged
5 tasks
panlinux added a commit that referenced this issue Jun 3, 2024
@panlinux
machines.py: trigger GH #3137
49be30b
panlinux added a commit that referenced this issue Jun 3, 2024
@panlinux
Revert "apparmor: esm_cache: allow /var/lib/dpkg/**"
fedf486 
Let's see if this fails the tests now that the trigger for GH: #3137 is
in place.

This reverts commit f559821.
panlinux added a commit that referenced this issue Jun 3, 2024
@panlinux
machines.py: trigger GH #3137
22d20cd
panlinux added a commit that referenced this issue Jun 7, 2024
@panlinux
apparmor: esm_cache: allow /var/lib/dpkg/**
95d1186 
This is needed in systems that have a /var/lib/dpkg/arch file, which can
be triggered by `dpkg --add-architecture i386` on amd64, for example.

It's best to allow reading of all files therein.

LP: #2067810
Fixes: #3137
panlinux added a commit that referenced this issue Jun 7, 2024
@panlinux
machines.py: trigger GH #3137
58d49ca
@renanrodrigo renanrodrigo mentioned this issue Jun 10, 2024
Closed
@dominicraf
Copy link

Confirm that I have this problem on 2 machines running bionic which have empty /var/lib/dpkg/arch

@renanrodrigo
Copy link
Member

@dominicraf ok that is interesting
The presence of the file was, for us, a guaranteed way to trigger the bug, but of course other situations may cause it.
The fix we applied (and are in process to release) will cover any case by fixing the apparmor profile, but I'm curious - what happens when you run sudo aa-exec -p ubuntu_pro_esm_cache apt-cache policy ?

@panlinux
Copy link
Contributor Author

Confirm that I have this problem on 2 machines running bionic which have empty /var/lib/dpkg/arch

That statement is a bit ambiguous. Do you have a /var/lib/dpkg/arch file that is empty, or do you not have that file?

@renanrodrigo
Copy link
Member

Aah yes good catch - the presence of the file itself is enough, content won't matter

@dominicraf
Copy link

Sorry, correction, and apologies for any confusion, my comment was rather careless. I am running jammy (22.04), not bionic, and the file /var/lib/dpkg/arch exists and has 2 lines as contents:

# cat /var/lib/dpkg/arch
amd64
i386

The following returns nothing (so it seems I actually have no i386 packages):
# dpkg -l | awk '/^ii/ && $4 == "i386" { print }'

And in answer to the q above:

# aa-exec -p ubuntu_pro_esm_cache apt-cache policy
Package files:
 100 /var/lib/dpkg/status
     release a=now
 510 https://esm.ubuntu.com/infra/ubuntu jammy-infra-updates/main amd64 Packages
     release v=22.04,o=UbuntuESM,a=jammy-infra-updates,n=jammy,l=UbuntuESM,c=main,b=amd64
     origin esm.ubuntu.com
 510 https://esm.ubuntu.com/infra/ubuntu jammy-infra-security/main amd64 Packages
     release v=22.04,o=UbuntuESM,a=jammy-infra-security,n=jammy,l=UbuntuESM,c=main,b=amd64
     origin esm.ubuntu.com
 500 https://packagecloud.io/ookla/speedtest-cli/ubuntu jammy/main amd64 Packages
     release v=1,o=packagecloud.io/ookla/speedtest-cli,a=jammy,n=jammy,l=speedtest-cli,c=main,b=amd64
     origin packagecloud.io
 500 https://ppa.launchpadcontent.net/maxmind/ppa/ubuntu jammy/main amd64 Packages
     release v=22.04,o=LP-PPA-maxmind,a=jammy,n=jammy,l=MaxMind Libraries and Software,c=main,b=amd64
     origin ppa.launchpadcontent.net
 500 http://ppa.launchpad.net/maxmind/ppa/ubuntu jammy/main amd64 Packages
     release v=22.04,o=LP-PPA-maxmind,a=jammy,n=jammy,l=MaxMind Libraries and Software,c=main,b=amd64
     origin ppa.launchpad.net
 500 https://ppa.launchpadcontent.net/adiscon/v8-stable/ubuntu jammy/main amd64 Packages
     release v=22.04,o=LP-PPA-adiscon-v8-stable,a=jammy,n=jammy,l=rsyslog v8-stable,c=main,b=amd64
     origin ppa.launchpadcontent.net
 500 http://ppa.launchpad.net/adiscon/v8-stable/ubuntu jammy/main amd64 Packages
     release v=22.04,o=LP-PPA-adiscon-v8-stable,a=jammy,n=jammy,l=rsyslog v8-stable,c=main,b=amd64
     origin ppa.launchpad.net
 500 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-security,n=jammy,l=Ubuntu,c=multiverse,b=amd64
     origin security.ubuntu.com
 500 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-security,n=jammy,l=Ubuntu,c=universe,b=amd64
     origin security.ubuntu.com
 500 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-security,n=jammy,l=Ubuntu,c=restricted,b=amd64
     origin security.ubuntu.com
 500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-security,n=jammy,l=Ubuntu,c=main,b=amd64
     origin security.ubuntu.com
 100 http://gb.archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-backports,n=jammy,l=Ubuntu,c=universe,b=amd64
     origin gb.archive.ubuntu.com
 100 http://gb.archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-backports,n=jammy,l=Ubuntu,c=main,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-updates,n=jammy,l=Ubuntu,c=multiverse,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-updates,n=jammy,l=Ubuntu,c=universe,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-updates,n=jammy,l=Ubuntu,c=restricted,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy-updates,n=jammy,l=Ubuntu,c=main,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy,n=jammy,l=Ubuntu,c=multiverse,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy,n=jammy,l=Ubuntu,c=universe,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy,n=jammy,l=Ubuntu,c=restricted,b=amd64
     origin gb.archive.ubuntu.com
 500 http://gb.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
     release v=22.04,o=Ubuntu,a=jammy,n=jammy,l=Ubuntu,c=main,b=amd64
     origin gb.archive.ubuntu.com
Pinned packages:
#

This was referenced Jun 13, 2024
Merged
Draft
@eugenesan
Copy link

Until the next release, the fix is offered at https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2067810

Just install ubuntu-pro-client (32.3.1~24.04) from Proposed.

@zenmonkeykstop zenmonkeykstop mentioned this issue Jul 1, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Issue and problem confirmed to fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Apparmor dpkg foreign arch canonical/ubuntu-pro-client
4 participants
@dominicraf @eugenesan @panlinux @renanrodrigo