Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building Ubuntu container fails with dpkg errors #6976

Closed
trgeiger opened this issue Jul 14, 2020 · 17 comments
Closed

Building Ubuntu container fails with dpkg errors #6976

trgeiger opened this issue Jul 14, 2020 · 17 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. stale-issue

Comments

@trgeiger
Copy link

trgeiger commented Jul 14, 2020

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
Recently my existing container setup has stopped building when using rootless podman. It's a simple Ubuntu container with a Ruby on Rails application.

One of the first steps in the build is STEP 4: RUN apt-get update && apt-get -y install build-essential. This fails with a couple of error lines: dpkg: error processing archive /var/cache/apt/archives/libtimedate-perl_2.3000-2_all.deb (--unpack): cannot get security labeling handle: No such file or directory

and

Errors were encountered while processing:
 /var/cache/apt/archives/libtimedate-perl_2.3000-2_all.deb
 /var/cache/apt/archives/dpkg-dev_1.17.27_all.deb

I haven't changed any configuration files or changed my Dockerfiles for this build, which used to work without root. It does build successfully when using sudo.

Steps to reproduce the issue:

  1. Build an Ubuntu container with ruby:2.2.4

  2. During the build process, install build-essential

Describe the results you received:
The error messages above, and a failed build.

Describe the results you expected:
Successful build of the image and container.

Additional information you deem important (e.g. issue happens only occasionally):
I reached out on the Kubernetes Slack crio channel and was asked to include the AVCs that came up in my journal:

Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name="hostname" dev="tmpfs" ino=1415264 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c684,c852 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { read } for  pid=352245 comm="systemd-user-ru" name="secrets" dev="tmpfs" ino=1407330 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c817,c908 tclass=dir permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name=".containerenv" dev="tmpfs" ino=1407328 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c817,c908 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name="hostname" dev="tmpfs" ino=1407327 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c817,c908 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name="hosts" dev="tmpfs" ino=1407326 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name="resolv.conf" dev="tmpfs" ino=1407325 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { read } for  pid=352245 comm="systemd-user-ru" name="secrets" dev="tmpfs" ino=165481 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c812,c1019 tclass=dir permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name=".containerenv" dev="tmpfs" ino=165479 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c812,c1019 tclass=file permissive=0
Jul 14 15:16:22 ada audit[352245]: AVC avc:  denied  { unlink } for  pid=352245 comm="systemd-user-ru" name="hostname" dev="tmpfs" ino=165478 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:object_r:container_file_t:s0:c812,c1019 tclass=file permissive=0
Jul 14 15:19:01 ada audit[355336]: AVC avc:  denied  { write } for  pid=355336 comm="dpkg" name="create" dev="selinuxfs" ino=7 scontext=system_u:system_r:container_t:s0:c54,c71 tcontext=system_u:object_r:security_t:s0 tclass=file permissive=0

Output of podman version:

Version:      2.0.2
API Version:  1
Go Version:   go1.14.3
Built:        Wed Dec 31 18:00:00 1969
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.15.0
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.18-1.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.18, commit: 6e8799f576f11f902cd8a8d8b45b2b2caf636a85'
  cpus: 12
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: file
  hostname: ada
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.7.8-200.fc32.x86_64
  linkmode: dynamic
  memFree: 8402546688
  memTotal: 33268961280
  ociRuntime:
    name: crun
    package: crun-0.14.1-1.fc32.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.14.1
      commit: 598ea5e192ca12d4f6378217d3ab1415efeddefa
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.1-1.fc32.x86_64
    version: |-
      slirp4netns version 1.1.1
      commit: bbf27c5acd4356edb97fa639b4e15e0cd56a39d5
      libslirp: 4.2.0
      SLIRP_CONFIG_VERSION_MAX: 2
  swapFree: 16697536512
  swapTotal: 16701714432
  uptime: 29h 30m 13.76s (Approximately 1.21 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/tayler/.config/containers/storage.conf
  containerStore:
    number: 7
    paused: 0
    running: 0
    stopped: 7
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.1.2-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /home/tayler/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 43
  runRoot: /run/user/1000/containers
  volumePath: /home/tayler/.local/share/containers/storage/volumes
version:
  APIVersion: 1
  Built: 0
  BuiltTime: Wed Dec 31 18:00:00 1969
  GitCommit: ""
  GoVersion: go1.14.3
  OsArch: linux/amd64
  Version: 2.0.2

Package info (e.g. output of rpm -q podman or apt list podman):

podman-2.0.2-1.fc32.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):
Fedora 32

@openshift-ci-robot openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jul 14, 2020
@vrothberg
Copy link
Member

Thanks for reaching out! Can you share the Dockerfile as a reproducer?

@rhatdan PTAL

@rhatdan
Copy link
Member

rhatdan commented Jul 15, 2020

This looks like selinux is enabled within the container?
Could you execute

$ podman run fedora id -Z

@Jakuje
Copy link

Jakuje commented Jul 15, 2020

I am having the same error in Fedora 31 while trying to build oss-fuzz images based on Ubuntu.

The command above returns that it is

id: --context (-Z) works only on an SELinux-enabled kernel

@trgeiger
Copy link
Author

The above command returns the same error as @Jakuje . I'll attach a version of the dockerfile here where I've just removed where our source code is attached, all the apt-get commands are the same (also added .txt file extension because Github yelled at me):

Dockerfile.txt

@vrothberg
Copy link
Member

Thanks, @trgeiger! I can reproduce on F32 with with Podman v2.0.2 and the Dockerfile:

FROM ruby:2.2.4

ENV PDFTK_VERSION 2.02

RUN sed -i '/jessie-updates/d' /etc/apt/sources.list  # Now archived

RUN apt-get update && apt-get -y install build-essential

# System prerequisites
RUN apt-get update && apt-get -y install libpq-dev

# If you require additional OS dependencies, install them here:
# Added libxml2-dev to avoid errors compiling libxml-ruby-3.0.0(libxml/xmlversion.h)
RUN apt-get update && apt-get -y install imagemagick libmysqlclient-dev libxml2-dev
RUN DEBIAN_FRONTEND=noninteractive \
    apt-get -y install default-jre-headless && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install zip -y

ENV PORT 3002
EXPOSE 3002
EXPOSE 8982

@rhatdan there must be something going on with SELinux in Podman. Using buildah bud works.

@rhatdan
Copy link
Member

rhatdan commented Jul 16, 2020

Did you get AVC messages?

ausearch -m avc -ts recent

@vrothberg
Copy link
Member

Did you get AVC messages?

ausearch -m avc -ts recent

@rhatdan, you should be able to reproduce on Fedora using the upper Dockerfile.

@Zarquan
Copy link

Zarquan commented Jul 17, 2020

I have encountered the same, building an Ubuntu container on a Fedora 31 host.

sudo ausearch -m avc -ts recent

type=AVC msg=audit(1594984366.515:5972): avc:  denied  { setfscreate } for  pid=1515242 comm="cp" scontext=system_u:system_r:container_t:s0:c456,c623 tcontext=system_u:system_r:container_t:s0:c456,c623 tclass=process permissive=0

@Zarquan
Copy link

Zarquan commented Jul 17, 2020

Can also confirm that passing exactly the same Dockerfile and command line options to buildah bud works with no problems.

@rhatdan
Copy link
Member

rhatdan commented Jul 20, 2020

I added the setfscreate access to container-selinux-2.139.0

@vrothberg
Copy link
Member

I added the setfscreate access to container-selinux-2.139.0

@rhatdan, so it's a policy issue? Can we close this issue?

@rhatdan
Copy link
Member

rhatdan commented Jul 20, 2020

Sure not sure why this happens in one way and not the other, but the AVC is not a problem to be allowed.

@rhatdan rhatdan closed this as completed Jul 20, 2020
@Jakuje
Copy link

Jakuje commented Jul 30, 2020

With current container-selinux-2:2.142.0-1.fc31.noarch (from updates testing) I see the follow-up issue:

Jul 30 16:51:52 t470s.jjelen.redhat.com audit[671755]: AVC avc: denied { write } for pid=671755 comm="dpkg" name="create" dev="selinuxfs" ino=7 scontext=system_u:system_r:container_t:s0:c119,c503 tcontext=system_u:object_r:security_t:>

Even after complete cleanup, I an unable to run the container:

podman system prune -a

The complete error from podman:

debconf: delaying package configuration, since apt-utils is not installed
dpkg: error processing archive /var/cache/apt/archives/libsigsegv2_2.10-4_amd64.deb (--unpack):
 cannot get security labeling handle: No such file or directory
Selecting previously unselected package libpython2.7-minimal:amd64.
(Reading database ... 17484 files and directories currently installed.)
Preparing to unpack .../libpython2.7-minimal_2.7.12-1ubuntu0~16.04.12_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.12) ...
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../python2.7-minimal_2.7.12-1ubuntu0~16.04.12_amd64.deb ...
dpkg (subprocess): cannot set security execution context for maintainer script: Permission denied
dpkg: error processing archive /var/cache/apt/archives/python2.7-minimal_2.7.12-1ubuntu0~16.04.12_amd64.deb (--unpack):
 subprocess new pre-installation script returned error exit status 2
dpkg (subprocess): cannot set security execution context for maintainer script: Permission denied
dpkg: error while cleaning up:
 subprocess new post-removal script returned error exit status 2

Please, reopen.

@rhatdan rhatdan reopened this Jul 30, 2020
@rhatdan
Copy link
Member

rhatdan commented Jul 30, 2020

This is fixed in Buildah, and a PR is being prepared to vendor into Podman to fix this everywhere.

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@Jakuje
Copy link

Jakuje commented Aug 31, 2020

Seems like it is finally fixed in my Fedora 32 and I am able to build oss-fuzz containers. Please, confirm. In that case, we can probably close this issue.

@mheon
Copy link
Member

mheon commented Aug 31, 2020

I'm going to go ahead and close, given that the Buildah fix appears to have landed in several releases.

@mheon mheon closed this as completed Aug 31, 2020
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. stale-issue
Projects
None yet
Development

No branches or pull requests

7 participants