Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot run example httpd under systemd in a container as a normal user. #1836

Closed
timcoote opened this issue Nov 20, 2018 · 17 comments
Closed

cannot run example httpd under systemd in a container as a normal user. #1836

timcoote opened this issue Nov 20, 2018 · 17 comments
Labels

Comments

@timcoote
Copy link

@timcoote timcoote commented Nov 20, 2018

kind bug

Description

I am following this example use of running systemd services in docker containers as a normal user: https://red.ht/2RkJyti, on various flavours of f29, including virtualbox/vagrant (fedora/29-cloud-base), and native install.

The examples works for an normal user with docker (if I fixup the file permissions for /var/run/docker*: sudo chmod root:wheel /var/run/docker* and add the user to the group wheel).

However, for podman, it does not. It does work as expected with sudo podman run -it httpd, but fails without sudo.

Steps to reproduce the issue:

  1. In a clean directory, create the Dockerfile:
FROM         fedora:24
ENV container docker
RUN dnf -y install httpd; dnf clean all; systemctl enable httpd
STOPSIGNAL SIGRTMIN+3
EXPOSE 80:8088
CMD [ "/sbin/init" ]

buildah bud -t httpd .
sudo buildah bud -t httpd-su .

podman run -it httpd

  1. In a second terminal:
    podman ps
    podman stop <container hash from above command>

then, re-run 3 and 4 using sudo podman run -it httpd-su to confirm correct behaviour.

Describe the results you received:

podman run -it httpd-pod
error reading container (probably exited) json message: EOF

Describe the results you expected:

sudo podman run -it httpd-su
systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.
Running with unpopulated /etc.

Welcome to Fedora 24 (Twenty Four)!

Set hostname to <6a8fb2cbd1c1>.
Initializing machine ID from random generator.
Failed to populate /etc with preset unit settings, ignoring: No such file or directory
Failed to install release agent, ignoring: No such file or directory
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Listening on Journal Socket.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Local File Systems.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Reached target Swap.
[  OK  ] Created slice System Slice.
[  OK  ] Reached target Slices.
         Starting Rebuild Journal Catalog...
         Starting Load/Save Random Seed...
         Starting Journal Service...
         Starting Rebuild Dynamic Linker Cache...
         Starting First Boot Wizard...
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Rebuild Journal Catalog.
[  OK  ] Started First Boot Wizard.
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
         Starting Create System Users...
[  OK  ] Started Rebuild Dynamic Linker Cache.
[  OK  ] Started Create System Users.
         Starting Update is Completed...
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Update is Completed.
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Reached target System Initialization.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started dnf makecache timer.
[  OK  ] Reached target Basic System.
[  OK  ] Started D-Bus System Message Bus.
         Starting The Apache HTTP Server...
         Starting Permit User Sessions...
[  OK  ] Reached target Timers.
[  OK  ] Started Permit User Sessions.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 0.10.1.3

Output of podman info:

podman info
host:
  BuildahVersion: 1.5-dev
  Conmon:
    package: podman-0.10.1.3-4.gitdb08685.fc29.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 1.12.0-dev, commit: 1f741a8b8381375b068b147605704a02a91167a2-dirty'
  Distribution:
    distribution: fedora
    version: "29"
  MemFree: 104271872
  MemTotal: 2089209856
  OCIRuntime:
    package: runc-1.0.0-57.dev.git9e5aa74.fc29.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc5+dev
      commit: ff195010cbfd3c62a98a3fd2f7a1e1594afdda1a
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 1
  hostname: localhost.localdomain
  kernel: 4.18.16-300.fc29.x86_64
  os: linux
  uptime: 39m 4.89s
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ContainerStore:
    number: 3
  GraphDriverName: vfs
  GraphOptions: []
  GraphRoot: /home/vagrant/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 4
  RunRoot: /run/user/1000/run

Additional environment details (AWS, VirtualBox, physical, etc.):
The results are similar on vagrant/virtualbox with the fedora image mentioned above and f29 on a physical computer.

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 20, 2018

Can you run the rootless podman run again with --log-level=debug? When you do, also check journalctl logs for anything from conmon - error messages may be printed to syslog

@timcoote
Copy link
Author

@timcoote timcoote commented Nov 20, 2018

I suspect that it's pretty obvious from the debug output. I'll add the journalctl later.

[vagrant@localhost vagrant]$ podman --log-level=debug run -it  httpd-non-su
INFO[0000] running as rootless                          
DEBU[0000] Not configuring container store              
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist 
DEBU[0000] Initializing boltdb state at /home/vagrant/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Set libpod namespace to ""                   
WARN[0000] AppArmor security is not available in rootless mode 
DEBU[0000] Using bridge netmode                         
INFO[0000] running as rootless                          
DEBU[0000] [graphdriver] trying provided driver "vfs"   
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist 
DEBU[0000] Initializing boltdb state at /home/vagrant/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]docker.io/library/httpd-non-su:latest" 
DEBU[0000] reference "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]docker.io/library/httpd-non-su:latest" does not resolve to an image ID 
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]docker.io/library/httpd-non-su:latest" 
DEBU[0000] reference "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]docker.io/library/httpd-non-su:latest" does not resolve to an image ID 
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]localhost/httpd-non-su:latest" 
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] exporting opaque data as blob "sha256:1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] exporting opaque data as blob "sha256:1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
WARN[0000] AppArmor security is not available in rootless mode 
DEBU[0000] Using bridge netmode                         
DEBU[0000] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] exporting opaque data as blob "sha256:1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0000] Creating dest directory: /home/vagrant/.local/share/containers/storage/vfs/dir/542800f69f8d4eab262c667ea1cee8aee79c656250b73de4aa977bb4fa305261 
DEBU[0000] Calling TarUntar(/home/vagrant/.local/share/containers/storage/vfs/dir/6d82dd6f0bf1d80056c07d343e4c378ca013ac5ea5de9a5fda7e40ea05e65247, /home/vagrant/.local/share/containers/storage/vfs/dir/542800f69f8d4eab262c667ea1cee8aee79c656250b73de4aa977bb4fa305261) 
DEBU[0000] TarUntar(/home/vagrant/.local/share/containers/storage/vfs/dir/6d82dd6f0bf1d80056c07d343e4c378ca013ac5ea5de9a5fda7e40ea05e65247 /home/vagrant/.local/share/containers/storage/vfs/dir/542800f69f8d4eab262c667ea1cee8aee79c656250b73de4aa977bb4fa305261) 
DEBU[0001] created container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" 
DEBU[0001] container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" has work directory "/home/vagrant/.local/share/containers/storage/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata" 
DEBU[0001] container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" has run directory "/run/user/1000/run/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata" 
DEBU[0001] New container created "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" 
DEBU[0001] container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" has CgroupParent "/libpod_parent/libpod-a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" 
DEBU[0001] Handling terminal attach                     
DEBU[0001] mounted container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" at "/home/vagrant/.local/share/containers/storage/vfs/dir/542800f69f8d4eab262c667ea1cee8aee79c656250b73de4aa977bb4fa305261" 
DEBU[0001] Created root filesystem for container a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84 at /home/vagrant/.local/share/containers/storage/vfs/dir/542800f69f8d4eab262c667ea1cee8aee79c656250b73de4aa977bb4fa305261 
WARN[0001] error mounting secrets, skipping: getting host secret data failed: failed to read secrets from "/usr/share/rhel/secrets": open /usr/share/rhel/secrets: permission denied 
DEBU[0001] /etc/system-fips does not exist on host, not mounting FIPS mode secret 
DEBU[0001] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] exporting opaque data as blob "sha256:1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] exporting opaque data as blob "sha256:1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] parsed reference into "[vfs@/home/vagrant/.local/share/containers/storage+/run/user/1000/run]@1db0ebd250d5a351e4f990a071cdec69014b41941ece7c3f8a1ac8ccea70ff48" 
DEBU[0001] Created OCI spec for container a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84 at /home/vagrant/.local/share/containers/storage/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata/config.json 
DEBU[0001] /usr/libexec/podman/conmon messages will be logged to syslog 
DEBU[0001] running conmon: /usr/libexec/podman/conmon    args=[-c a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84 -u a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84 -r /usr/bin/runc -b /home/vagrant/.local/share/containers/storage/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata -p /run/user/1000/run/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata/pidfile -l /home/vagrant/.local/share/containers/storage/vfs-containers/a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84/userdata/ctr.log --exit-dir /run/user/1000/libpod/tmp/exits --socket-dir-path /run/user/1000/libpod/tmp/socket -t --log-level debug --syslog]
WARN[0001] Failed to add conmon to cgroupfs sandbox cgroup: mkdir /sys/fs/cgroup/systemd/libpod_parent: permission denied 
DEBU[0001] Cleaning up container a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84 
DEBU[0001] Network is already cleaned up, skipping...   
DEBU[0001] unmounted container "a73911475033dfca5843567d57fae81026d20daac1ac93d529fdbf43ac5d5e84" 
ERRO[0001] error reading container (probably exited) json message: EOF 

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 20, 2018

Nothing really obvious here - the EOF json message is conmon failing from a runc error without forwarding it (we have a patch to fix this and report better errors, but it hasn't landed yet). I'll try and reproduce locally after lunch, see if I can get a full error out of runc.

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 20, 2018

Question - why are you using fedora:24 in your Dockerfile? That's a very old systemd - I think we might need a newer version to successfully work in rootless Podman?

@timcoote
Copy link
Author

@timcoote timcoote commented Nov 20, 2018

fedora 24 is the example from Dan Walsh's blog (see link at top). I've tried all releases between 24 and 29. 24 is the only one that gives any console output, at least on x86_64. When I've cracked this, I've still got to demonstrate something working on aarch64, with fedora-iot (f29). I have confirmed that f24 does not work in a container on f29-iot/aarch64. I'm trying to start from a known-working baseline :-)

nb I started with docker, but want podman for f-iot.

Here's the journalctl output:

Nov 20 20:35:43 localhost.localdomain audit[12702]: USER_END pid=12702 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pa>
Nov 20 20:35:43 localhost.localdomain audit[12702]: CRED_DISP pid=12702 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" ho>
Nov 20 20:35:55 localhost.localdomain audit[12707]: USER_ACCT pid=12707 uid=1000 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="vagrant" exe="/usr/bin/sudo" h>
Nov 20 20:35:55 localhost.localdomain sudo[12707]:  vagrant : TTY=pts/1 ; PWD=/vagrant ; USER=root ; COMMAND=/usr/bin/journalctl
Nov 20 20:35:55 localhost.localdomain audit[12707]: USER_CMD pid=12707 uid=1000 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/vagrant" cmd="journalctl" terminal=pts/1 res=success'
Nov 20 20:35:55 localhost.localdomain audit[12707]: CRED_REFR pid=12707 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" ho>
Nov 20 20:35:55 localhost.localdomain sudo[12707]: pam_systemd(sudo:session): Cannot create session: Already running in a session or user slice
Nov 20 20:35:55 localhost.localdomain sudo[12707]: pam_unix(sudo:session): session opened for user root by vagrant(uid=0)
Nov 20 20:35:55 localhost.localdomain audit[12707]: USER_START pid=12707 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,p>
Nov 20 20:35:58 localhost.localdomain sudo[12707]: pam_unix(sudo:session): session closed for user root
Nov 20 20:35:58 localhost.localdomain audit[12707]: USER_END pid=12707 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pa>
Nov 20 20:35:58 localhost.localdomain audit[12707]: CRED_DISP pid=12707 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" ho>
Nov 20 20:36:06 localhost.localdomain conmon[12743]: conmon b36b15f3bf41c1e655b2 <ninfo>: addr{sun_family=AF_UNIX, sun_path=/tmp/conmon-term.VQY5SZ}
Nov 20 20:36:06 localhost.localdomain conmon[12743]: conmon b36b15f3bf41c1e655b2 <ninfo>: about to accept from console_socket_fd: 13
Nov 20 20:36:06 localhost.localdomain conmon[12743]: conmon b36b15f3bf41c1e655b2 <ninfo>: about to recvfd from connfd: 19
Nov 20 20:36:06 localhost.localdomain kernel: SELinux: mount invalid.  Same superblock, different security settings for (dev mqueue, type mqueue)
Nov 20 20:36:06 localhost.localdomain conmon[12743]: conmon b36b15f3bf41c1e655b2 <ninfo>: console = {.name = '(null)'; .fd = 0}
Nov 20 20:36:06 localhost.localdomain conmon[12743]: conmon b36b15f3bf41c1e655b2 <error>: Failed to get console terminal settings Inappropriate ioctl for device
Nov 20 20:36:09 localhost.localdomain audit[12764]: USER_ACCT pid=12764 uid=1000 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="vagrant" exe="/usr/bin/sudo" h>
Nov 20 20:36:09 localhost.localdomain sudo[12764]:  vagrant : TTY=pts/1 ; PWD=/vagrant ; USER=root ; COMMAND=/usr/bin/journalctl
Nov 20 20:36:09 localhost.localdomain audit[12764]: USER_CMD pid=12764 uid=1000 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/vagrant" cmd="journalctl" terminal=pts/1 res=success'
Nov 20 20:36:09 localhost.localdomain audit[12764]: CRED_REFR pid=12764 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" ho>
Nov 20 20:36:09 localhost.localdomain sudo[12764]: pam_systemd(sudo:session): Cannot create session: Already running in a session or user slice
Nov 20 20:36:09 localhost.localdomain sudo[12764]: pam_unix(sudo:session): session opened for user root by vagrant(uid=0)
Nov 20 20:36:09 localhost.localdomain audit[12764]: USER_START pid=12764 uid=0 auid=1000 ses=9 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,p>
Waiting for data... (interrupt to abort)


@mheon
Copy link
Collaborator

@mheon mheon commented Nov 20, 2018

@giuseppe Do we need an extra patch on top of systemd to get this working? I think I recall that

@rhatdan
Copy link
Member

@rhatdan rhatdan commented Nov 20, 2018

@timcoote Use fedora 29. That is a very old blog.

 cat ~/Dockerfile.systemd
FROM fedora
RUN dnf -y install httpd; dnf -y clean all; systemctl enable httpd
STOPSIGNAL  SIGRTMIN+3
CMD ["/usr/sbin/init"]
EXPOSE 80:8088
LABEL foo=bar
# podman build -f ~/Dockerfile.systemd -t systemd ~
STEP 1: FROM fedora
STEP 2: RUN dnf -y install httpd; dnf -y clean all; systemctl enable httpd
Fedora Modular 29 - x86_64                      220 kB/s | 1.5 MB     00:06    
Fedora Modular 29 - x86_64 - Updates            404 kB/s | 1.6 MB     00:04    
Fedora 29 - x86_64 - Updates                    2.9 MB/s |  13 MB     00:04    
Fedora 29 - x86_64                              2.0 MB/s |  62 MB     00:31    
Dependencies resolved.
================================================================================
 Package                  Arch         Version              Repository     Size
================================================================================
Installing:
 httpd                    x86_64       2.4.37-3.fc29        updates       1.3 M
Installing dependencies:
 httpd-filesystem         noarch       2.4.37-3.fc29        updates        15 k
 httpd-tools              x86_64       2.4.37-3.fc29        updates        76 k
 mod_http2                x86_64       1.11.1-1.fc29        updates       147 k
 apr                      x86_64       1.6.5-1.fc29         fedora        106 k
 apr-util                 x86_64       1.6.1-8.fc29         fedora         90 k
 fedora-logos-httpd       noarch       28.0.3-2.fc29        fedora         13 k
 mailcap                  noarch       2.1.48-4.fc29        fedora         31 k
Installing weak dependencies:
 apr-util-bdb             x86_64       1.6.1-8.fc29         fedora         12 k
 apr-util-openssl         x86_64       1.6.1-8.fc29         fedora         14 k

Transaction Summary
================================================================================
Install  10 Packages

Total download size: 1.8 M
Installed size: 5.8 M
Downloading Packages:
(1/10): httpd-filesystem-2.4.37-3.fc29.noarch.r  12 kB/s |  15 kB     00:01    
(2/10): httpd-tools-2.4.37-3.fc29.x86_64.rpm     56 kB/s |  76 kB     00:01    
(3/10): mod_http2-1.11.1-1.fc29.x86_64.rpm      549 kB/s | 147 kB     00:00    
(4/10): apr-1.6.5-1.fc29.x86_64.rpm             384 kB/s | 106 kB     00:00    
(5/10): apr-util-bdb-1.6.1-8.fc29.x86_64.rpm    207 kB/s |  12 kB     00:00    
(6/10): apr-util-1.6.1-8.fc29.x86_64.rpm        446 kB/s |  90 kB     00:00    
(7/10): apr-util-openssl-1.6.1-8.fc29.x86_64.rp 311 kB/s |  14 kB     00:00    
(8/10): fedora-logos-httpd-28.0.3-2.fc29.noarch 145 kB/s |  13 kB     00:00    
(9/10): mailcap-2.1.48-4.fc29.noarch.rpm        186 kB/s |  31 kB     00:00    
(10/10): httpd-2.4.37-3.fc29.x86_64.rpm         638 kB/s | 1.3 MB     00:02    
--------------------------------------------------------------------------------
Total                                           356 kB/s | 1.8 MB     00:05     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
Installed: apr-1.6.5-1.fc29.x86_64
  Installing       : apr-1.6.5-1.fc29.x86_64                               1/10 
  Running scriptlet: apr-1.6.5-1.fc29.x86_64                               1/10 
Installed: apr-1.6.5-1.fc29.x86_64
Installed: apr-util-bdb-1.6.1-8.fc29.x86_64
  Installing       : apr-util-bdb-1.6.1-8.fc29.x86_64                      2/10 
Installed: apr-util-bdb-1.6.1-8.fc29.x86_64
Installed: apr-util-openssl-1.6.1-8.fc29.x86_64
  Installing       : apr-util-openssl-1.6.1-8.fc29.x86_64                  3/10 
Installed: apr-util-openssl-1.6.1-8.fc29.x86_64
Installed: apr-util-1.6.1-8.fc29.x86_64
  Installing       : apr-util-1.6.1-8.fc29.x86_64                          4/10 
  Running scriptlet: apr-util-1.6.1-8.fc29.x86_64                          4/10 
Installed: apr-util-1.6.1-8.fc29.x86_64
Installed: httpd-tools-2.4.37-3.fc29.x86_64
  Installing       : httpd-tools-2.4.37-3.fc29.x86_64                      5/10 
Installed: httpd-tools-2.4.37-3.fc29.x86_64
Installed: mailcap-2.1.48-4.fc29.noarch
  Installing       : mailcap-2.1.48-4.fc29.noarch                          6/10 
Installed: mailcap-2.1.48-4.fc29.noarch
Installed: fedora-logos-httpd-28.0.3-2.fc29.noarch
  Installing       : fedora-logos-httpd-28.0.3-2.fc29.noarch               7/10 
Installed: fedora-logos-httpd-28.0.3-2.fc29.noarch
Installed: httpd-filesystem-2.4.37-3.fc29.noarch
  Running scriptlet: httpd-filesystem-2.4.37-3.fc29.noarch                 8/10 
  Installing       : httpd-filesystem-2.4.37-3.fc29.noarch                 8/10 
Installed: httpd-filesystem-2.4.37-3.fc29.noarch
Installed: mod_http2-1.11.1-1.fc29.x86_64
  Installing       : mod_http2-1.11.1-1.fc29.x86_64                        9/10 
Installed: mod_http2-1.11.1-1.fc29.x86_64
Installed: httpd-2.4.37-3.fc29.x86_64
  Installing       : httpd-2.4.37-3.fc29.x86_64                           10/10 
  Running scriptlet: httpd-2.4.37-3.fc29.x86_64                           10/10 
Installed: httpd-2.4.37-3.fc29.x86_64
  Verifying        : httpd-2.4.37-3.fc29.x86_64                            1/10 
  Verifying        : httpd-filesystem-2.4.37-3.fc29.noarch                 2/10 
  Verifying        : httpd-tools-2.4.37-3.fc29.x86_64                      3/10 
  Verifying        : mod_http2-1.11.1-1.fc29.x86_64                        4/10 
  Verifying        : apr-1.6.5-1.fc29.x86_64                               5/10 
  Verifying        : apr-util-1.6.1-8.fc29.x86_64                          6/10 
  Verifying        : apr-util-bdb-1.6.1-8.fc29.x86_64                      7/10 
  Verifying        : apr-util-openssl-1.6.1-8.fc29.x86_64                  8/10 
  Verifying        : fedora-logos-httpd-28.0.3-2.fc29.noarch               9/10 
  Verifying        : mailcap-2.1.48-4.fc29.noarch                         10/10 

Installed:
  httpd-2.4.37-3.fc29.x86_64              apr-util-bdb-1.6.1-8.fc29.x86_64     
  apr-util-openssl-1.6.1-8.fc29.x86_64    httpd-filesystem-2.4.37-3.fc29.noarch
  httpd-tools-2.4.37-3.fc29.x86_64        mod_http2-1.11.1-1.fc29.x86_64       
  apr-1.6.5-1.fc29.x86_64                 apr-util-1.6.1-8.fc29.x86_64         
  fedora-logos-httpd-28.0.3-2.fc29.noarch mailcap-2.1.48-4.fc29.noarch         

Complete!
29 files removed
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
--> ff28601d570ca43a02c92a522e5b363360936e5e9cbd218e7a2901b57de77761
STEP 3: FROM ff28601d570ca43a02c92a522e5b363360936e5e9cbd218e7a2901b57de77761
STEP 4: STOPSIGNAL  SIGRTMIN+3
--> 932fd53b1da4604cf54cb55c70de9c1b1458988cd49dd712e1ed16a183ed5373
STEP 5: FROM 932fd53b1da4604cf54cb55c70de9c1b1458988cd49dd712e1ed16a183ed5373
STEP 6: CMD ["/usr/sbin/init"]
--> 56385fe195f784dd6f4abcccff055f6733c0f6b301883541ad897d73e1309367
STEP 7: FROM 56385fe195f784dd6f4abcccff055f6733c0f6b301883541ad897d73e1309367
STEP 8: EXPOSE 80:8088
--> a41e94bc62db66b2f86932b6ae1342b73f611a5616738eb57947c0420fed2b17
STEP 9: FROM a41e94bc62db66b2f86932b6ae1342b73f611a5616738eb57947c0420fed2b17
STEP 10: LABEL foo=bar
--> 3705a1201a8d9882adb7cf4c5d5039aa
 rpm -q podman
podman-0.11.1-1.gita4adfe5.fc29.x86_64
# setsebool -P container_manage_cgroup 1
# podman run -ti systemd
systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
Detected virtualization container-other.
Detected architecture x86-64.

Welcome to Fedora 29 (Twenty Nine)!

Set hostname to <ab0d894ab23a>.
Initializing machine ID from random generator.
Failed to install release agent, ignoring: No such file or directory
File /usr/lib/systemd/system/systemd-journald.service:36 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[  OK  ] Reached target Local File Systems.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Swap.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Reached target Slices.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Socket.
         Starting Rebuild Dynamic Linker Cache...
         Starting Create System Users...
         Starting Rebuild Journal Catalog...
         Starting Journal Service...
[  OK  ] Started Create System Users.
[  OK  ] Started Rebuild Journal Catalog.
[  OK  ] Started Rebuild Dynamic Linker Cache.
         Starting Update is Completed...
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Started Update is Completed.
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Reached target System Initialization.
[  OK  ] Started daily update of the root trust anchor for DNSSEC.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Timers.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
[  OK  ] Started D-Bus System Message Bus.
         Starting Permit User Sessions...
         Starting The Apache HTTP Server...
[  OK  ] Started Permit User Sessions.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

@rhatdan
Copy link
Member

@rhatdan rhatdan commented Nov 20, 2018

As seen above this works well for me. I used the entrypoint of /usr/sbin/init, which is the only difference I see. The SELinux change also needs to be made.

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 20, 2018

I think we might need an addition patch on systemd (written by @giuseppe) for rootless, which I believe was merged, but I don't know if it landed in Fedora yet.

@rhatdan
Copy link
Member

@rhatdan rhatdan commented Nov 20, 2018

@timcoote Ok getting this to work with Docker from root means that the container is running as root. Not as the User. The issue we have in systemd running as non root is being worked on.

But their is no difference security wise between running sudo podman ... and running docker with the docker.socket open to non priv users. Actually sudo is more secure.

So until systemd is fixed, I would use podman run from sudo for this workload.

@giuseppe
Copy link
Member

@giuseppe giuseppe commented Nov 21, 2018

With the last versions of the packages on Fedora 29 we are good to go.

The Dockerfile @rhatdan provided, works well with rootless podman, I've prepared a test image on quay.io with the following Dockerfile:

FROM fedora
RUN dnf -y install httpd; dnf -y clean all; systemctl enable httpd
STOPSIGNAL  SIGRTMIN+3
CMD ["/usr/sbin/init"]

and I get:

$ podman run --rm -ti --name systemd-test quay.io/giuseppe/systemd-test
systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
Detected virtualization container-other.
Detected architecture x86-64.

Welcome to Fedora 29 (Twenty Nine)!

Set hostname to <e93480482843>.
Initializing machine ID from random generator.
Failed to read AF_UNIX datagram queue length, ignoring: No such file or directory
Failed to install release agent, ignoring: Permission denied
File /usr/lib/systemd/system/systemd-journald.service:36 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Reached target Local File Systems.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Listening on Journal Socket.
         Starting Rebuild Dynamic Linker Cache...
         Starting Rebuild Journal Catalog...
         Starting Journal Service...
[  OK  ] Reached target Slices.
[  OK  ] Reached target Remote File Systems.
         Starting Create System Users...
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Started Create System Users.
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Started Rebuild Journal Catalog.
[  OK  ] Started Rebuild Dynamic Linker Cache.
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
         Starting Update is Completed...
[  OK  ] Started Update is Completed.
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started daily update of the root trust anchor for DNSSEC.
[  OK  ] Reached target Timers.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
[  OK  ] Started D-Bus System Message Bus.
         Starting The Apache HTTP Server...
         Starting Permit User Sessions...
[  OK  ] Started Permit User Sessions.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

@timcoote
Copy link
Author

@timcoote timcoote commented Nov 21, 2018

@timcoote Ok getting this to work with Docker from root means that the container is running as root. Not as the User. The issue we have in systemd running as non root is being worked on.
Just to be clear, the docker daemon is running as root, but the container process is running as a normal user. Is that consistent with what you thought I meant?

But their is no difference security wise between running sudo podman ... and running docker with the docker.socket open to non priv users. Actually sudo is more secure.

So until systemd is fixed, I would use podman run from sudo for this workload.
I'm content with that atm. I'm looking for a simple way to migrate from Raspbian systemd services to fedora-iot.

I've got this to work for me now using root, even for the later fedora distros in the container - which didn't work with docker.

One question and one concern:
Q:- is it better to use podman build, rather than buildah bud, or are they, the same in any case? (building with buildah seems to work, too.
C:- I cannot find that version of podman. To get beyond 10.1, I installed from rawhide, which gave me an f30 version, I'm assuming that this is a timing issue:

rpm -q podman
podman-0.11.2-4.dev.gitea928f2.fc30.x86_64

Thanks for the excellent help

@timcoote timcoote closed this Nov 21, 2018
@timcoote timcoote reopened this Nov 21, 2018
@timcoote
Copy link
Author

@timcoote timcoote commented Nov 21, 2018

Sorry, I thought it worth re-opening this as I noticed that the package version that worked is not automatically being pushed into f29 as it's classified as an enhancement. If it had been, I wouldn't have stumbled across this issue at all.

Would it make sense to get the newer build classified as a bugfix/automatically pushed? I'm not even sure that making that push is an issue for this issue, but it's where I'm starting.

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 21, 2018

@rhatdan
Copy link
Member

@rhatdan rhatdan commented Nov 21, 2018

@timcoote I believe we had issues with that version of podman, correct @mheon, which is why it has not been pushed. We are working on a fix for the issue.

@mheon
Copy link
Collaborator

@mheon mheon commented Nov 21, 2018

Yeah, we're waiting for a few patches to cut a new release and get that into Fedora - 0.11.1 developed some serious regressions with rootless containers. Probably have a 0.12.x when we return next week after the holiday.

@mheon
Copy link
Collaborator

@mheon mheon commented Dec 7, 2018

v0.12.1 has been released, so closing

@mheon mheon closed this Dec 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants