Container based on opensuse:42.3 doesn't build because sudo errors

[eduardoj]

Summary

Building a openbuildservice/frontend-base image from newly built openbuildservice/base image (that is built from opensuse:42.3 image) results in this error:

[...]
Step 11/13 : RUN export NOKOGIRI_USE_SYSTEM_LIBRARIES=1; bundle install --jobs=3 --retry=3 || bundle install --jobs=3 --retry=3
 ---> Running in bbf2d55ff87c
Fetching gem metadata from https://rubygems.org/.............
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
sudo: pam_open_session: Permission denied
sudo: policy plugin failed session initialization
sudo: pam_open_session: Permission denied
sudo: policy plugin failed session initialization
[...]

Building succesfully with a previous version of opensuse:42.3 image

Building openbuildservice/frontend-base from the currently uploaded openbuildservice/base image still works.

Make sure you have docker and docker-compose installed.

> git clone --depth 1 git@github.com:openSUSE/open-build-service.git
> cd open-build-service
> docker rmi opensuse:42.3
> docker-compose up

Press Ctrl-C when all the containers are running.

> rake docker:maintainer:rebuild:frontend-base

The openbuildservice/frontend-base image should be built without any errors.

Steps to reproduce the error

> git clone --depth 1 git@github.com:openSUSE/open-build-service.git
> cd open-build-service
> docker rmi opensuse:42.3
> rake docker:maintainer:rebuild:base docker:maintainer:rebuild:frontend-base

The openbuildservice/frontend-base image throw the errors pointed out above. Probably because something changed in the opensuse:42.3 image recently (between last two months).

[davidcassany]

@eduardoj, I am puzzled, it only happens in leap 42.3, not in 42.2 neither in Tumbleweed... Looking for any difference at moment, but so far I couldn't see anything special in 42.3 yet. I am digging it.

[davidcassany]

@eduardoj, finally I found the issue, apparently there is something wrong with the pam package in Leap 42.3. For some reason the last line listed below in /etc/security/limits.conf causes this sudo issue:

# harden against fork-bombs
*               hard    nproc           16384
*               soft    nproc           4096
root            -       nproc           unlimited

No idea why it happens in Leap 42.3 and not in Leap 42.2, in both distros pam is exactly the same package, but anyway, it is solved in Tumbleweed by changing the limits configuration:

# harden against fork-bombs
*               hard    nproc           16384
*               soft    nproc           4096
# root            hard    nproc           3000
# root            soft    nproc           1850

So deleting the offending line in limits.conf solves the issue. I am not in favor of including such a hack in the base image definition, I will file a bug on pam with my findings and let the pam maintainers handle it, meanwhile you can easily include a sed call (sed -i 's/.*root.*-.*nproc.*unlimited.*$//g' /etc/security/limits.conf) in your Dockerfile to delete that line.

@eduardoj could you verify that this change in /etc/security/limits.conf solves your problem? If so then I will close the issue.

Thanks

-------- edit -------------

Just for reference I am including the bug number: bsc#1072826

[eduardoj]

So deleting the offending line in limits.conf solves the issue. I am not in favor of including such a hack in the base image definition, I will file a bug on pam with my findings and let the pam maintainers handle it, meanwhile you can easily include a sed call (sed -i 's/.root.-.*nproc.unlimited.$//g' /etc/security/limits.conf) in your Dockerfile to delete that line.

@eduardoj could you verify that this change in /etc/security/limits.conf solves your problem? If so then I will close the issue.

Adding this line to Dockerfile.423 worked like a charm:

RUN sed -i 's/.*root.*-.*nproc.*unlimited.*$//g' /etc/security/limits.conf

Thanks!!

[davidcassany]

Adding this line to Dockerfile.423 worked like a charm

Great, closing the issue then

[solvingj]

I don't think this issue is resolved for leap 42.3.

FROM opensuse/leap:42.3

RUN zypper --non-interactive install \
    sudo

#fix bug: https://github.com/openSUSE/docker-containers/issues/82
RUN sed -i 's/.*root.*-.*nproc.*unlimited.*$//g' /etc/security/limits.conf

CMD sudo ls

docker build . -t leap-pam-issue
docker run --rm leap-pam-issue

sudo: pam_open_session: Permission denied
sudo: policy plugin failed session initialization

[solvingj]

Also of note, the default limits.conf in leap 42.3 does not have the offending line anymore:

# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#<domain>        <type>  <item>  <value>
#
#Where:
#<domain> can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open file descriptors
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

# harden against fork-bombs
*               hard    nproc           16384
*               soft    nproc           4096

# End of file

[solvingj]

I just determined that this issue is occurring on our Windows hosts running linux containers. It does not occur when running on a Linux host.

[Vogtinator]

If I read the bug report correctly, the issue seems to be that the policies specified in the container are higher than the ones the host allows. That might explain why it only fails on some hosts.

[solvingj]

No, sorry to bother, per my previous comment, this only happens on windows, thus, it's an issue with the LCOW layer. I've opened a new ticket with them accordingly. LCOW is known to have incomplete support for a number of things. For example, it just ignores the docker USER directive currently.

[Vogtinator]

It might not be entirely. What's the output of ulimit -a before sudo ls?

[solvingj]

af5ded9cb5bf:/ # ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3766
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3766
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
af5ded9cb5bf:/ #

[Vogtinator]

Yeah, that's it - LCOW sets the nproc limit to 3766, but sudo tries to set it higher, to 16384.
Can be reproduced on any host:

63c4e99d15e1:/ # ulimit -Su 1024
63c4e99d15e1:/ # ulimit -Hu 4096
63c4e99d15e1:/ # sudo ls
sudo: pam_open_session: Permission denied
sudo: policy plugin failed session initialization

[solvingj]

Thanks for diagnosing! Since you understand the issue, can you see any manual workaround we can do?

[Vogtinator]

Yes, removing the limits completely: echo > /etc/security/limits.conf

[solvingj]

Works, thanks so much!