ulimits not merged when used with extends #2607

Closed
ChristianKniep opened this Issue Jan 5, 2016 · 4 comments

Comments

Projects
None yet
2 participants
@ChristianKniep

Hey there,

I try to tweak the max locked memory within a container, which could be done by starting it with this options:

--ulimit memlock="68719476736"

This results in max locked memory (kbytes, -l) 67108864.

If I am not mistaken (code) the compose script works with arbitrary key/value pairs.

If I start a container like this:

hpcg2:
    extends:
      file: base.yml
      service: hpcg
    hostname: hpcg2
    container_name: hpcg2
    environment:
     - constraint:node==venus002
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
      memlock:
        soft: 68719476736
        hard: 68719476736

The .HostConfig.Ulimits stays empty.

[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg2
[]

Started like this:

docker run -d --net=global \
                -v "/scratch/:/scratch/" \
                --dns=127.0.0.1 \
                --privileged \
                -e DC_NAME=dc1 -e CONSUL_CLUSTER_IPS=consul1,consul2 -e COLLECT_METRICS=true \
                -e SENSU_CHECK_IB=true \
                --hostname=hpcg${1} \
                --name=hpcg${1} \
                -e constraint:node==venus00${1} \
                --ulimit memlock="68719476736" \
        qnib/hpcg tail -f /dev/null

It's fine:

[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg3
[memlock=68719476736:68719476736]

I use 1.5.1:

[root@venus001 hpcg]# docker-compose version
docker-compose version: 1.5.1
docker-py version: 1.5.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
[root@venus001 hpcg]#

and 1.5.2:

[root@venus001 hpcg]# compose version
docker-compose version 1.5.2, build 7240ff3
docker-py version: 1.5.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
[root@venus001 hpcg]#

Any hint why the ulimits do not show up?
EDIT: Should docker-py be in lockstep with docker-compose? Since ulimit came with 1.5.1...

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 5, 2016

I updated docker/compose by hand (download the zip and python setup.py installed it).

The version changed, but the behavior is the same..

docker-compose version 1.6.0dev, build unknown
docker-py version: 1.7.0-dev
CPython version: 2.7.5
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013

I updated docker/compose by hand (download the zip and python setup.py installed it).

The version changed, but the behavior is the same..

docker-compose version 1.6.0dev, build unknown
docker-py version: 1.7.0-dev
CPython version: 2.7.5
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013

@dnephin dnephin added the kind/bug label Jan 5, 2016

@dnephin dnephin added this to the 1.6.0 milestone Jan 7, 2016

@dnephin dnephin added the area/config label Jan 7, 2016

@dnephin dnephin changed the title from `memlock` not part of `ulimits`? to ulimits not merged when used with extends Jan 8, 2016

@aanand aanand closed this in #2620 Jan 13, 2016

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 13, 2016

Uhi... thx mate!

Uhi... thx mate!

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 17, 2016

I am afraid it did not work for me. I installed the latest docker-compose by cloning master and execute pip install --upgrade . within the directory.

Yet, if I extend docker-compose files the ulimit is not picked up.

This works:

[root@venus001 hpcg]# cat venus001_single.yml
hpcg1:
    hostname: hpcg1
    container_name: hpcg1
    image: qnib/hpcg
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
      memlock:
        soft: 68719476736
        hard: 68719476736
    devices:
     - "/dev/infiniband/uverbs0:/dev/infiniband/uverbs0"
     - "/dev/infiniband/rdma_cm:/dev/infiniband/rdma_cm"
    net: global
    volumes:
    - /scratch/:/scratch/
    dns: 127.0.0.1
    privileged: true
    environment:
     - DC_NAME=dc1
     - CONSUL_CLUSTER_IPS=consul
     - COLLECT_METRICS=true
     - SUPERVISOR_SKIP_SRV=sensu-api,sensu-server,sensu-client
[root@venus001 hpcg]#  docker-compose -f venus001_single.yml up -d
Creating hpcg1
[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg1
[nofile=20000:40000 memlock=68719476736:68719476736]
[root@venus001 hpcg]# docker exec -ti hpcg1 bash
[root@hpcg1 /]# ulimit -l
67108864

This don't:

[root@venus001 hpcg]# cat ../base/base.yml
base:
    net: global
    volumes:
    - /scratch/:/scratch/
    dns: 127.0.0.1
    privileged: true
    environment:
     - DC_NAME=dc1
     - CONSUL_CLUSTER_IPS=consul
     - COLLECT_METRICS=true
     - SUPERVISOR_SKIP_SRV=sensu-api,sensu-server,sensu-client
[root@venus001 hpcg]# cat base.yml
base:
    extends:
      file: ../base/base.yml
      service: base
    volumes:
    - /chome/:/chome/

hpcg:
    image: qnib/hpcg
    extends:
      file: base.yml
      service: base
    environment:
     - SENSU_CHECK_IB=true
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
      memlock:
        soft: 68719476736
        hard: 68719476736
    devices:
     - "/dev/infiniband/uverbs0:/dev/infiniband/uverbs0"
     - "/dev/infiniband/rdma_cm:/dev/infiniband/rdma_cm"

[root@venus001 hpcg]# cat venus001.yml
hpcg1:
    extends:
      file: base.yml
      service: hpcg
    hostname: hpcg1
    container_name: hpcg1
[root@venus001 hpcg]# docker-compose -f venus001.yml up -d
Creating hpcg1
[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg1
[]
[root@venus001 hpcg]#

I am afraid it did not work for me. I installed the latest docker-compose by cloning master and execute pip install --upgrade . within the directory.

Yet, if I extend docker-compose files the ulimit is not picked up.

This works:

[root@venus001 hpcg]# cat venus001_single.yml
hpcg1:
    hostname: hpcg1
    container_name: hpcg1
    image: qnib/hpcg
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
      memlock:
        soft: 68719476736
        hard: 68719476736
    devices:
     - "/dev/infiniband/uverbs0:/dev/infiniband/uverbs0"
     - "/dev/infiniband/rdma_cm:/dev/infiniband/rdma_cm"
    net: global
    volumes:
    - /scratch/:/scratch/
    dns: 127.0.0.1
    privileged: true
    environment:
     - DC_NAME=dc1
     - CONSUL_CLUSTER_IPS=consul
     - COLLECT_METRICS=true
     - SUPERVISOR_SKIP_SRV=sensu-api,sensu-server,sensu-client
[root@venus001 hpcg]#  docker-compose -f venus001_single.yml up -d
Creating hpcg1
[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg1
[nofile=20000:40000 memlock=68719476736:68719476736]
[root@venus001 hpcg]# docker exec -ti hpcg1 bash
[root@hpcg1 /]# ulimit -l
67108864

This don't:

[root@venus001 hpcg]# cat ../base/base.yml
base:
    net: global
    volumes:
    - /scratch/:/scratch/
    dns: 127.0.0.1
    privileged: true
    environment:
     - DC_NAME=dc1
     - CONSUL_CLUSTER_IPS=consul
     - COLLECT_METRICS=true
     - SUPERVISOR_SKIP_SRV=sensu-api,sensu-server,sensu-client
[root@venus001 hpcg]# cat base.yml
base:
    extends:
      file: ../base/base.yml
      service: base
    volumes:
    - /chome/:/chome/

hpcg:
    image: qnib/hpcg
    extends:
      file: base.yml
      service: base
    environment:
     - SENSU_CHECK_IB=true
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
      memlock:
        soft: 68719476736
        hard: 68719476736
    devices:
     - "/dev/infiniband/uverbs0:/dev/infiniband/uverbs0"
     - "/dev/infiniband/rdma_cm:/dev/infiniband/rdma_cm"

[root@venus001 hpcg]# cat venus001.yml
hpcg1:
    extends:
      file: base.yml
      service: hpcg
    hostname: hpcg1
    container_name: hpcg1
[root@venus001 hpcg]# docker-compose -f venus001.yml up -d
Creating hpcg1
[root@venus001 hpcg]# docker inspect -f '{{ .HostConfig.Ulimits }}' hpcg1
[]
[root@venus001 hpcg]#
@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Feb 5, 2016

Contributor

@ChristianKniep

I just tried with this minimal example:

base:
    image: alpine:3.3
    command: sh

web:
    extends: base
    ulimits:
      nofile:
        soft: 20000
        hard: 40000

It worked for me with compose 1.6. docker inspect -f '{{ .HostConfig.Ulimits }}' ulimits_web_1 shows the ulimits are included.

If you have old containers you might need to rm them first.

Contributor

dnephin commented Feb 5, 2016

@ChristianKniep

I just tried with this minimal example:

base:
    image: alpine:3.3
    command: sh

web:
    extends: base
    ulimits:
      nofile:
        soft: 20000
        hard: 40000

It worked for me with compose 1.6. docker inspect -f '{{ .HostConfig.Ulimits }}' ulimits_web_1 shows the ulimits are included.

If you have old containers you might need to rm them first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment