Skip to content

Commit

Permalink
Add flags for OSD 'docker run --cpuset-{cpus,mems}'
Browse files Browse the repository at this point in the history
Add the variables ceph_osd_docker_cpuset_cpus and
ceph_osd_docker_cpuset_mems, so that a user may specify
the CPUs and memory nodes of NUMA systems on which OSD
containers are run.

Provides a example in osds.yaml.sample to guide user
based on sample `lscpu` output since cpuset-mems refers
to the memory by NUMA node only while cpuset-cpus can
refer to individual vCPUs within a NUMA node.
  • Loading branch information
fultonj authored and guits committed Dec 14, 2017
1 parent a8a2c13 commit 8cba442
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 0 deletions.
8 changes: 8 additions & 0 deletions group_vars/osds.yml.sample
Expand Up @@ -263,6 +263,14 @@ dummy:
#ceph_osd_docker_memory_limit: 1g
#ceph_osd_docker_cpu_limit: 1

# The next two variables are undefined, and thus, unused by default.
# If `lscpu | grep NUMA` returned the following:
# NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16
# NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17
# then, the following would run the OSD on the first NUMA node only.
#ceph_osd_docker_cpuset_cpus: "0,2,4,6,8,10,12,14,16"
#ceph_osd_docker_cpuset_mems: "0"

# PREPARE DEVICE
#
# WARNING /!\ DMCRYPT scenario ONLY works with Docker version 1.12.5 and above
Expand Down
8 changes: 8 additions & 0 deletions roles/ceph-osd/defaults/main.yml
Expand Up @@ -255,6 +255,14 @@ ceph_config_keys: [] # DON'T TOUCH ME
ceph_osd_docker_memory_limit: 1g
ceph_osd_docker_cpu_limit: 1

# The next two variables are undefined, and thus, unused by default.
# If `lscpu | grep NUMA` returned the following:
# NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16
# NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17
# then, the following would run the OSD on the first NUMA node only.
#ceph_osd_docker_cpuset_cpus: "0,2,4,6,8,10,12,14,16"
#ceph_osd_docker_cpuset_mems: "0"

# PREPARE DEVICE
#
# WARNING /!\ DMCRYPT scenario ONLY works with Docker version 1.12.5 and above
Expand Down
6 changes: 6 additions & 0 deletions roles/ceph-osd/templates/ceph-osd-run.sh.j2
Expand Up @@ -54,6 +54,12 @@ expose_partitions "$1"
{% else -%}
--cpu-quota={{ ceph_osd_docker_cpu_limit * 100000 }} \
{% endif -%}
{% if ceph_osd_docker_cpuset_cpus is defined -%}
--cpuset-cpus='{{ ceph_osd_docker_cpuset_cpus }}' \
{% endif -%}
{% if ceph_osd_docker_cpuset_mems is defined -%}
--cpuset-mems='{{ ceph_osd_docker_cpuset_mems }}' \
{% endif -%}
-v /dev:/dev \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/ceph:/var/lib/ceph \
Expand Down

0 comments on commit 8cba442

Please sign in to comment.