rgw: set tuning parameters #1458
Conversation
alimaredia
requested review from
cbodley,
mdw-at-linuxbox,
mattbenjamin,
yehudasa and
oritwas
|
@alfredodeza this is my first foray into ceph-ansible. After reading the docs I think I got the syntax (mostly) right. ATM we still need to finalize comments and parameters. |
alimaredia
force-pushed
the
wip-rgw-tuning
branch
from
18c0f7a
to
62850ab
Compare
group_vars/all.yml.sample
Outdated
| @@ -326,6 +326,8 @@ dummy: | |||
| #radosgw_dns_s3website_name: your.subdomain.tld # subdomain used by radosgw for website bucket hosting. | |||
| # Rados Gateway options | |||
| #email_address: foo@bar.com | |||
| #TODO: PROVIDE SUGGESTIONS FOR DIFFERENT CONFIGURATIONS | |||
There was a problem hiding this comment.
Do not edit this file directly, if it's a rgw option then this should go into roles/ceph-rgw/defaults/main.yml, if it's global this goes into: roles/ceph-common/defaults/main.yml. Then run generate_group_vars_sample.sh.
| @@ -0,0 +1,9 @@ | |||
| # Uncomment below lines to create pools for the rgw | |||
There was a problem hiding this comment.
These are variables in a dict so they should go into: roles/ceph-mon/defaults/main.yml
roles/ceph-mon/tasks/ceph_keys.yml
Outdated
| @@ -56,6 +56,8 @@ | |||
|
|
|||
| - include: set_osd_pool_default_pg_num.yml | |||
|
|
|||
| - include: set_rgw_pool_pg_num.yml | |||
There was a problem hiding this comment.
See my last comment, you can remove this file.
| @@ -0,0 +1,7 @@ | |||
| --- | |||
| - name: create rgw pools if rgw_pool is defined | |||
| shell: | | |||
There was a problem hiding this comment.
the command module should be enough here.
| # Uncomment below lines to create pools for the rgw | ||
| # TODO: recommend tuning pg_nums for different configurations | ||
| #rgw_pool: | ||
| # rgw_pool_name1: |
There was a problem hiding this comment.
@cbodley are there any default values we can add to this, so the entire file isn't commented out?
| ceph --connect-timeout 5 --cluster {{ cluster }} osd pool {{ item.key }} {{ item.value.pg_num }} | ||
| changed_when: false | ||
| failed_when: false | ||
| with_dict: "{{ rgw_pool }}" |
There was a problem hiding this comment.
this task looks useful outside of rgw, maybe we should call it something more generic like create_pool_pgs.yml, and use create_pools for the dict name?
| # rgw_pool_name2: | ||
| # pg_num: y2 | ||
| # rgw_pool_name3: | ||
| # pg_num: y3 |
There was a problem hiding this comment.
does this stuff not belong under group_vars for users to customize?
alimaredia
force-pushed
the
wip-rgw-tuning
branch
from
62850ab
to
46d793b
Compare
roles/ceph-rgw/defaults/main.yml
Outdated
| # buckets which causes performance and stability issues. | ||
| #ceph_conf_overrides: | ||
| # rgw: | ||
| # radosgw_civetweb_num_threads: 50 |
There was a problem hiding this comment.
@mattbenjamin is the suggested number for this supposed to be lower? This is what it defaults to now?
roles/ceph-rgw/defaults/main.yml
Outdated
| # The below values were determined to be safe defaults to | ||
| # prevent users from putting too many objects into their | ||
| # buckets which causes performance and stability issues. | ||
| #ceph_conf_overrides: |
There was a problem hiding this comment.
@leseb is this the appropriate location and method to set rgw ceph.conf values? Also would it make sense to move these values (https://github.com/ceph/ceph-ansible/blob/master/group_vars/all.yml.sample#L290-#L328) into this file?
There was a problem hiding this comment.
No keep it here :). However, this won't work as expected I'm afraid since we have one rgw section per rgw. The issue reported into this PR: #1018 should be fixed before we can merge this.
| @@ -21,6 +21,28 @@ cephx: true | |||
| rgw_pull_port: "{{ radosgw_civetweb_port }}" | |||
| rgw_pull_proto: "http" | |||
|
|
|||
| ######## | |||
| #TUNING# | |||
| ######## | |||
There was a problem hiding this comment.
@cbodley here's my first crack at the english ;)
alimaredia
force-pushed
the
wip-rgw-tuning
branch
from
46d793b
to
8c755a5
Compare
roles/ceph-rgw/defaults/main.yml
Outdated
| # buckets which causes performance and stability issues. | ||
| #ceph_conf_overrides: | ||
| # rgw: | ||
| # radosgw_civetweb_num_threads: 100 |
There was a problem hiding this comment.
TODO: double check to make sure this conf variable is necessary to override.
|
jenkins test kraken-ansible2.2-centos7_cluster |
alimaredia
force-pushed
the
wip-rgw-tuning
branch
2 times, most recently
from
2e458e8
to
e0c75c5
Compare
alimaredia
changed the title
Change civetweb_num_thread default to 100 Add capability to override number of pgs for rgw pools. Add ceph.conf vars to enable default bucket object quota at users choosing into the ceph.conf.j2 template Resolves: rhbz#1437173 Resolves: rhbz#1391500 Signed-off-by: Ali Maredia <amaredia@redhat.com>
alimaredia
force-pushed
the
wip-rgw-tuning
branch
from
e0c75c5
to
5192e3c
Compare
Add more rgw ceph config variables and create
pools for rgw.
Suggestions for parameters in comments
Signed-off-by: Ali Maredia amaredia@redhat.com