mon: centralized config #20172

liewegas · 2018-01-29T20:13:07Z

FUTURE TODO (later PR)

clean up tell|daemon config set output (gross json that makes no sense)
converge with mgr module config options
convert ConfigKeyService to a PaxosService (and unify with ConfigMonitor?)

TODO

liewegas · 2018-01-29T20:13:39Z

jenkins render docs

liewegas · 2018-01-31T20:29:48Z

jenkins render docs

jcsp · 2018-02-02T10:41:45Z

src/mgr/MgrClient.cc

@@ -327,6 +329,9 @@ void MgrClient::send_report()
  }

  report->osd_health_metrics = std::move(osd_health_metrics);
+
+  cct->_conf->get_config_bl(&report->config_bl);


It feels a bit much to send this on every report from every service: the value_bl part in md_config_t is already sorta-stateful and explicitly reset when changes are made, so get_config_bl could return a bool to indicate whether anything changed (i.e. whether value_bl was rewritten), and then this function would only include the config in the report in that situation.

jcsp · 2018-02-02T12:21:34Z

src/mon/MonCommands.h

+	" name=who,type=CephString" \
+	" name=name,type=CephString" \
+	" name=value,type=CephString", \
+	"Cet a configuration option for one or more entities",


liewegas · 2018-02-18T15:15:42Z

retest this please

liewegas · 2018-02-18T15:15:52Z

http://pulpito.ceph.com/sage-2018-02-17_20:07:47-rados-wip-sage3-testing-2018-02-17-1222-distro-basic-smithi/
http://pulpito.ceph.com/sage-2018-02-18_02:06:12-rados-wip-sage3-testing-2018-02-17-1222-distro-basic-smithi/

liewegas · 2018-02-19T19:21:08Z

@tchaikov @ktdreyer I may need some help here. The make check build is failing with a compilation error that I can't reproduce locally (on multiple xenial boxes). And the error doesn't make any sense to me--the code is identical to that in several other mon/*.cc files that build just fine. Any ideas?

liewegas · 2018-02-19T21:47:50Z

Nevermind, I fixed it! Used cmdmap_t instead of the raw type and it seemed to resolve it (somehow one variant added a std::less<void> template arg).

liewegas · 2018-02-19T21:48:12Z

jenkins render docs

liewegas · 2018-02-19T22:22:04Z

@jcsp @jdurgin this is ready for a final look!

Pass --no-mon-config so we don't get failed to fetch mon config (--no-mon-config to skip) instead. Signed-off-by: Sage Weil <sage@redhat.com>

Some of these cases make sense to fetch mon configs, but we are deprecating ceph-disk anyway, and the tests currently make use of ceph-disk in places that do not have a mon_host defined via a ceph.conf or other environment. This avoids breaking those test cases without any real impact on users (which will either use ceph-volume or presumably remain in a legacy config environment). Signed-off-by: Sage Weil <sage@redhat.com>

Return success if the mons are pre-mimic (and thus have no config). Signed-off-by: Sage Weil <sage@redhat.com>

The normal timeouts automatically apply during the authenticate() stage, but not to the explicit wait for a config. If we don't get that quickly we shoudl retry another monitor because it is possible we will connect to an out-of-quorum (or otherwise unresponsive) mon. Signed-off-by: Sage Weil <sage@redhat.com>

Signed-off-by: Sage Weil <sage@redhat.com>

Seems to resolve a build error on some compilers! Meh. Signed-off-by: Sage Weil <sage@redhat.com>

…option If we are looking at a new value from the mon and comparing it to what we already have active, compare the non-meta-substituted form. This way a value from the mon that can't update at runtime but we have already set to the same value will not be falsely flagged as ignored. Signed-off-by: Sage Weil <sage@redhat.com>

Common_init_finish does start_service_thread and does set_safe_to_start_threads() on the cct, which switches us to 'runtime' mode where we can't accept many config options. Do that *after* we fetch our config from the mon so that we can accept+set runtime options (and not complain to stderr about it). Signed-off-by: Sage Weil <sage@redhat.com>

The keyfile arg might be - (stdin), which we can only read once. Ensure that we consume it once by intercepting the CLI value early and inserting the value into the 'key' option. This robs future code of the knowledge that the key came from --keyfile and not --key, but avoids the issue of multiple users (notably, KeyRing.cc and the OSD mkfs code). Remove the - special case from OSD at the same time, since it can no longer be reached (unless something other than the CLI specified '-', but neither ceph.conf nor the mon config make sense here). Signed-off-by: Sage Weil <sage@redhat.com>

…mons These qualify as 'daemon', but their path is usually not unique. Signed-off-by: Sage Weil <sage@redhat.com>

Signed-off-by: Sage Weil <sage@redhat.com>

It's possible that we successfully set active_con *and* time out the cond WaitUntil. Only set the error if we don't have a connection; if we set it *and* time out then let's call it success. Signed-off-by: Sage Weil <sage@redhat.com>

This is what OSDMonitor and MDSMonitor do. Signed-off-by: Sage Weil <sage@redhat.com>

Signed-off-by: Sage Weil <sage@redhat.com>

dillaman · 2018-03-09T16:14:17Z

src/vstart.sh

@@ -906,6 +842,67 @@ fi

 if [ $CEPH_NUM_MON -gt 0 ]; then
    start_mon
+
+    echo Populating config ...
+    cat <<EOF | $CEPH_BIN/ceph -c ceph.conf config assimilate-conf -i -


Note: this breaks clusters started w/ mstart.sh since you assume the cluster config is located at ceph.conf instead of $conf_fn, which in turn breaks qa/workunits/rbd/rbd_mirror.sh on a dev box.

dillaman · 2018-03-09T16:19:05Z

src/vstart.sh

 	    OSD_SECRET=$($CEPH_BIN/ceph-authtool --gen-print-key)
+	    echo "{\"cephx_secret\": \"$OSD_SECRET\"}" > dev/osd$osd/new.json


... also broken here w/ the hard-coded dev instead of $CEPH_DEV_DIR

Signed-off-by: John Spray <john.spray@redhat.com> (cherry picked from commit bd80cf9) Signed-off-by: Ernesto Puerta <epuertat@redhat.com> As Centralized Config feature (PR ceph#20172) has not been backported to Luminous, `set_store` and `get_store` methods are just wrappers around `set_config` and `get_config`, respectively.

Signed-off-by: John Spray <john.spray@redhat.com> (cherry picked from commit bd80cf9) Signed-off-by: Ernesto Puerta <epuertat@redhat.com> As Centralized Config feature (PR ceph#20172) has not been backported to Luminous, `set_store` and `get_store` methods are just wrappers around `set_config` and `get_config`, respectively. Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

liewegas added feature core wip-sage3-testing labels Jan 29, 2018

liewegas mentioned this pull request Jan 29, 2018

WIP: mon: centralized config #19289

Closed

21 tasks

liewegas force-pushed the wip-config branch from c871940 to b5058be Compare January 31, 2018 14:33

liewegas force-pushed the wip-config branch from 9cabb4b to 645c2b3 Compare February 1, 2018 21:21

jcsp reviewed Feb 2, 2018

View reviewed changes

liewegas force-pushed the wip-config branch 6 times, most recently from ab95d08 to bbe1e8a Compare February 13, 2018 03:57

liewegas force-pushed the wip-config branch 3 times, most recently from 54a683c to a74f765 Compare February 17, 2018 15:56

liewegas force-pushed the wip-config branch from d9acb4c to 0aaaddc Compare February 19, 2018 21:03

liewegas force-pushed the wip-config branch from 0aaaddc to f54d93e Compare February 19, 2018 22:21

liewegas added wip-sage3-testing and removed wip-sage3-testing labels Feb 19, 2018

liewegas added 17 commits March 6, 2018 14:44

qa/workunits/cephtool/test.sh: fix mon_caps

11e287c

Pass --no-mon-config so we don't get failed to fetch mon config (--no-mon-config to skip) instead. Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: tolerate pre-mimic mons

bee6ced

Return success if the mons are pre-mimic (and thus have no config). Signed-off-by: Sage Weil <sage@redhat.com>

common/config: add --mon-config option (opposite of --no-mon-config)

2bc34bb

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/rebuild_mondb.py: --no-mon-config to ceph-objectstore-tool

9c42d88

Signed-off-by: Sage Weil <sage@redhat.com>

test/cli/ceph-conf: fix test

250be6d

Signed-off-by: Sage Weil <sage@redhat.com>

mon/ConfigMonitor: add missing #include, tweak types

3d0b38a

Seems to resolve a build error on some compilers! Meh. Signed-off-by: Sage Weil <sage@redhat.com>

common/common_init: use unique admin_socket path for unprivileged dae…

59ee2e8

…mons These qualify as 'daemon', but their path is usually not unique. Signed-off-by: Sage Weil <sage@redhat.com>

mon/ConfigMonitor: fix dump when by_type is missing an item

b349fbe

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: fix auth timeout vs error race

148db39

It's possible that we successfully set active_con *and* time out the cond WaitUntil. Only set the error if we don't have a connection; if we set it *and* time out then let's call it success. Signed-off-by: Sage Weil <sage@redhat.com>

mon/ConfigMonitor: process subs from update_from_paxos

6d13ab2

This is what OSDMonitor and MDSMonitor do. Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: -c to ceph cli

6eba775

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: normalize key name for get_val (external) variants

a5597f0

Signed-off-by: Sage Weil <sage@redhat.com>

liewegas force-pushed the wip-config branch from 3e8d29a to a5597f0 Compare March 6, 2018 20:45

liewegas merged commit d7692a2 into ceph:master Mar 6, 2018

ceph-jenkins added the documentation label Mar 6, 2018

liewegas deleted the wip-config branch March 6, 2018 21:24

dillaman reviewed Mar 9, 2018

View reviewed changes

mon: centralized config #20172

mon: centralized config #20172

liewegas commented Jan 29, 2018 •

edited

liewegas commented Jan 29, 2018

liewegas commented Jan 31, 2018

jcsp Feb 2, 2018

jcsp Feb 2, 2018

liewegas commented Feb 18, 2018

liewegas commented Feb 18, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

dillaman Mar 9, 2018 •

edited

dillaman Mar 9, 2018

		OSD_SECRET=$($CEPH_BIN/ceph-authtool --gen-print-key)
		echo "{\"cephx_secret\": \"$OSD_SECRET\"}" > dev/osd$osd/new.json

mon: centralized config #20172

mon: centralized config #20172

Conversation

liewegas commented Jan 29, 2018 • edited

liewegas commented Jan 29, 2018

liewegas commented Jan 31, 2018

jcsp Feb 2, 2018

Choose a reason for hiding this comment

jcsp Feb 2, 2018

Choose a reason for hiding this comment

liewegas commented Feb 18, 2018

liewegas commented Feb 18, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

liewegas commented Feb 19, 2018

dillaman Mar 9, 2018 • edited

Choose a reason for hiding this comment

dillaman Mar 9, 2018

Choose a reason for hiding this comment

liewegas commented Jan 29, 2018 •

edited

dillaman Mar 9, 2018 •

edited