Create ceph-mgr #10328
Create ceph-mgr #10328
Conversation
| # CALAMARI_CONFIG=src/pybind/mgr/calamari.conf \ | ||
| # django-admin api_docs | ||
| # | ||
| # This will create resources.rst (the API docs), and rest.log (which will |
There was a problem hiding this comment.
This deviates from the standard of using Sphinx to generate documentation from code by introducing custom mechanisms to introspect the code and produce docs per url endpoints. Although fine for a first pass, I don't think it helps in the long term when contributors are faced with a custom system like this.
Sphinx already has introspection and can populate sections in RST files (although it will not create a url-to-api-doc mapping), those mechanisms are well known (well tested!) and documented thoroughly.
Perhaps even more positive than the familiarity a Python developer (or some other contributor) might find with Sphinx, it is the ability to extend and customize using Sphinx (vs. a custom Django command).
There was a problem hiding this comment.
This part is carried forward from the Calamari tree -- revisiting how docs are generated would be welcome if you or anyone else is interested in working on it.
There was a problem hiding this comment.
In my completely biased opinion (what with me being the one who massaged the calamari docs generator into a somewhat functional state for this PR), I'd prefer to leave it as-is now, and fix it later. I am also happy to be the one to attempt to fix it later :-)
|
Just an addendum to @jcsp's initial comment - if you want to play with ceph-mgr (say, via vstart), you need django, python-dateutil, gevent and djangorestframework installed. The first three of these are somewhat obvious (you'll see an ImportError in the mgr log for each of those if they're not present), but that doesn't seem to be the case for djangorestframework. You'll just end up with nothing listening on port 8002, and not know why. TL;DR: do this:
(The version restriction on django may or may not be necessary -- I had trouble with 1.9 a while ago, so went with 1.8, and haven't yet re-tested with 1.9) |
jcsp
force-pushed
the
wip-ceph-mgr
branch
2 times, most recently
from
6ace069
to
75e6436
Compare
jcsp
force-pushed
the
wip-ceph-mgr
branch
2 times, most recently
from
0a551ad
to
7aedf67
Compare
| @@ -3321,7 +3321,6 @@ bool OSDMonitor::preprocess_command(MonOpRequestRef op) | |||
| // Drop error, continue to get other daemons' metadata | |||
| dout(4) << "No metadata for osd." << i << dendl; | |||
| r = 0; | |||
| continue; | |||
|
will continue the review tmr. |
jcsp
force-pushed
the
wip-ceph-mgr
branch
7 times, most recently
from
15c6d15
to
0aa6d6d
Compare
Signed-off-by: John Spray <john.spray@redhat.com>
Previously only the errors were getting through, because other stuff was getting held up in the python logging framework. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
...to avoid leaking them and eventually running out of fds. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
So that we're not throwing out new mgrmap epochs every beacon. Signed-off-by: John Spray <john.spray@redhat.com>
...no need to tell us about all the things you're *not* handling. Signed-off-by: John Spray <john.spray@redhat.com>
...instead of stopping at the first python module that fails to load. Signed-off-by: John Spray <john.spray@redhat.com>
A demonstration of creating a simple ASCII-art status display by fusing the metadata, perf counters, and maps that are available to python modules in ceph-mgr. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
Although ceph-mgr isn't packaged yet, builders use these files to cue installation of dependencies. Signed-off-by: John Spray <john.spray@redhat.com>
Avoid handling out raw pointers to the ops, which are in fact owned by the table. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
...and remove the copypasta from mgr. mgr still doesn't do all the same logic (it doesn't have osdmap history handy) but it is now much easier to see which bits are used one place vs. the other. Signed-off-by: John Spray <john.spray@redhat.com>
This used to be the friend relationship to PGMonitor. It was never really needed anyway, it was only being used to skip the getter on `pools` Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
This broken when MgrStandby was separated out, because it now eats the original OSDMap message without calling maybe_request_map. Signed-off-by: John Spray <john.spray@redhat.com>
...and change the "if False" django-rest-framework conditionals to actual version checks, with a view to genuinely supporting 2.x in future if needed. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>
This adds ceph-mgr following the same pattern used for ceph-mon, ceph-mds, ceph-osd. I've removed the boost-python BuildRequire for SUSE distros, because we don't ship that (the relevant bits are already included in boost-devel). NOTE: This gives an installable ceph-mgr, but not necessarily a runnable ceph-mgr. This is because ceph-mgr needs the following python packages: - CherryPy - Django - gevent - python-dateutil - djangorestframework Not all of these are available as distro packages. For example, on openSUSE Tumbleweed, we don't have djangorestframework available in the base distro, and our Django version is 1.9, which seems to be incompatible with ceph-mgr. I'm not sure of the situation on other distros. Signed-off-by: Tim Serong <tserong@suse.com>
This change introduces the following behaviour: - When ceph-mon starts, it will try to start ceph-mgr with the same instance id (Wants=), but will *not* fail to start if ceph-mgr doesn't start (i.e. the mon still works as it always did). - ceph-mgr will start After= ceph-mon, and will stop and start when ceph-mon stops and starts, because it's PartOf= ceph-mon. If you don't want ceph-mgr to run on the mons, you need to mask the service, i.e. `systemctl mask ceph-mgr@INSTANCE`. Hostnames are typically instance names, so `systemctl mask ceph-mgr@$(hostname)` should suffice if you wish to disable ceph-mgr on the mons. Signed-off-by: Tim Serong <tserong@suse.com>
This is a hack to inject a key for the mgr daemon, using whatever key already exists on the mon on this node to gain sufficient permissions to create the mgr key. Failure is ignored at every step (the '-' prefix) in case someone has already used some other trick to set everything up manually. Signed-off-by: Tim Serong <tserong@suse.com>
...at least, try to! This is a copy-paste of the ceph-mds packaging with a search and replace mds to mgr. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
This is useful for ceph-mgr because it means that any in-mgr modules that want to use e.g. rbd can find it when running in a vstart cluster. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
This dates back from the Calamari/Salt days. Signed-off-by: John Spray <john.spray@redhat.com>
(See ceph-mgr threads on ceph-devel for background, and docs in this PR for the python interface)
This PR includes the core functionality for enabling python modules that provide services (such as the 'rest' module here derived from calamari) and expose CLI hooks. Python modules can also store persistent pieces of data, which ceph-mgr stashes using the existing "config-key" mon mechanism.
The Mgr class and its encapsulated classes gather the cluster maps, the daemon metadata, and performance counters, and then expose all these to the python modules.
We have a new MgrMonitor class in the mon for tracking the up ceph-mgr daemons and publishing their addresses for consumption by MgrClient instances, such as those embedded in OSDs for sending telemetry to the mgrs. The perf counters from OSDs are currently sent to mgrs.
There is no auth cap enforcement for access to the mgr yet, that still needs inventing. The rest module also stores its authentication credentials in config-keys which need some greater restriction on access (either build an alternative store into MgrMonitor, or give the config-key mechanism some way of limiting who can read certain keys).
The ceph-mgr service is built with cmake, but there is no packaging yet. Add a "MGR=1" environment variable to vstart to have it start an instance for you. Use "ceph tell mgr enable_auth false" to switch off authentication and point your browser at localhost:8002/api/v2/cluster/osd to see the REST output.