Create ceph-mgr #10328

...rather than having a mixture of explicit __u8 and implicit enums/ints. Signed-off-by: John Spray <john.spray@redhat.com>

This is useful for correlating daemon metadata to cluster maps. Signed-off-by: John Spray <john.spray@redhat.com>

This bufferlist was being kept around as a class member but it didn't need to be. Signed-off-by: John Spray <john.spray@redhat.com>

This was a typo in 29535ca Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

The JSON output was getting broken by continuing in the wrong place. Signed-off-by: John Spray <john.spray@redhat.com>

Analogous to Objecter::with_osdmap, for safe access to the map. Signed-off-by: John Spray <john.spray@redhat.com>

...so that we get this piece of metadata from every service uniformly. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

For dumping a cmdmap to a Formatter, and for checking the prefix of a cmddesc. Used by ceph-mgr when routing commands to python modules. Signed-off-by: John Spray <john.spray@redhat.com>

The 'save_exception' local would otherwise be referenced before assignment during exception handling. Signed-off-by: John Spray <john.spray@redhat.com>

Create a CommandTable structure for places in Client, MgrClient, Objecter where we do basically the same kind of thing for sending and tracking MCommands. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Mgr will probably ultimately have different auth caps, so let's make it a separate entity up front instead of e.g. pretending to be a mon. Signed-off-by: John Spray <john.spray@redhat.com>

This is responsible for learning the locations of mgrs, picking an active one, and sharing that info with MgrClient consumers who subscribe to 'mgrmap' Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

...and store the list by a string path, for consumption by the world outside of integer perf counter/subsystem IDs. This is for consumption by MgrClient. Signed-off-by: John Spray <john.spray@redhat.com>

The ceph-mgr equivalent to monclient Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This involves giving RadosClient an embedded MgrClient. Signed-off-by: John Spray <john.spray@redhat.com>

This is "tell mgr" at the moment but it should be a slicker syntax later (ceph.in is awkward to refactor just now) Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This is derived from what used to be Calamari. Signed-off-by: John Spray <john.spray@redhat.com>

The ceph_state module is implemented in C++, and isn't available at build time, which causes import failures when api_docs.py tries to import calamari_rest in order to introspect it (MgrModule is eventually imported, which in turn tries to import ceph_state, which fails unless we stub it out). Additional changes: * Search calamari_rest.urls (not calamari_web) calamari_web isn't available - it largely held static content in Calamari, and in turn included calamari_rest.urls. Here in ceph-mgr we only have calamari_rest. * note that --list-urls does nothing, apparently * Pass actions to old as_view method django-rest-framework 3.x raises TypeError if the actions argument isn't passed to as_view(). * Use view().get_view_name() instead of metadata Attempting to access view().metadata(None)['name'] results in "AttributeError: 'super' object has no attribute 'metadata'". * Use somewhat unfriendly field class names django-rest-framework 3.x seems to have done away with type_label for fields, so instead use the field's class name, which is somewhat unfriendly, but arguably better than nothing. * import global_instance to fix ImportError This is necessary to avoid what seems to be weird import loops (if this isn't present, we later get "ImportError: cannot import name UserRequest" or similar). * Make api_examples.json optional This way we at least get bare docs with no examples, rather than no docs at all. * Explain how to generate API docs Signed-off-by: Tim Serong <tserong@suse.com>

Locks are great but it helps if you actually bother acquiring them... Signed-off-by: John Spray <john.spray@redhat.com>

No longer need to set CALAMARI_CONFIG when running. Signed-off-by: John Spray <john.spray@redhat.com>

This was going to a text file, hook it in to the proper logging. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

serve() each one in a separate thread, include a shutdown() hook so that we can tear down cleanly. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Rather than respawn a whole process, just fall back to being a standby. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Using an unsigned causes a build failure when expanding the dout macro. Signed-off-by: Tim Serong <tserong@suse.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Let's make each module its own directory. Signed-off-by: John Spray <john.spray@redhat.com>

...and expose it to python modules. Also fix some of the code around how we updated DaemonState. Signed-off-by: John Spray <john.spray@redhat.com>

The send_report loop doesn't get kicked off on configure messages unless the period is set to its initial value (0). Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This is a rough implementation that sends all the stats, without the efficiency of the queue+acks that we use when talking to the mon. Signed-off-by: John Spray <john.spray@redhat.com>

No longer need the mon to send us the pg_summary json hack. Signed-off-by: John Spray <john.spray@redhat.com>

...which is now generated properly from PGMap instead of reconstituted from some JSON. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

...so that ceph-mgr can use it too. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Previously only the errors were getting through, because other stuff was getting held up in the python logging framework. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

...to avoid leaking them and eventually running out of fds. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

So that we're not throwing out new mgrmap epochs every beacon. Signed-off-by: John Spray <john.spray@redhat.com>

...no need to tell us about all the things you're *not* handling. Signed-off-by: John Spray <john.spray@redhat.com>

...instead of stopping at the first python module that fails to load. Signed-off-by: John Spray <john.spray@redhat.com>

A demonstration of creating a simple ASCII-art status display by fusing the metadata, perf counters, and maps that are available to python modules in ceph-mgr. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

Although ceph-mgr isn't packaged yet, builders use these files to cue installation of dependencies. Signed-off-by: John Spray <john.spray@redhat.com>

Avoid handling out raw pointers to the ops, which are in fact owned by the table. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

...and remove the copypasta from mgr. mgr still doesn't do all the same logic (it doesn't have osdmap history handy) but it is now much easier to see which bits are used one place vs. the other. Signed-off-by: John Spray <john.spray@redhat.com>

This used to be the friend relationship to PGMonitor. It was never really needed anyway, it was only being used to skip the getter on `pools` Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This broken when MgrStandby was separated out, because it now eats the original OSDMap message without calling maybe_request_map. Signed-off-by: John Spray <john.spray@redhat.com>

...and change the "if False" django-rest-framework conditionals to actual version checks, with a view to genuinely supporting 2.x in future if needed. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: Tim Serong <tserong@suse.com>

This adds ceph-mgr following the same pattern used for ceph-mon, ceph-mds, ceph-osd. I've removed the boost-python BuildRequire for SUSE distros, because we don't ship that (the relevant bits are already included in boost-devel). NOTE: This gives an installable ceph-mgr, but not necessarily a runnable ceph-mgr. This is because ceph-mgr needs the following python packages: - CherryPy - Django - gevent - python-dateutil - djangorestframework Not all of these are available as distro packages. For example, on openSUSE Tumbleweed, we don't have djangorestframework available in the base distro, and our Django version is 1.9, which seems to be incompatible with ceph-mgr. I'm not sure of the situation on other distros. Signed-off-by: Tim Serong <tserong@suse.com>

This change introduces the following behaviour: - When ceph-mon starts, it will try to start ceph-mgr with the same instance id (Wants=), but will *not* fail to start if ceph-mgr doesn't start (i.e. the mon still works as it always did). - ceph-mgr will start After= ceph-mon, and will stop and start when ceph-mon stops and starts, because it's PartOf= ceph-mon. If you don't want ceph-mgr to run on the mons, you need to mask the service, i.e. `systemctl mask ceph-mgr@INSTANCE`. Hostnames are typically instance names, so `systemctl mask ceph-mgr@$(hostname)` should suffice if you wish to disable ceph-mgr on the mons. Signed-off-by: Tim Serong <tserong@suse.com>

This is a hack to inject a key for the mgr daemon, using whatever key already exists on the mon on this node to gain sufficient permissions to create the mgr key. Failure is ignored at every step (the '-' prefix) in case someone has already used some other trick to set everything up manually. Signed-off-by: Tim Serong <tserong@suse.com>

...at least, try to! This is a copy-paste of the ceph-mds packaging with a search and replace mds to mgr. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This is useful for ceph-mgr because it means that any in-mgr modules that want to use e.g. rbd can find it when running in a vstart cluster. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

This dates back from the Calamari/Salt days. Signed-off-by: John Spray <john.spray@redhat.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create ceph-mgr #10328

Create ceph-mgr #10328

Commits on Sep 29, 2016