Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mgr: python interface rework + enable modules to run in standby mode #16651

Merged
merged 32 commits into from Nov 1, 2017

Conversation

Projects
None yet
5 participants
@jcsp
Copy link
Contributor

commented Jul 28, 2017

  • Update documentation + release notes for "mgr services" command.
  • Add mechanism for python modules to serve in standby mode

@jcsp jcsp added the mgr label Jul 28, 2017

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from fe2ebda to dd6a676 Aug 3, 2017

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from dd6a676 to c987acb Aug 15, 2017

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from 3022d7f to 41fc275 Aug 25, 2017

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch 2 times, most recently from 5644228 to 927920b Oct 4, 2017

@jcsp jcsp changed the title [DNM] mgr module interface improvements mgr: python interface rework + enable modules to run in standby mode Oct 4, 2017

@ceph ceph deleted a comment from amitkumar50 Oct 4, 2017

@ceph ceph deleted a comment from amitkumar50 Oct 4, 2017

@wido

This comment has been minimized.

Copy link
Member

commented Oct 5, 2017

Do we already want to include the "can_run()" method here? Like we discussed on the Ceph day. Meaning that we check if a module is able to run before we attempt it.

@jcsp

This comment has been minimized.

Copy link
Contributor Author

commented Oct 5, 2017

@wido definitely want it, but this PR is already huge so let's do it separately afterwards

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from 927920b to 15f4add Oct 6, 2017

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Oct 6, 2017

OK - docs built

@wido

This comment has been minimized.

Copy link
Member

commented Oct 9, 2017

@jcsp: Understood! Was scanning through the code and didn't see it. Just wanted to double-check I didn't miss anything.

@tchaikov tchaikov self-requested a review Oct 9, 2017

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch 5 times, most recently from 6e5dd29 to 86f0766 Oct 17, 2017

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Oct 24, 2017

make check succeeded

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from c8f0882 to 0baa5b6 Oct 25, 2017

@ceph ceph deleted a comment from amitkumar50 Oct 26, 2017



typedef struct {

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

nit, no need to typedef a struct in C++.

// Load python code
for (const auto& module_name : mgr_map.modules) {
dout(1) << "Loading python module '" << module_name << "'" << dendl;
auto mod = std::unique_ptr<PyModule>(new PyModule(module_name));

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

could use make_unique()

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

done

@@ -184,6 +184,14 @@ def notify(self, notify_type, notify_val):
self.log_buffer.appendleft(notify_val)
elif notify_type == "pg_summary":
self.update_pool_stats()

#pg_summary = self.get("pg_summary")

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

this change is not related to the noop stub of os._exit(). can we drop it?

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

oops! done.


PyModuleRunnerThread thread;

std::string const &get_name() { return module_name; }

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

nit, make this method const.

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

done

serve_threads[i.first].reset(thread);
}
int r = modules[module_name]->load(this);
if (r != 0) {

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

shall we remove the module failed to load from modules?

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

For the moment I suggest we keep them there -- later we might want to add a mechanism for reporting to the MgrMonitor about the list of modules that we tried and failed to load, so they need remembering somewhere.

@@ -134,7 +134,19 @@ int ActivePyModule::load_commands()
// Don't need a Gil here -- this is called from ActivePyModule::load(),
// which already has one.
PyObject *command_list = PyObject_GetAttrString(pClassInstance, "COMMANDS");
assert(command_list != nullptr);
if (command_list == nullptr) {

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

Hmm. I kind of disagree with the guidelines on this one, I like being explicit that I'm checking a pointer rather than evaluating a class instance for truthiness. I don't remember if the google C++ guide takes a position on this.

Py_INCREF(&BaseMgrModuleType);
PyModule_AddObject(ceph_module, "BaseMgrModule",
(PyObject *)&BaseMgrModuleType);
auto load_class = [ceph_module](std::string name, PyTypeObject *type)

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

nit, could just pass const char* instead of a std::string

Py_InitModule("ceph_crushmap", CRUSHMapMethods);
*/

auto load_class = [ceph_module](std::string name, PyTypeObject *type)

This comment has been minimized.

Copy link
@tchaikov

tchaikov Oct 26, 2017

Contributor

nit, pass const char* instead.

This comment has been minimized.

Copy link
@jcsp

jcsp Nov 1, 2017

Author Contributor

done

@tchaikov

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

@jcsp, could we split the mds related changes into separated PR(s)?

John Spray and others added some commits Aug 15, 2017

John Spray
mgr: clean up python source file naming
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: enable running modules in standby mode
Modules can implement a second, separate class
that has access to very little state about the
system and can't implement commands.

They have just enough information to redirect
or forward incoming requests/traffic to the
active instance of the module on the active mgr.

This enables module authors to create modules
that end users can access via any (running) mgr node
at any time, rather than having to first work out
which mgr node is active.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
pybind/mgr: add MgrStandbyModule
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: standby modules come up and run now
...they still don't have access to any config though.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr/dashboard: implement standby mode
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mon/MgrMonitor: reset services map on drop_active
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: fix os._exit overrides
These would throw an exception when passed
a status code.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: cut down duplication between active+standby
...by using PyModuleRunner class from ActivePyModule too.

Signed-off-by: John Spray <john.spray@redhat.com>
mgr/PyOSDMap: add CRUSH get_item_weight
Signed-off-by: Sage Weil <sage@redhat.com>
John Spray
mgr: fix thread naming
Was passing a reference to a local stringstream into
Thread::create, not realising that it was taking a char*
reference instead of a copy.  Result was garbage (or usually,
all threads having the name of the last one created)

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: fix py_module_registry shutdown
Was calling way too early, which did a
Py_Finalize before the modules had been
joined.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: refactor PyOSDMap etc implementation
Implement real python classes from the C side,
rather than exposing only module methods.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: update for SafeThreadState
A bunch of the previous commits were done
before this class existed, so updating in
one go instead of trying to edit history
in fine detail.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: fix ~MonCommandCompletion
This was doing a Py_DECREF outside of the Gil.

Fixes: http://tracker.ceph.com/issues/21593
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr: drop GIL around set_uri, set_health_checks
These didn't need to keep the GIL to go and do their
pure C++ parts, and by keeping it they could deadlock
while trying to take ActiveMgrModules::lock.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mgr/selftest: extend test and add background spam mode
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
qa: expand mgr testing
Some extra coverage of the dashboard, including its standby
redirect mode and the publishing of URIs.

Also invoking the command_spam mode of the selftest module.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
qa: fix mgr caps
This was still using Kraken era settings

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
doc: describe how to implement standby modules
Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
doc: describe using `mgr module ...` commands
...including the new "mgr services" command.

Signed-off-by: John Spray <john.spray@redhat.com>
John Spray
mon: include disabled modules in `mgr module ls`
Otherwise, when someone wants to see what's possible
to do with `mgr module enable` they have to trawl
through the whole mgr map dump.

Signed-off-by: John Spray <john.spray@redhat.com>

@jcsp jcsp force-pushed the jcsp:wip-mgr-module-interface branch from aff38be to 5861c4c Nov 1, 2017

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

all commits in this PR are signed

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

submodules for project are unmodified

@liewegas liewegas merged commit 2912364 into ceph:master Nov 1, 2017

2 of 5 checks passed

Docs: build check Docs: building
Details
make check running make check
Details
make check (arm64) running make check
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

OK - docs built

@jcsp jcsp deleted the jcsp:wip-mgr-module-interface branch Nov 1, 2017

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

make check succeeded

1 similar comment
@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

make check succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.