Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libflux: clean up bulk message handler registration functions #1277

Merged
merged 7 commits into from Nov 13, 2017

Conversation

Projects
None yet
6 participants
@garlick
Copy link
Member

garlick commented Nov 7, 2017

This is following up on an idea @morrone had in #1135 to clean up the bulk message handler reg/unreg functions. The flux_msg_handler_t member is dropped from struct flux_msg_handler_spec, and instead a new array of message handlers is returned from flux_msg_handler_addvec().

flux_msg_handler_delvec() then accepts the array of message handlers, destroying them and freeing the array.

In addition, a small mention about rolemask security is added to the message handler man pages.

Updated users, tests, and man pages.

This will require a PR to flux-sched. Please don't merge until that's queued up.

@garlick garlick requested review from morrone and chu11 Nov 7, 2017

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 8, 2017

Coverage Status

Coverage increased (+0.03%) to 78.541% when pulling ec12939 on garlick:msg_handlers into 4456339 on flux-framework:master.

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Nov 8, 2017

Codecov Report

Merging #1277 into master will increase coverage by 0.04%.
The diff coverage is 80.19%.

@@            Coverage Diff             @@
##           master    #1277      +/-   ##
==========================================
+ Coverage   77.92%   77.97%   +0.04%     
==========================================
  Files         154      154              
  Lines       29084    29104      +20     
==========================================
+ Hits        22665    22693      +28     
+ Misses       6419     6411       -8
Impacted Files Coverage Δ
src/broker/hello.c 78.89% <100%> (+0.19%) ⬆️
src/broker/log.c 72.89% <100%> (ø) ⬆️
src/broker/content-cache.c 73.43% <100%> (ø) ⬆️
src/broker/shutdown.c 89.33% <100%> (+0.14%) ⬆️
src/broker/heaptrace.c 89.28% <100%> (ø) ⬆️
src/broker/broker.c 72.33% <100%> (-0.02%) ⬇️
src/broker/exec.c 70.38% <100%> (ø) ⬆️
src/broker/attr.c 85.78% <100%> (-0.14%) ⬇️
src/cmd/builtin/proxy.c 75.19% <100%> (+0.04%) ⬆️
src/broker/sequence.c 94.31% <100%> (+2.39%) ⬆️
... and 27 more
@chu11
Copy link
Contributor

chu11 left a comment

Nothing caught my eye, looks pretty good to me.

Just to double check, a flux-sched fixup is also needed?

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 8, 2017

Yes, I'll post a sched PR once @morrone chimes in here.

@garlick garlick force-pushed the garlick:msg_handlers branch from ec12939 to 51dc19f Nov 8, 2017

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 8, 2017

Coverage Status

Coverage increased (+0.04%) to 78.592% when pulling 51dc19f on garlick:msg_handlers into b10c1c7 on flux-framework:master.

@grondo

This comment has been minimized.

Copy link
Contributor

grondo commented Nov 8, 2017

Looks good to me as well. I would have leaned toward renaming the functions to something more like create/destroy, but after going through the code I think addvec/delvec works just as well so probably a better choice to keep the existing naming.

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 8, 2017

Thanks disembodied voice of @grondo from vacation :-)

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 8, 2017

Coverage Status

Coverage increased (+0.04%) to 78.57% when pulling d0ea037 on garlick:msg_handlers into cc5cd0a on flux-framework:master.

@garlick garlick force-pushed the garlick:msg_handlers branch from d0ea037 to 73a65bd Nov 9, 2017

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 9, 2017

Coverage Status

Coverage increased (+0.02%) to 78.577% when pulling 73a65bd on garlick:msg_handlers into b989b36 on flux-framework:master.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 10, 2017

Coverage Status

Coverage increased (+0.04%) to 78.568% when pulling 4497b47 on garlick:msg_handlers into 819d8c5 on flux-framework:master.

@morrone
Copy link
Contributor

morrone left a comment

Mainly, I would like to see the "cleanup addvec/delvec" commit reworked to only deal with the logical changes to separate the flux_msg_handler_t * member from flux_msg_handler_spec. Other logical changes should be in other commits.

There are some other smaller comments inline.

{ FLUX_MSGTYPE_REQUEST, "attr.set", setattr_request_cb, 0, NULL },
static const struct flux_msg_handler_spec handlers[] = {
{ FLUX_MSGTYPE_REQUEST, "attr.get", getattr_request_cb, FLUX_ROLE_ALL },
{ FLUX_MSGTYPE_REQUEST, "attr.list", lsattr_request_cb, FLUX_ROLE_ALL },

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

This change from 0 (FLUX_ROLE_NONE) to FLUX_ROLE_ALL looks like something that should be broken out into a separate commit.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

That's a hard one to split out. The code was setting the rolemask to 0, then changing it with code like

flux_msg_handler_allow_rolemask (handlers[0].w, FLUX_ROLE_ALL);

Since w is gone, I thought it best to change the initial setting rather than walk the handlers list or whatever.

This comment has been minimized.

@morrone

morrone Nov 11, 2017

Contributor

I'm confused about why that is hard. Doesn't

 flux_msg_handler_allow_rolemask (handlers[0].w, FLUX_ROLE_ALL);

just change into

flux_msg_handler_allow_rolemask (attrs->handlers[0], FLUX_ROLE_ALL);

No walking seems to be required.

But again, I perfectly fine with changing the place the rolemask is set. You could even make that rolemask location change in a commit before this one. It is just a change that isn't really pertinent to this commit.

This is minor enough though that I am willing to let it slide.

@@ -668,6 +669,8 @@ int main (int argc, char *argv[])
service_switch_destroy (ctx.services);
hello_destroy (ctx.hello);
attr_destroy (ctx.attrs);
shutdown_destroy (ctx.shutdown);

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

This looks like a good improvement, but unrelated to this commit's logical topic.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

Agreed, will split out.

@@ -668,6 +669,8 @@ int main (int argc, char *argv[])
service_switch_destroy (ctx.services);
hello_destroy (ctx.hello);
attr_destroy (ctx.attrs);
shutdown_destroy (ctx.shutdown);
flux_msg_handler_delvec (ctx.handlers);

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

This looks good, but I'm not a fan of the lack of symmetry. Since the matching flux_msg_handler_addvec() is hidden down one call level in broker_add_services(), I think it would be nice to have a matching broker_remove_services() where flux_msg_handler_delvec() is called.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

Good suggestion.

@@ -38,6 +38,8 @@
#include <flux/core.h>
#include "heaptrace.h"

static flux_msg_handler_t **handlers = NULL;

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

In C, file scope variables are already initialized to 0 so the "= NULL" is redundant. Not a big deal.

return -1;
}

void flux_msg_handler_delvec (struct flux_msg_handler_spec tab[])
void flux_msg_handler_delvec (flux_msg_handler_t **handlers)

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

*handlers[]?

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

OK

break; /* FLUX_MSGHANDLER_TABLE_END */
if (!h || !tab || !hp) {
errno = EINVAL;
goto error;

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

There would appear to be no reason to call flux_msg_handler_delvec() if this memory allocation fails. This should probably jump to another label just before "return -1".

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

OK.

while (!at_end (tab[count]))
count++;
if (!(handlers = calloc (count + 1, sizeof (flux_msg_handler_t *))))
goto error;

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

This shouldn't call flux_msg_handler_delvec() on error either.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

Sure

tab[i].w = NULL;
if (handlers) {
int i;
for (i = 0; handlers[i] != NULL; i++) {

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

Since we are using C99, I would prefer to see this be "for (int i =0;" rather than declare i on its own line.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

OK...

@@ -1144,19 +1147,19 @@ static int notify_status_obj (flux_t *h, jsc_handler_obj_f func, void *d)
rc = -1;
goto done;
}
if (flux_msg_handler_addvec (h, htab, (void *)ctx) < 0) {
if (flux_msg_handler_addvec (h, htab, NULL, &handlers) < 0) {

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

The elimination of (void *)ctx doesn't appear to be related to this logical commit.

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

Yes, will split out.

flux_log_error (h, "flux_msg_handler_addvec");
return (-1);
goto done;

This comment has been minimized.

@morrone

morrone Nov 10, 2017

Contributor

This jumps to the wrong place. If flux_msg_handler_addvec() failed, we shouldn't be calling flux_msg_handler_delvec().

This comment has been minimized.

@garlick

garlick Nov 10, 2017

Author Member

delvec is a no-op so it's not wrong per se. Sometimes it's nice to have a single exit path rather than returns sprinkled throughout a function. I could go either way here but I'll do as you suggest.

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 10, 2017

Mainly, I would like to see the "cleanup addvec/delvec" commit reworked to only deal with the logical changes to separate the flux_msg_handler_t * member from flux_msg_handler_spec. Other logical changes should be in other commits.

I can't separate the libflux/msg_handler changes from the rest without breaking git bisect, so unfortunately it's a long commit, but it'll be a bit more focused after I split out the other bits you've suggested.

Thanks for the detailed feedback!

@morrone

This comment has been minimized.

Copy link
Contributor

morrone commented Nov 10, 2017

I can't separate the libflux/msg_handler changes from the rest without breaking git bisect

I'm not entirely sure what you think I was asking for, but I don't think it was that. :) I was viewing "separate the flux_msg_handler_t * member from flux_msg_handler_spec" as the one big change in that commit. There was no additional implication meant to "separate the libflux/msg_handler changes from the rest".

@garlick garlick force-pushed the garlick:msg_handlers branch from 4497b47 to ec30bff Nov 10, 2017

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 10, 2017

Just pushed an update with the suggested changes.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 10, 2017

Coverage Status

Coverage increased (+0.02%) to 78.552% when pulling ec30bff on garlick:msg_handlers into 819d8c5 on flux-framework:master.

garlick added a commit to garlick/flux-sched that referenced this pull request Nov 11, 2017

track flux_msg_handler_addvec(), _delvec() changes
Track API changes proposed in flux-framework/flux-core#1277,
namely the definition of struct flux_msg_handler_spec
and the function prototypes for flux_msg_handler_addvec()
and flux_msg_handler_delvec().
@@ -158,7 +159,7 @@ static void broker_handle_signals (broker_ctx_t *ctx, zlist_t *sigwatchers);
static void broker_unhandle_signals (zlist_t *sigwatchers);

static void broker_add_services (broker_ctx_t *ctx);
static void broker_remove_services (void);
static void broker_remove_services (flux_msg_handler_t *handlers[]);

This comment has been minimized.

@morrone

morrone Nov 11, 2017

Contributor

Thanks, that is a nice improvement! But there is still a symmetry problem here. broker_add_services takes a single "ctx" and returns nothing. The caller basically knows nothing about what happened. But then to use broker_remove_services(), somehow we need to know to give it "handlers". What handlers? How do we know that we have to dive into "ctx" to provide the correct handlers? From a code reviewer and developer's perspective, the only way to know how this works is to read all of the code involved. That tells us the interface can be improved still.

A simple solution would be to make broker_remove_services look like this:

static void broker_remove_services (broker_ctx_t *ctx);

But I think we can do better still. How about making the prototypes this:

static flux_msg_handler_t **broker_add_services (broker_ctx_t *ctx);
static void broker_remove_services (flux_msg_handler_t *handlers[]);

The add function simply returns the pointer to the handlers so the caller can pass it to the remove function. This is even better because "flux_msg_handler_t **handlers;" can be defined in main() rather than in the globalish "ctx". It is always good to keep variables as local as possible.

This comment has been minimized.

@garlick

garlick Nov 11, 2017

Author Member

That's reasonable.

This comment has been minimized.

@garlick

garlick Nov 11, 2017

Author Member

I'm confused about why that is hard

Oh duh, not sure what I was thinking there. You are right.

{ FLUX_MSGTYPE_REQUEST, "attr.set", setattr_request_cb, 0, NULL },
static const struct flux_msg_handler_spec handlers[] = {
{ FLUX_MSGTYPE_REQUEST, "attr.get", getattr_request_cb, FLUX_ROLE_ALL },
{ FLUX_MSGTYPE_REQUEST, "attr.list", lsattr_request_cb, FLUX_ROLE_ALL },

This comment has been minimized.

@morrone

morrone Nov 11, 2017

Contributor

I'm confused about why that is hard. Doesn't

 flux_msg_handler_allow_rolemask (handlers[0].w, FLUX_ROLE_ALL);

just change into

flux_msg_handler_allow_rolemask (attrs->handlers[0], FLUX_ROLE_ALL);

No walking seems to be required.

But again, I perfectly fine with changing the place the rolemask is set. You could even make that rolemask location change in a commit before this one. It is just a change that isn't really pertinent to this commit.

This is minor enough though that I am willing to let it slide.


The _rolemask_ member of _struct flux_msg_handler_spec_ is passed
to `flux_msg_handler_allow_rolemask()`. If only FLUX_ROLE_OWNER should
be allowed to send messages to the handler, set _rolemask_ to zero.

This comment has been minimized.

@morrone

morrone Nov 11, 2017

Contributor

I find this very confusing. FLUX_ROLE_NONE is 0, and FLUX_ROLE_OWNER is 1. So essentially you are saying that if we want to allow FLUX_ROLE_OWNER, we should set FLUX_ROLE_NONE.

I'm not a fan of using zero in the code when we have explicitly set a name for that, FLUX_ROLE_NONE. So at the least, I would prefer to see "zero" changed to "FLUX_ROLE_NONE".

I read the flux_msg_handler_create(3) man page as instructed below, and it says:

"A message handler only receives a message if a bit-wise AND of the message
rolemask and the message handler rolemask evaluates to a non-zero value.
By default, the message handler rolemask contains only FLUX_ROLE_OWNER
(the instance owner) but may be changed with
flux_msg_handler_allow_rolemask() and flux_msg_handler_deny_rolemask(),
which add or remove roles from the message handler rolemask. FLUX_ROLE_OWNER
cannot be dropped from the message handler rolemask."

It is pretty unclear about what happens when we try to set the role to FLUX_ROLE_NONE, despite that being exactly what is instructed in this man page. I'm assuming that it means that FLUX_ROLE_NONE is ignored and the default value of FLUX_ROLE_OWNER remains.

As I can tell from the documentation, at current time there is no practical difference between FLUX_ROLE_NONE and FLUX_ROLE_OWNER. But perhaps in the future there will be? (Otherwise, why have them both?) So perhaps better than setting it to 0/FLUX_ROLE_NONE, it would be better to instruct people to set FLUX_ROLE_OWNER if they want FLUX_ROLE_OWNER.

This comment has been minimized.

@garlick

garlick Nov 11, 2017

Author Member

I'm pondering this one.

In general, I don't see a problem with 0 as a value for a bitmask as opposed to a symbolic value. FLUX_ROLE_NONE was not really intended to be used here; it was defined to be used as the initial value for the rolemask field in a message, e.g. from RFC 11

FLUX_ROLE_NONE (0) SHALL indicate "invalid rolemask".

Possibly that was a poor naming choice.

I think you are right that the way FLUX_ROLE_OWNER as described is confusing. It cannot be denied and is meaningless to allow, so perhaps instead of being described as implicitly set in the "allow mask" of a handler, it should be an error to include it in the mask or to attempt to allow/deny it. It would be unfortunate to give the impression that the owner can be denied, since the fact that the owner is all powerful is a fundamental part of the security design.

In any case this pr is about something else so maybe the best thing is to drop the man page update from this PR and do another.

garlick added some commits Nov 7, 2017

doc/flux_msg_handler_addvec(3): topic_glob is const
Problem: man page has out of date definition of
struct flux_msg_handler_spec.

Add "const" to topic_glob definition.
broker/shutdown: free shutdown resources on exit
Problem: the broker shutdown class did not properly
free its resources.

Call shutdown_destroy() from the broker, and add a
missing call to shutdown_disarm() within shutdown_destroy(),
frees frees resources associated with the shutdown timer.
broker: unregister message handlers on exit
Problem: broker exits without freeing message handlers
associated with internal services.

Add broker_remove_service() to do this, and call it
from the broker exit path.

garlick added some commits Nov 10, 2017

libflux/msg_handler: clean up addvec/delvec
Problem: use of struct flux_msg_handler_spec for both
addvec and delvec functions is confusing.

Drop flux_msg_handler_t * member from struct flux_msg_handler_spec.
In flux_msg_handler_addvec(), make 'tab' parameter const (purely
input), and add an output parameter which is a NULL terminated
list of flux_msg_handler_t's.  The list of message handlers becomes
the sole input to flux_msg_handler_delvec.

Update users and tests.

Fixes #1135.
libjsc: drop confusing addvec parameter
Problem: flux_msg_handler_addvec() is called with the
'arg' parameter set to (void *)ctx, but ctx is not yet
initialized and is not used in callbacks.

Set the 'arg' parameter to NULL to improve clarity.
doc/flux_msg_handler_addvec(3): update prototype
Update function prototypes and descriptions to reflect
removal of 'w' from struct flux_msg_handler_spec, and
addition of 'handlers' parameter to flux_msg_handler_addvec()
and replacement of 'htab' with 'handlers' in
flux_msg_handler_delvec().

@garlick garlick force-pushed the garlick:msg_handlers branch 2 times, most recently from 83fb802 to 08cd966 Nov 11, 2017

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 11, 2017

security documentation dropped for another PR, and requested change to broker_add_services() added.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 11, 2017

Coverage Status

Coverage increased (+0.04%) to 78.566% when pulling 08cd966 on garlick:msg_handlers into 819d8c5 on flux-framework:master.

@morrone

This comment has been minimized.

Copy link
Contributor

morrone commented Nov 12, 2017

Looks pretty good. Just missing the part in broker.c about keeping the "handlers" variable local to main() rather than more globally in ctx where it is not needed.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 12, 2017

Coverage Status

Coverage increased (+0.04%) to 78.565% when pulling 5d58f38 on garlick:msg_handlers into 819d8c5 on flux-framework:master.

broker: return handlers from broker_add_services
Return array of message handlers from broker_add_services()
rather than setting the result in the broker context directly,
to make the function's purpose more evident from its prototype.

Move handlers out of ctx to a local variable in main().

@garlick garlick force-pushed the garlick:msg_handlers branch from 5d58f38 to 43fa0c8 Nov 13, 2017

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 13, 2017

I went ahead and squashed that last change (where handlers is declared).

@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 13, 2017

Coverage Status

Coverage increased (+0.06%) to 78.586% when pulling 43fa0c8 on garlick:msg_handlers into 819d8c5 on flux-framework:master.

@garlick

This comment has been minimized.

Copy link
Member Author

garlick commented Nov 13, 2017

Got a buildbot failure after that rebase: a recurrence of #1103 proxy event distribution (probably a racy test) seems to be the cause. There are some false positives reported by cppcheck there too, but they were there in the last build that succeeded so I guess they do not affect overall pass/fail of the build?

I would rerun the build but not sure how to do that yet...

@morrone morrone merged commit 3c3890c into flux-framework:master Nov 13, 2017

5 checks passed

buildbot/core_standard Build done.
Details
codecov/patch 80.19% of diff hit (target 77.92%)
Details
codecov/project 77.97% (+0.04%) compared to 819d8c5
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.06%) to 78.586%
Details

@garlick garlick deleted the garlick:msg_handlers branch Nov 15, 2017

@grondo grondo referenced this pull request May 10, 2018

Closed

0.9.0 Release #1479

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.