libflux: add flux_event_publish() interface #1512
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #1512 +/- ##
==========================================
+ Coverage 78.53% 78.55% +0.02%
==========================================
Files 164 165 +1
Lines 30653 30814 +161
==========================================
+ Hits 24074 24207 +133
- Misses 6579 6607 +28
|
|
Yeah, I'll work on that coverage a bit! |
|
Nice work! I'll merge once you are happy with the coverage. |
Problem: the broker cmb.pub RPC handler does not handle raw (non-JSON) payloads or "private" events, and is generally a little half baked. Add a new broker service "event.pub" that is loaded only on rank 0. The event.pub request contains a topic string, flags, and optional base64-encoded payload. The event's userid and rolemask parameters are copied from the request message.
Add functions for publishing events. These functions return a future which is fulfilled once the event has been allocated a sequence number: flux_event_publish() flux_event_publish_pack() flux_event_publish_raw() The sequence number assigned to the event is available via flux_event_publish_get_seq()
Add an option: flux event pub --synchronous ... which uses flux_event_publish() to publish the event (synchronously), and prints the event's sequence number.
Switch the code in send_create_message() from the hardwired cmb.pub RPC over to flux_event_publish().
Problem: make clean in doc/man3 does not remove flux_future_then.3 build product. Add it to MAN3_FILES_SECONDARY.
Problem: the event_pub() function was getting too large and unclear. Factor out four event publication functions.
Add option to publish event with FLUX_MSGFLAG_PRIVATE set, so only sender and instance owner will receive.
Call flux-event with the -s option to exercise the event.pub RPC, with different payload types, and with and without the privacy flag.
Add descriptions for -s,--synchronous use event.pub RPC -p,--privacy set message privacy flag
Problem: flux_msg_set_payload() allows a FLUX_MSGFLAG_JSON payload to be added that is not valid JSON. Move the cursory check (not a full JSON object parse) from flux_msg_set_json() to flux_msg_set_payload(), conditional upon FLUX_MSGFLAG_JSON being set. In addtion, veryfiy that the '\0' string terminator is included in the payload, and that the string ends with '}'. Finally, add an EINVAL check that the msg parameter != NULL, since that would cause a segfault.
Problem: some tests in the message unit test will no longer be able to encode invalid messages using flux_msg_set_payload() after we tighten up the checks for JSON payloads. Expose flux_msg_set_flags() for use by the unit test.
Problem: flux_msg_get_json()'s cursory JSON check verifies open brace for JSON object but not closing one. Check for closing brace and fail with EPROTO if missing.
Problem: flux_msg_get_json unit test coverage relies on the the ability to set an invalid payload with flux_msg_set_payload(). This loophole is about to be closed. Use flux_msg_set_flags() to allow the same thing to be accomplished with a lower level interface. Also add coverage for additional some bad input to flux_msg_set_payload().
Send event.pub RPC with malformed input and ensure correct response is received.
|
I was able to get coverage up by adding a lua test to send malformed requests to the new service, and by augmenting the |
|
LGTM! |
|
Thanks! |
This was referenced May 10, 2018
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds functions for "synchronous" event publication. That is, they encapsulate an event in a request message and send it as an RPC to a broker service that assigns a sequence number to it, then responds to the RPC in parallel with the event propagation.
This was added in a more ad-hoc way recently, with the
cmb.pubbroker service. That is replaced here with a more elaborateevent.pubservice that breaks event publishing out from broker.c topublisher.[ch]. The new implementation adds a couple of missing features, like raw payloads and privacy flag.The following missing API functions were added:
The job module was switched from the ad-hoc
cmb.pubinterface to this one.flux-event(1)was extended to optionally use the above interface (flux event pub -s ...).Man pages and unit tests were (minimally) updated.