Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
libflux: add flux_msg_incref() and _decref() #2334
This PR was peeled off from work on #2302. It adds
One tradeoff is allowing a
We had discussed providing some real support for the recurring "deferrred request" thing. I was more focused the potential copy avoidance in plumbing for #2302 and didn't have any great ideas on how to generalize that here, even having walked through several use cases, sorry.
Something people might complain of: in several places I added a little container for deferred messages, when they were stored as bare
Speaking of, I'm not optimistic that this is a big performance win. The reduced copying in this PR probably doesn't amount to a lot. However, it's not a bad thing to have in our API IMHO.
Haven't given this a full look yet, but it does seem a bit heavy handed to add a whole new type with constructor/destructor in order to preserve const here. Could the
In the end, since the new container isn't used in every instance of flux_msg_incref(), I don't find your approach distasteful at all, certainly not enough to change it in this PR.
I have a series of fixups queued that get rid of all the
Problem: it's sometimes necessary to make an effecively "const" copy of a message just to defer the message's destruction. For example, to retain a request message for asynchronous processing after a message request handler has returned. Introduce flux_msg_incref() and flux_msg_decref(). These functions take a (const flux_msg_t *) argument to allow the messsage's destruction to be deferred to the last user, without compromising its read-only protection, which promotes sharing. Fixes #2039 Also, the following cleanup: - drop inconsistently used and untested internal "magic cookie" assertions - factor out flux_msg_create_common() so we don't need to set the initial refcount in three different places.
Problem: I'm tired of doing the errno save/restore around destructor code when the only function that cannot be trusted not to clobber errno is free(3). Create ERRNO_SAFE_FREE() macro that saves and restores errno in self-contained fashion. Name the header errno_safe.h, leaving the door open for other such wrappers.
Problem: flux_msg_decode() sets errno to EINVAL in out of memory situations. Change the errno in those cases to ENOMEM. Clean up errno handling in function, now that flux_msg_destroy() takes pains not to clobber errno.
Problem: struct map_struct violates RFC 7. Rename to "struct typemap". Also rename the table from "msgtype_map" to "typemap" to make it use less horizontal space for readability.
Instead of copying a deferred request message, simply increment its refcount. Unrelated cleanup: use ERRNO_SAFE_FREE() macro in jobreq_destroy(), since errno is used afterwards by flux_log_error() and free(3) cannot be trusted not to clobber it.
Instead of copying a deferred request message, use message refcounts. Rename some functions that operate on lists of requests to have a common request_list_* prefix, for clarity.
Instead of copying a deferred job ingest request, simply increment its refcount. In job_clean(), remove code that dropped the payload in a copy of the deferred request. The message is const now so we can't do that.
Instead of copying request messages, use message refcounts in waitqueue, kvssync, and treq modules. Note: there are a couple more cases in kvs.c where request copies might be replaced with refcounted originals, but the code seemed fairly complex, so save for later.
Instead of copying deferred rexec request into future aux hash, create a struct containing refcounted message and server context, and put that in the aux hash.
@@ Coverage Diff @@ ## master #2334 +/- ## ========================================== + Coverage 80.82% 80.84% +0.01% ========================================== Files 215 215 Lines 34256 34238 -18 ========================================== - Hits 27687 27679 -8 + Misses 6569 6559 -10
This is ready to go IMHO. The coverage drop is (I think) due to the removed containers, which would have increased the ratio of new (covered) code to old (uncovered) error code that this PR touches. Anyway, I don't see an easy way to improve the coverage.