Massage System V context switching out of MTasker #3427

Merged
merged 4 commits into from Feb 26, 2016

Projects

None yet

4 participants

@nlyan
Contributor
nlyan commented Feb 22, 2016

Let's yank all the System-V ucontext switching out of MTasker, massaging it in to a an implementation neutral, hopefully trivial, pdns_* API.

Stack management and context chaining (similar to ucontexts 'uc_link') are left exposed, but the implementation is type-erased (void*'d), to force a clean break. This currently introduces an extra allocation, which hopefully won't matter once an alternative backend is in place, but means all the
threadWrapper guff can be hidden away.

At the same time, all manual memory management has been removed, with MThread stacks now being allocated via std::vector with a "lazy_allocator" which eliminates needless zero-initialization. (Which, compared to a context switch, is actually quite expensive: diddling 8192 bytes takes ~500ns over
~15 GB/s of memory bandwidth).

pdns_makecontext now takes a std::function object by reference, which must live until the MThread is started with pdns_swapcontext, at which point it is std::move'd on to the MThreads stack and can go away. This means a task can never be started twice (because std::function will be empty and throw). I think using std::function could simplify recursor code, but atm the MTasker makeThread() API has been left as-is. In MTasker, the MThread start routine is stashed in ThreadInfo, which also owns the context jointly (via std::shared_ptr) with any waiters. std::function shouldn't introduce any allocations when used with
trivial function pointers.

In addition, exceptions can hopefully now propagate safely from MThreads back up to, and through, schedule(), thanks to C++11s exception_ptr.

splitPointer/joinPtr, required to deal with SysVss hairy API, has also been reimplemented, because master currently still passes one pointer, which only works on 64-bit thanks to a GNU extension. The new implementation, despite using memcpy and looking verbose, still compiles down to 2-4 CPU instructions
on both sides under GCC -O2, and doesn't depend on any undefined behaviour.

This whole thing is WORKSFORME

@nlyan nlyan Massage System V context switching out of MTasker
Let's yank all the System-V ucontext switching out of MTasker, massaging it
in to a an implementation neutral, hopefully trivial, pdns_* API.

Stack management and context chaining (similar to ucontexts 'uc_link') are
left exposed, but the implementation is type-erased (void*'d), to force a
clean break. This currently introduces an extra allocation, which hopefully
won't matter once an alternative backend is in place, but means all the
threadWrapper guff can be hidden away.

At the same time, all manual memory management has been removed, with
MThread stacks now being allocated via std::vector with a "lazy_allocator"
which eliminates needless zero-initialization. (Which, compared to a context
switch, is actually quite expensive: diddling 8192 bytes takes ~500ns over
~15 GB/s of memory bandwidth).

pdns_makecontext now takes a std::function object by reference, which must
live until the MThread is started with pdns_swapcontext, at which point it is
std::move'd on to the MThreads stack and can go away. This means a task can
never be started twice (because std::function be empty and throw). I think using
std::function could simplify recursor code, but atm the MTasker makeThread() API
has been left as-is. In MTasker, the MThread start routine is stashed in
ThreadInfo, which also owns the context jointly (via std::shared_ptr) with
any waiters. std::function shouldn't introduce any allocations when used with
trivial function pointers.

In addition, exceptions can hopefully now propagate safely from MThreads
back up to, and through, schedule(), thanks to C++11s exception_ptr.

splitPointer/joinPtr, required to deal with SysVss hairy API, has also been
reimplemented, because master currently still passes one pointer, which only
works on 64-bit thanks to a GNU extension. The new implementation, despite
using memcpy and looking verbose, still compiles down to 2-4 CPU instructions
on each side under GCC -O2, and doesn't depend on any undefined behaviour.

This whole thing is WORKSFORME
5cf909f
@nlyan nlyan changed the title from Massage System V context handling out of MTasker to Massage System V context switching out of MTasker Feb 22, 2016
@nlyan
Contributor
nlyan commented Feb 23, 2016

swapcontext is dead. Long live swapcontext.

Added a Boost Context implementation, and made it the default. Tested against 1.55 on Jessie, and 1.60 on Arch (because the API changed in 1.56), and lightly stressed. I will go back and document the scary bits more thoroughly if this proves workable, but, right now, it could use someone more experienced than I to stress the hell out of it, and reveal any instability or performance regressions.

@ahupowerdns
Member

HOT

@nlyan nlyan Switch from std::function to boost::function
GNUs implementation of std::function can't eat 3 pointers without allocating,
(24 bytes) whereas both LLVM libc++ and boost::function can. boost::function
has been move enabled since 1.52 (Nov 2012).
abc7505
@nlyan
Contributor
nlyan commented Feb 24, 2016

Moved to boost::function to eliminate a 24 byte allocation.

@pieterlexis pieterlexis added this to the rec-4.0.0 milestone Feb 25, 2016
@ahupowerdns ahupowerdns merged commit b0b475b into PowerDNS:master Feb 26, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@sspans
Contributor
sspans commented Feb 26, 2016

Cool stuff!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment