Fix multi-threaded task processing

[illuhad]

Hopefully fixes #728

The main change is that this introduces a mutex to lock dag_manager::flush_async(), thereby giving task submission atomic semantics.
Task submission involves two main steps:

1. Popping all DAG nodes that have been collected by the dag_builder from all threads
2. Enqueuing work to the worker thread to process and submit those nodes for execution.
   When task submission from another thread jumps in between those steps, it can happen it grabs those DAG nodes for processing. In this case, since flush_async() did not have atomic semantics, an incorrect submission order can arise.

This can lead to nodes, that should have already been submitted, being submitted too late, and the submit(); flush_sync() pattern not actually guaranteeing that everything is submitted.

This PR fixes this.

Also adds some minor improvements in this area, like improving diagnostic output, and additional debug assertions.

[al42and]

Can confirm, fixes #728 for me

[fxzjshm]

Also confirm this fixes #728 for me, by testing multi-threading on a searching pipeline. (It finally runs faster, thank you.)

[illuhad]

Thanks everyone for the confirmation and the patience!