Thread pinning

[plasorak]

Better thread pinning:

• Correct a bug that made thread-pinning throw an error in some situations
• Remove the spurious error logs that appear when one tries to pin the threads without a thread-pinning script setup in boot.json.
• Reduce the logging of the thread pinning. Now the full glory of the thread-pinning script execution is only printed when nanorc was started with --loglevel debug.
• The philosophy of the 2 points above is that one only gets a readable message that the thread-pinning script is executing, and, if the thread-pinning execution fails, nanorc will print the error.
• Generalise the thread pinning, users can now specify after which transition which thread pinning is executed.

[plasorak]

This needs to be used with DUNE-DAQ/fddaqconf#20 (but not necessarily, as this is backwards compatible)

[wesketchum]

Just tested at EHN1. Works / is backwards compatible with existing configs for workflow at NP04. Reduced messages are very nice, though could additionally suppress the details on the pin command. Fixes the spurious error on pin_threads for WIBs/hermes/etc.

I would be happy to see this in fddaq-v4.3.0, even without DUNE-DAQ/fddaqconf#20.

[wesketchum]

Actually, this change is not backwards compatible, right? It means that a pin_threads command now must have an argument of the thread pinning file, and won't default to what has been set in the environment. This may be ok, but would break instructions/workflows that have been given to others.

[plasorak]

Indeed, but if one has more than one thread pinning file, I'm not sure how I should choose it without specifying an argument here.

[wesketchum]

@plasorak is it possible to split out the changes on the verbosity of print messages and the errors on WIB/hermes/etc. blocks versus the other updates that are tied to the configuration? If so, that may be enough to get in now, and allow for further discussion if needed on the other elements, assuming that is needed.

[plasorak]

I can certainly cherry-pick the commits which reduce the printout. For the WIB/Hermes errors, if you are referring to DUNE-DAQ/ehn1-operations-issues#19, yes that can be done too.

[plasorak]

done here: #244

[wesketchum]

Tested at EHN1 and it seems to be working. Thanks!