[QC-472] Extract advanced topology

[Barthelemy]

Goal: be able to plug the QC with a different config file.
I would like to have that to test the aggregators in particular.

Also add documentation.

[Barthelemy]

@TimoWilken I am not sure what is going on but the tests are red while there are no details. Would you mind having a look when you have time ? thank you

[TimoWilken]

Hi @Barthelemy, I believe the build fails because there are conflicts between this PR and the master branch -- "Cannot merge PR into test area" is probably not the best message to show in this case, I'll try to think of a better one.

GitHub shows conflicts in Framework/CMakeLists.txt. Could you resolve that, e.g. by git merge-ing the latest master into your PR branch and fixing the conflict during that merge?

[Barthelemy]

Ok, totally missed the message, sorry about that :) I will merge manually

[Barthelemy]

@TimoWilken Hello, sorry to call again, it seems to me that o2-dataflow is not progressing.

[TimoWilken]

@Barthelemy I'll have a look!

[knopers8]

All good, I have just a few minor suggestions.

[Barthelemy]

Thank you , I have addressed all comments.

[Barthelemy]

The error on macOS is weird: [ERROR] SEVERE: Device QC-TASK-RUNNER-skeletonTask (7203) had at least one message above severity ERROR: Unhandled std::exception reached the top of main: failed setting ZMQ_EVENTS, reason: Interrupted system call, device shutting down.

I will trigger a rebuild

[Barthelemy]

@knopers8 any idea why testWorkflow is failing ? I cannot reproduce the problem on my mac and I don't see how my change can affect this test.

[knopers8]

No idea, but I've noticed that @ktf has been reworking integration with ZeroMQ in DPL, so it might be related.

[ktf]

Yes, it's definitely related to some of the reworking I was doing. Where do you get it?

[Barthelemy]

we get it in testWorkflow on Mac but only in this PR and I am not able to reproduce it on my Mac.
It happens when we quit: pctx.services().get<ControlService>().readyToQuit(QuitRequest::All);

[Barthelemy]

seems to be fixed

[ktf]

@MichaelLettrich could you check what is going on with the macOS builder?

[pzhristov]

In another pull request I saw that the brew updated python to 3.9 and then the Python-modules-list was not OK, for example there was no matching version of tensorflow. A quick solution is to link python@3.8. If we want to use python@3.9, I have a patch that needs some refinement.

[MichaelLettrich]

@pzhristov @ktf indeed the macOS machines have been updated to python 3.9 via brew yesterday. Do we need to downgrade them to 3.8?

[pzhristov]

@MichaelLettrich No need to downgrade, simply link python@3.8
brew link --force --overwrite python@3.8