Sync output of parallel make tasks in CI

[smlx]

Checklist

• Affected Issues have been mentioned in the Closing issues section
• Documentation has been written/updated.
• Changelog entry has been written

The parallel tasks in CI (-jN) all write to STDOUT at once and the output is garbled. This change syncs the output to avoid the interspersed messages from parallel make tasks.

Changelog Entry

Improvement - Sync output of parallel CI tasks

Closing issues

n/a

[tobybellwood]

much easier - nice!

[Schnitzel]

This is definitely much easier to read, but unfortunately this biffers the logs while they are happening and only paste them after they are done.
And many times to debug I need realtime logs to see check which test is maybe failing to login into the worker node and check stuff on the CLI.
If make is buffering the logs this won't be possible anymore :(

Is there maybe a possibility that we run multiple sh parallel steps of Jenkins which each runs just a single make test? I guess the question is then how do we explain Jenkins it should only run two tests parallel and not all of them? But I'm sure @smlx you will find a clever way :)

[Schnitzel]

see comment above

[smlx]

hmm. okay so I've done a bit of research. make's sync-output functionality uses tmpfile(), which uses open(..., O_TMPFILE). This means that the output buffer never actually gets a pathname so we can't tail it.

I think that re-implementing make's parallel functionality in the Jenkinsfile will get messy..

maybe the best way to do this is to have a variable in the Makefile that can be set to control sync-output functionality. By default output can be synced, and you can flip the variable to disable syncing when debugging. What do you think of that solution @Schnitzel?

[Schnitzel]

ugh, yea looks like Jenkins doesn't really have a way to do this, see also https://issues.jenkins-ci.org/browse/JENKINS-44085
I guess your suggestion @smlx with a variable would be the easiest, can you add this to this PR?

[smlx]

ready for review again @Schnitzel