Parallelized build (-T1C): Option to disable parallel tycho-surefire-plugin execution?

[sratz]

I've experimented with parallel builds using mvn -T1C and Tycho 2.7.0.

While this does give a significant performance boost, it also introduces some instability:

We have a lot of unit tests that do UI testing, e.g. verifying that certain controls have focus / are disabled / etc.

These tests become flaky when multiple test processes are executed simultaneously:

• Different Eclipse instances popping up and stealing focus
• Heavy load extends net test runtime per artifact leading to different timings

Is there a way to have -T1C only affect the dependency resolution/compile/package phases, but keep the test/verifiy/integration-test phases -- or specifically invocations of tycho-surefire-plugin -- linearized?

[laeubi]

I think the problem is more that different test-bundles getting executed in parallel correct?

That is something Tycho is not really in control of (the only think I could think of would be to add a new option to sync tests that have the UI-Harness), you have some option here:

1. You can make your test "concurrent-aware" e.g. open a port on the local host before they run, if the port could not be opened sleep for a while and try again
2. You can first run your bundles/features/... in parallel mode and then another serial run for the tests

[sratz]

Our tests are currently set to a hard 600s timeout (surefire.timeout). So 1. is not really an option, since that waiting would need to happen inside the test, making it impossible to predict a meaningful timeout.

Option 2 I don't understand, you mean

mvn -T1C clean package -DskipTests
mvn verify

?
Since we currently spend about 10 minutes just in the resolving dependencies phase, pretty much all the time that we save in the first step with T1C would be lost again in a 2nd run.

[laeubi]

I think more about having two profiles one that includes all "non test" and all "test" bundles then resolving should be faster, at laest that are the option with current state, but I think we could add an option to the TestPluginMojo that synchronizes on a lock to allow only one test-bundle to be executed.

[laeubi]

@sratz do you like to try out #739 ? It allows to set <reactorConcurrencyLevel>1</reactorConcurrencyLevel> then only one integration test should run at a given time. This could potentially slow down the build but should fix the issue you see.

[sratz]

I tried out #739 on our repository. Works very well!

• Much faster dependency computation
• Much faster compilation
• Sequential integration test execution (no test failures anymore)

The total build time went down almost 25%

(Only downside: Log file is really messed up with so much going on in parallel. So if something does go wrong its hard to match it to the affected project. But that's of course out of scope here)

[laeubi]

So if something does go wrong its hard to match it to the affected project

yeah that really far from optimal. Actually maven is open source, so maybe it would be worth to consider submitting a patch, e.g. I could think about collecting the output for one module in a parallel build and output it as a complete block once the project is done, or even adding at least a thread-id so one could match build outputs.

[hd42]

So if something does go wrong its hard to match it to the affected project

yeah that really far from optimal. Actually maven is open source, so maybe it would be worth to consider submitting a patch, e.g. I could think about collecting the output for one module in a parallel build and output it as a complete block once the project is done, or even adding at least a thread-id so one could match build outputs.

mvnd does something like this, it collects logs from parallel builds and outputs them en block. Maybe they could be persuaded to contribute that code so nobody must reinvent the wheel. If not, tycho could do this at least for its own logs.

[laeubi]

Tycho has no control over maven logs directly. If mvnd do something similar, it might be good to suggest contributing this to maven in their bugtracker.