Skip build and test on OSes that have no changes

[jonahgraham]

Many of the PRs on SWT are OS specific, this change only runs the build on OSes if there are changes that can affect that OS.

This is going to be more important with #2714 because potentially a number of new jobs will be run per PR and avoiding some of the extra time waiting will be a benefit to all.

The minor negative of skipping tests conditionally is that the Test Results summary comment will report less runs that base commit.

Part of #2714

[github-actions]

Test Results

  118 files  ±0    118 suites  ±0   17m 25s ⏱️ +43s
4 652 tests ±0  4 634 ✅ ±0  18 💤 ±0  0 ❌ ±0 
  338 runs  ±0    334 ✅ ±0   4 💤 ±0  0 ❌ ±0 

Results for commit 73e542b. ± Comparison against base commit e84097b.

♻️ This comment has been updated with latest results.

[jonahgraham]

The minor negative of skipping tests conditionally is that the Test Results summary comment will report less runs that base commit.

I put together an example of what this looks like on my fork. I pushed this commit as my master HEAD, and then made three PRs:

• test results on master: https://github.com/jonahgraham/eclipse.platform.swt/actions/runs/19306877246/job/55218442382
• cocoa only change cocoa only jonahgraham/eclipse.platform.swt#12
• win32 only change win32 only change jonahgraham/eclipse.platform.swt#11
• other change, so all run other change (workflow file change) jonahgraham/eclipse.platform.swt#13

You end up with, for example, on win32 this output:

It shows number of tests/runs down compared to master because so many tests were skipped, but it still shows 0 failures as expected. Further down you can see the non-win32 runs are skipped

[HeikoKlare]

I totally like the idea of skipping tests for OSes that are not affected by a change. I am just a bit concerned about the condensed test results posted by the bot being rendered less useful. Maybe we could check if we can adapt that the results to be separated by OS, such that you can easily and properly compare for the OS(es) of interest for the current PR.

Edit: just saw that Christoph already made a similar comment: #2762 (comment)

[jonahgraham]

I am going to split this PR out from the rest of them for #2741

[laeubi]

It shows number of tests/runs down compared to master because so many tests were skipped, but it still shows 0 failures as expected. Further down you can see the non-win32 runs are skipped

As mentioned on the other PR (sorry for confusion!) I think we then better should have three check runs for all three oss to prevent the problem you see with the test drops and then have one comment for each os e.g.

• Test Results GTK
• Test Results Win32
• Test Results Cocoa

beside that I'm wondering if it really pay off much to skip the tests, the Jenkins if often the slowest one and Github has completed most often much earlier anyways.

Also I'm wondering what happens with changes to the common parts of SWT that can change behavior on other OS of course as well.

[HeikoKlare]

Also I'm wondering what happens with changes to the common parts of SWT that can change behavior on other OS of course as well.

From my understanding, there is a classification for those changes that enables all OS builds via needs.changes.outputs.other == 'true'

beside that I'm wondering if it really pay off much to skip the tests, the Jenkins if often the slowest one and Github has completed most often much earlier anyways.

Even though this is often the case, in my experience "often" rather means 60% of the cases than 90%, so avoiding unnecessary GHA runs will my in opinion still increase the average roundtrip time. And I see at least two further reasons to do this:

• Flakyness: we repeatedly run into the situation that some builds start to get flaky for different reasons; reducing the number of (unnecessary) builds that can prevent you from making progress (both in moving actual changes forward as well as in investigating the reason for builds not working properly) is beneficial.
• Resources: why should we utilize resources when we definitely know that the results are useless? It consumers material and energy without any benefit. You can argue that SWT builds are comparably small such that this does not pay off, but it's at least an additional point in combination with the other arguments.

[jonahgraham]

@laeubi / @HeikoKlare thanks for your input - I agree and will see if I have time to bring this to a conclusion.

Also I'm wondering what happens with changes to the common parts [...]

From my understanding, [...]

Yes, that is correct, anything that isn't in the platform specific changes (e.g **/win32/**) will mean everything gets run. What this means is any PR that includes/updates a test, snippet, or example change will cause all platforms to run.