Add timeout with Duration and apply larger timeouts to tests that oft…

[gnodet]

…en fail on windows

[gnodet]

I've merged some changes to the CI, but we still have tests that often fail. The goal is to raise the timeout when possible to see if the CI is more green.

[lgoldstein]

I like the idea - see 2-3 minor comments

[lgoldstein]

Looks fine

[lgoldstein]

The goal is to raise the timeout when possible to see if the CI is more green.

Let's be careful not to raise it too much or failing tests will slow the tests as a whole - if each test is allowed ~30 sec. to fail and we have 100's of tests we might get a total build time of hours.

[gnodet]

That's right, but CI systems are usually slower than our development laptops. It does seem to help passing tests however. But I'm not sure to understand the argument, the problem would only happen while developing and if some tests are wrong, right ? That should not be the usual case and test cases usually fail fast for other problems. But you're right, in theory we could raise the build time _when something is wrong_. Le jeu. 27 févr. 2020 à 15:53, Lyor Goldstein <notifications@github.com> a écrit :

…

The goal is to raise the timeout when possible to see if the CI is more green. Let's be careful not to raise it too much or failing tests will slow the tests as a whole - if each test is allowed ~30 sec. to fail and we have 100's of tests we might get a total build time of *hours*. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#115?email_source=notifications&email_token=AAAUQNVI6U6A7I3SGFF7RKLRE7HWPA5CNFSM4K433GGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENEUPNQ#issuecomment-592005046>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAUQNXDIHJBG5Z4PF3PILTRE7HWPANCNFSM4K433GGA> .

-- ----------------------- Guillaume Nodet ------------------------ Red Hat, Open Source Integration Email: gnodet@redhat.com Web: http://fusesource.com Blog: http://gnodet.blogspot.com/

[lgoldstein]

My concern is that while right now tests are failing fast, in the future there might be some new code we add that causes many tests to fail after waiting a long time - thus increasing the build time even on our laptops significantly from a few minutes to many minutes. This would have a negative impact on the development cycles - and we are both pressed for time since we are doing this in our spare time. Little as it is I would rather spend my time fixing bugs rather than fixing, waiting 40 minutes for the relevant tests to fail and thus signal that I have to go and re-examine the fix...

[lgoldstein]

My concern is that while right now tests are failing fast, in the future there might be some new code we add that causes many tests to fail after waiting a long time - thus increasing the build time even on our laptops significantly from a few minutes to many minutes. This would have a negative impact on the development cycles - and we are both pressed for time since we are doing this in our spare time. Little as it is I would rather spend my time fixing bugs rather than fixing, waiting 40 minutes for the relevant tests to fail and thus signal that I have to go and re-examine the fix...

[gnodet]

I understand. I'll investigate if we can use a maven profile specific to CI to raise the timeouts and keep them lower when running the default `mvn package` command. Le jeu. 27 févr. 2020 à 16:30, Lyor Goldstein <notifications@github.com> a écrit :

…

My concern is that while right now tests are failing fast, in the future there might be some new code we add that causes many tests to fail after waiting a long time - thus increasing the build time even on our laptops significantly from a few minutes to many minutes. This would have a negative impact on the development cycles - and we are both pressed for time since we are doing this in our spare time. Little as it is I would rather spend my time fixing bugs rather than fixing, waiting 40 minutes for the relevant tests to fail and thus signal that I have to go and re-examine the fix... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#115?email_source=notifications&email_token=AAAUQNXF5AU3UMJ5T7KR3ZTRE7L77A5CNFSM4K433GGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENEYK4A#issuecomment-592020848>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAUQNSMSZVNJQXTCRNWS4TRE7L77ANCNFSM4K433GGA> .

-- ----------------------- Guillaume Nodet ------------------------ Red Hat, Open Source Integration Email: gnodet@redhat.com Web: http://fusesource.com Blog: http://gnodet.blogspot.com/

[lgoldstein]

That would be ideal... if possible, the the relevant timeouts in the tests can simply be written as

private static final Duration TIMEOUT = Duration.ofSeconds(System.getIntProperty("property.controlled.by.profile", ...some default value...)); // or something to this effect.

[lgoldstein]

I thought of something extra: if indeed we can use a Maven profile then instead of an absolute number for the timeouts we can use a "factor":

 // or something to this effect.
private static final Duration TIMEOUT =
    Duration.ofSeconds(
        (int) (...default value... * System.getFloatProperty("property.controlled.by.profile", 1.0f))));

or both (using 2 separate properties) - as the case may be: whether we need an absolute of a relative number