Allow local runs of test targets to parametrize parallelism and heap size.

[jcferretti]

cfs@kultrun 12:14:31 ~/dh/oss1/deephaven-core
$ ./gradlew --no-build-cache :engine-table:testParallel -PallowParallel=true -PmaxParallelForks=80 -PforkEvery=10 -PmaxHeapSize=30g -parallel --max-workers=80 --scan                                       
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 21m 47s
181 actionable tasks: 1 executed, 180 up-to-date

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes
Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/sxumwoedurs3s

[jcferretti]

This shows timings per tests:
https://scans.gradle.com/s/sxumwoedurs3s/tests/overview

[jcferretti]

Scan for testOutOfBand: https://scans.gradle.com/s/n3prkxpli5yuk/tests/slowest-tests

[jcferretti]

Got testParallel down to 4 minutes with the latest changes:

cfs@kultrun 19:07:23 ~/dh/oss1/deephaven-core
$ ./gradlew --no-build-cache :engine-table:testParallel -PallowParallel=true -PmaxParallelForks=80 -PforkEvery=10 -PmaxHeapSize=30g -PRefreshingTableTestCase.shortTest=true -parallel --max-workers=80 --scan

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 3m 47s
181 actionable tasks: 1 executed, 180 up-to-date

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes
Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/oxl4e4xfaugfc

[jcferretti]

Scan for testSerial: https://gradle.com/s/o5wrni24nwkvi

[jcferretti]

I can run the three targets testSerial, testParallel and testOutOfBound in 13 minutes now, so I think I am satisfied.

cfs@kultrun 16:23:07 ~/dh/oss1/deephaven-core
$ ./gradlew cleanTest{Serial,Parallel,OutOfBand}

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 2s
6 actionable tasks: 3 executed, 3 up-to-date

cfs@kultrun 16:24:54 ~/dh/oss1/deephaven-core
$ ./gradlew --no-build-cache :engine-table:{testSerial,testParallel,testOutOfBand} -PallowParallel=true -PmaxParallelForks=80 -PforkEvery=10 -PmaxHeapSize=30g -PshortTests=true -parallel --max-workers=80 --scan

> Task :engine-table:testOutOfBand
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Initiating shutdown processing
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke FIRST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking FIRST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke MIDDLE shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking MIDDLE shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke LAST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking LAST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Finished shutdown processing

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 12m 57s
183 actionable tasks: 3 executed, 180 up-to-date

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes
Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/fnzohjp4osask

(note the parallel an other flags above do not apply to testSerial, they are not applied in that case)

[devinrsmith]

No major holdups, a few small things to change. As far as I could tell, the existing CI should be running exactly the same. I can't vouch that there aren't logical or race-condition issues running these tests in a more parallel nature locally. I think there is a lot of continued work we'll want / need to do as we continue to improve our test-ability.

[devinrsmith]

From an architectural standpoint, I think everything that relies on TstUtils.SHORT_TESTS (or more generally, some arbitrary number of iterations/seeds for testing) is a "code smell", at least from the perspective of "unit tests" that guard PRs from merging. It may just be around the nomenclature we use, but my future wishlist would include rebrading and reconfiguring how we think about these types of "tests".

That said, I understand the practical nature of why it is the way it is, and that we want it to be faster - and I'm not proposing you fix our architectural shortcomings. At the very least, I'm happy that we'll have hard references to a lot of these places now that SHORT_TESTS is being introduced.

[devinrsmith]

I think "Step" is a misnomer - in my mind, it implies equally-spaced additions. I think "Mult" may be better? (Note: same comment in a lot of other files too).

[jcferretti]

I actually did not notice int this case this was a factor instead of an additive step; thanks.

[devinrsmith]

As mentioned earlier, this file may be redundant depending on setup.

[jcferretti]

Run after last batch of changes:

cfs@kultrun 17:12:42 ~/dh/oss1/deephaven-core
$ ./gradlew --no-build-cache :engine-table:{testSerial,testParallel,testOutOfBand} -PallowParallel=true -PmaxParallelForks=80 -PforkEvery=10 -PmaxHeapSize=30g -PshortTests=true -parallel --max-workers=80 --scan

> Task :engine-table:testOutOfBand
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Initiating shutdown processing
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke FIRST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking FIRST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke MIDDLE shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking MIDDLE shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Starting to invoke LAST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Done invoking LAST shutdown tasks
[Thread-4] WARN io.deephaven.util.process.ShutdownManagerImpl - Finished shutdown processing

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 13m 8s
183 actionable tasks: 3 executed, 180 up-to-date

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes
Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/rjcge3fgirrzc