IGNITE-24213 Add jmh gradle configuration and an example of usage

[PakhomovAlexander]

https://issues.apache.org/jira/browse/IGNITE-24213

[lowka]

Looks like it not possible to run a benchmark multiple times w/o any code modifications

Initial run (reduced the number of iterations to get the results a wee bit faster)

>./gradlew :ignite-transactions:jmh -PjmhBench=TransactionExpirationRegistryBenchmark.registerUnregister -PwarmUpIterations=2 -Piterations=2
....
> Task :ignite-transactions:jmhRunBytecodeGenerator
Processing 2 classes from /Users/max/Projects/java/ignite-3/modules/transactions/build/classes/java/jmh with "reflection" generator
Writing out Java source to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-sources and resources to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-resources
Processing 27 classes from /Users/max/Projects/java/ignite-3/modules/transactions/build/classes/java/test with "reflection" generator
Writing out Java source to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-sources and resources to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-resources

But when I run the task again:

./gradlew :ignite-transactions:jmh -PjmhBench=TransactionExpirationRegistryBenchmark.registerUnregister -PwarmUpIterations=2 -Piterations=2
Configuration on demand is an incubating feature.

> Task :ignite-rest-api:compileJava
Note: Generating OpenAPI Documentation
Note: Using LOWER_CAMEL_CASE property naming strategy.
Note: Writing OpenAPI file to destination: /Users/max/Projects/java/ignite-3/modules/rest-api/build/openapi/openapi.yaml
Note: Writing OpenAPI views to destination: /Users/max/Projects/java/ignite-3/modules/rest-api/build/classes/java/main/META-INF/swagger/views
Note: Creating bean classes for 47 type elements
Note: /Users/max/Projects/java/ignite-3/modules/rest-api/src/main/java/org/apache/ignite/internal/rest/exception/handler/replacement/DefaultAuthorizationExceptionHandlerReplacement.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

[PakhomovAlexander]

Looks like it not possible to run a benchmark multiple times w/o any code modifications

Initial run (reduced the number of iterations to get the results a wee bit faster)

>./gradlew :ignite-transactions:jmh -PjmhBench=TransactionExpirationRegistryBenchmark.registerUnregister -PwarmUpIterations=2 -Piterations=2
....
> Task :ignite-transactions:jmhRunBytecodeGenerator
Processing 2 classes from /Users/max/Projects/java/ignite-3/modules/transactions/build/classes/java/jmh with "reflection" generator
Writing out Java source to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-sources and resources to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-resources
Processing 27 classes from /Users/max/Projects/java/ignite-3/modules/transactions/build/classes/java/test with "reflection" generator
Writing out Java source to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-sources and resources to /Users/max/Projects/java/ignite-3/modules/transactions/build/jmh-generated-resources

But when I run the task again:

./gradlew :ignite-transactions:jmh -PjmhBench=TransactionExpirationRegistryBenchmark.registerUnregister -PwarmUpIterations=2 -Piterations=2
Configuration on demand is an incubating feature.

> Task :ignite-rest-api:compileJava
Note: Generating OpenAPI Documentation
Note: Using LOWER_CAMEL_CASE property naming strategy.
Note: Writing OpenAPI file to destination: /Users/max/Projects/java/ignite-3/modules/rest-api/build/openapi/openapi.yaml
Note: Writing OpenAPI views to destination: /Users/max/Projects/java/ignite-3/modules/rest-api/build/classes/java/main/META-INF/swagger/views
Note: Creating bean classes for 47 type elements
Note: /Users/max/Projects/java/ignite-3/modules/rest-api/src/main/java/org/apache/ignite/internal/rest/exception/handler/replacement/DefaultAuthorizationExceptionHandlerReplacement.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

do you think this is an issue?

[lowka]

I would consider such behaviour counter-intuitive (At least I didn't expect that). It is possible to run tests multiple times using the same command w/o modifying any code.

[PakhomovAlexander]

I would consider such behaviour counter-intuitive (At least I didn't expect that). It is possible to run tests multiple times using the same command w/o modifying any code.

Generally it's how gradle works. It caches tasks and their results. To really re-run things in gradle you should use ./gradlew clean. Does it make sense?

[lowka]

This plugin seems to be misconfigured (see melix/jmh-gradle-plugin#270), since gradle does not cache test runs, so it should not test benchmark runs.

I think it would be better to fix configure this task to never cache the results. If that is not possible, the docs should be updated to show how to run this task w/o calling /.gradlew clean

./gradlew :foo:jmh -PjmhBench=bar --rerun