Replace Probe reflection implementation with Lmbda library
#25597
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JackPGreen
added
Type: Enhancement
Team: Core
Source: Internal
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/hazelcast/src/main/java/com/hazelcast/internal/metrics/impl/MethodHandleProbe.java:150:9: Cyclomatic Complexity is 13 (max allowed is 12). [CyclomaticComplexity] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/hazelcast/src/test/java/com/hazelcast/internal/metrics/impl/FieldProbeTest.java:127: Line has trailing spaces. [RegexpSingleline] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 2 errors reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/main/java/com/hazelcast/internal/metrics/impl/MethodHandleProbe.java:150:9: Cyclomatic Complexity is 13 (max allowed is 12). [CyclomaticComplexity] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/test/java/com/hazelcast/internal/metrics/impl/FieldProbeTest.java:127: Line has trailing spaces. [RegexpSingleline] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 2 errors reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/hazelcast/src/main/java/com/hazelcast/internal/metrics/impl/MethodHandleProbe.java:150:9: Cyclomatic Complexity is 13 (max allowed is 12). [CyclomaticComplexity] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There is 1 error reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/main/java/com/hazelcast/internal/metrics/impl/MethodHandleProbe.java:150:9: Cyclomatic Complexity is 13 (max allowed is 12). [CyclomaticComplexity] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There is 1 error reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.146 s <<< FAILURE! -- in com.hazelcast.osgi.CheckDependenciesIT -------------------------- [ERROR] com.hazelcast.osgi.CheckDependenciesIT.testNoMandatoryDependencyDeclared -- Time elapsed: 0.128 s <<< FAILURE! -------------------------- [ERROR] Failures: -------------------------- [ERROR] CheckDependenciesIT.testNoMandatoryDependencyDeclared:86->checkImport:124 Import org.lanternpowered.lmbda is not declared as optional -------------------------- [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 1 -------------------------- [ERROR] There are test failures. -------------------------- -------------------------- -------TEST FAILURE------- -------------------------- [INFO] Results: [INFO] [ERROR] Failures: [ERROR] CheckDependenciesIT.testNoMandatoryDependencyDeclared:86->checkImport:124 Import org.lanternpowered.lmbda is not declared as optional [INFO] [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 1 [INFO] [INFO] [INFO] --- failsafe:3.1.2:verify (default) @ hazelcast --- [INFO] Failsafe report directory: /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/hazelcast/target/failsafe-reports |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.446 s <<< FAILURE! -- in com.hazelcast.jet.core.metrics.JobMetrics_NonSharedClusterTest -------------------------- [ERROR] com.hazelcast.jet.core.metrics.JobMetrics_NonSharedClusterTest.when_noMetricCollectionYet_then_emptyMetrics -- Time elapsed: 3.289 s <<< FAILURE! -------------------------- [ERROR] Failures: -------------------------- [ERROR] JobMetrics_NonSharedClusterTest.when_noMetricCollectionYet_then_emptyMetrics:89 -------------------------- [ERROR] Tests run: 57966, Failures: 1, Errors: 0, Skipped: 213 -------------------------- [ERROR] There are test failures. -------------------------- -------------------------- -------TEST FAILURE------- -------------------------- [INFO] Results: [INFO] [ERROR] Failures: [ERROR] JobMetrics_NonSharedClusterTest.when_noMetricCollectionYet_then_emptyMetrics:89 [INFO] [ERROR] Tests run: 57966, Failures: 1, Errors: 0, Skipped: 213 [INFO] |
|
run-lab-run |
I think that's a deal-breaker: we can't depend on unreleased / unstable versions of 3rd party dependencies. In general, especially taking into account the embedded usage of Hazelcast JAR, we have always been reluctant to add dependencies to the main hazelcast artifact. There used to be a "zero dependency" policy; nowadays a few libraries are shaded in the artifact (jackson, snakeyaml), but they are still few. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When accessing the data provided by
MethodProbes/FieldProbes, reflection is used.If instead, the lmbda library is used, access performance can be improved by nearly 2x.
In #25279 steps were taken to move from core Java reflection to
MethodHandles to improve performance, but the dynamic nature of our access means the performance improvements cannot be fully realised.A similar scenario exists with
FieldProbes - a suggestion was to useVarHandlesthere, but the performance of this was worse.In both scenarios, ideally the reference should be
static finalto allow compiler folding to occur, but this isn't possible when theProbes are resolved at runtime.The lmbda library offers increased performance by internally generating a class, at runtime, that has a
static finalreference to theMethodHandlerequired, using ASM. And then wraps that in a functional interface to allow easy access.I've implemented this - for both
MethodProbes &FieldProbesand benchmarked it to compare access performance between:Probeimplementationprobe-lmbdaimplementationI've compared this for:
To get an overall picture of the effect of this change:
masterns/opprobe-lmbdans/oplongLonglongLongIt's nearly twice as fast, and there is no change in litter / memory allocation - both were already effectively 0 memory allocation.
However, it's still not as fast as direct access.
Full benchmark results:
Of note - JEP416 (Java 18+) is likely to make the current
FieldProbereflection access style slower, whereas the Lmbda implementation is unaffected - related investigation.Changes:
FieldProbes andMethodProbes now extendMethodHandleProbeMethodorFieldinto aMethodHandlewhichMethodHandleProbeuseLmbdadependency addedJitPackrepository added to get the latest version ofLmbdawith a blocking bug fixedPros:
Cons:
Fixes #25528
Breaking changes (list specific methods/types/messages):
internalclasses