Prime Number Cache Performance Testing Improvement #21
Conversation
* Rely on Statistics package for calculating StandardDeviation, eliminate duplicated code
* Extract targetedAccessTest from BytePrimeCacheTest Update BytePrimeCacheTest: * Refactor prime number list to be public and static, accessible from PerformanceTest class
* Add Statistics import for easy data summary * Create FindPrime method test case, which collects nanosecond time measurements for various prime number search cases
* Code style / format correction
DK96-OS
added
bug
* Create Print function for processing and displaying time results * Improve FindPrime Test stability by wrapping in a coroutine * Refactor TargetedAccess Test to use new measurement process
* Fix PrintTestResults, now removing the 3 longest measurements * Wrapped TargetedAccess Test in coroutines
* Create IsPrime Test case with a range of parameters * Add more Parameters to existing test cases to collect more data, doubling certain parameters with a high variance
GitHub Actions Test Results Accuracy IssuePerformance testing using Kotlin's built in ImplicationsWithout a common reliable computing resource, reproducibility is a challenge. Comparing results between changes over time will require consideration of the system running the tests, which unfortunately will not be GitHub. Potential ResolutionsOne way that this could be resolved, is to investigate alternative OpenJDK distributions available in GitHub Actions. Another is to try to use a newer version of OpenJDK. |
* Remove Coroutines after determining that they have no visible impact on the test results * Cleanup unused imports, including removing assertions, as this test class is not concerned with correctness
* Add Java versions 14, 15, 16
* Remove unused import
Java Version Performance ComparisonWhile the precision of each measurement is still limited to 100 nanoseconds by GitHub actions, there are noteable differences between Java versions. Java version 14 is clearly the performance winner, with 15 in second place. Version 16 delivers the worst performance overall, but appears to be more stable than 11. With monotonically increasing measurements, there are no spikes in performance at specific target indices, which cannot be said for any other version. |
|
Consider that:
These statements, while true, justify removing support for 8, and adding support for 16. |
* Remove java version 8, add version 16
Improve performance testing by recording small, independent time measurements, and then performing statistical analysis to determine the reliability of each measurement.