Make Jmeter able to support nanoseconds for testing and reporting #6406

junyejiang · 2025-01-10T08:00:45Z

In my test experience, when the performance of the system under test is much too fast (for example, when testing Redis), the milliseconds is too large to reflect the performance of the system under test, so the following changes are made:

Change the unit of Jmeter Result from milliseconds to nanoseconds.
Statistical calculations use BigDecimal/BigInteger to make the statistical results more accurate and no overflow.
The test report uses ms/ns as the final output unit according to the Properties configuration.

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

My code follows the code style of this project.
I have updated the documentation accordingly.

…is much too fast (for example, when testing Redis), the ms is too large to reflect the performance of the system under test, so the following changes are made: 1. Change the unit of Jmeter Result from ms to ns. 2. Statistical calculations use BigDecimal/BigInteger to make the statistical results more accurate and no overflow. 3. The test report uses ms/ns as the final output unit according to the Properties configuration.

vlsi

I have not checked the way the report works, however, I have a few comments:

Have you checked sampleresult.useNanoTime JMeter property? If so could you describe why the property is not enough for you?
Please refrain from modifying the existing tests. In most cases, if you modify an existing test it means you break backward compatibility which is unwanted.
Please create new tests for the feature you add
Please refrain from combining multiple unrelated changes into a single commit. I think the change of double -> BigDecimal should be discussed and measured separately

src/core/src/main/java/org/apache/jmeter/report/processor/SumAggregator.java

vlsi · 2025-02-10T06:21:22Z

src/jorphan/src/main/java/org/apache/jorphan/math/StatCalculatorLong.java

 /**
 * StatCalculator for Long values
 */
-public class StatCalculatorLong extends StatCalculator<Long> {
+public class StatCalculatorLong extends StatCalculator<BigInteger> {


The signature change would break backward compatibility, so previous usages would become invalid. I'm not sure breaking JMeter clients is the right thing to do.

vlsi · 2025-02-10T06:24:36Z

src/jorphan/src/main/java/org/apache/jorphan/math/StatCalculator.java

+    private BigDecimal sum = BigDecimal.ZERO;

-    private double sumOfSquares = 0;
+    private BigDecimal sumOfSquares = BigDecimal.ZERO;



Does this change belong to "support nanoseconds" PR?
If you want to improve precision of StatCalculator, I would suggest creating a separate PR and discuss it there.

Have you checked if the change from double to BigDecimal produces no significant performance regressions?

vlsi · 2025-02-10T06:25:54Z

src/core/src/test/kotlin/org/apache/jmeter/util/CalculatorTest.kt

+        calculator.addSample(SampleResult(10 * 1000000L, 40 * 1000000L))
        assertEquals(40.0, calculator.mean, 0.001, "mean(40)")
-        calculator.addSample(SampleResult(10, 42))
+        calculator.addSample(SampleResult(10 * 1000000L, 42 * 1000000L))
        assertEquals((40.0 + 42.0) / 2, calculator.mean, 0.001, "mean(40, 42)")
-        calculator.addSample(SampleResult(10, 48).apply { sampleCount = 2 })
+        calculator.addSample(SampleResult(10 * 1000000L, 48 * 1000000L).apply { sampleCount = 2 })


It does not sound right to modify the existing tests. Please refrain from modifying the existing tests unless it is absolutely necessary.
Otherwise it looks like you break backward compatibilty as we can't require all the plugins to rewrite and recompile their code to add * 1000000L all over the place.

vlsi · 2025-02-10T06:27:10Z

bin/testfiles/HTMLReportTestFile.csv

-1551783686523,267,JR-OK,200,Good request,Thread Group 1-1,text,true,,18,8,1,1,null,0,0,0
-1551783686790,280,JR-OK,200,Good request,Thread Group 1-1,text,true,,18,8,1,1,null,0,0,0
-1551783687071,305,JR-OK,200,Good request,Thread Group 1-1,text,true,,18,8,1,1,null,0,0,0
+1551783627191000000,351000000,JR-OK,200,Good request,Thread Group 1-1,text,true,,18,8,1,1,null,0,0,0


Please refrain from modifying the existing tests. Please create a new test for the feature you add.

vlsi · 2025-02-10T06:28:52Z

bin/reportgenerator.properties

@@ -34,6 +34,8 @@
 # Reporting configuration
 #---------------------------------------------------------------------------

+jmeter.reportgenerator.msns=ms


jmeter.reportgenerator.msns is a strange property name. Consider someone would need minutes later. I would suggest something like jmeter.reportgenerator.time.unit=ns|ms|minutes|....

However,

junyejiang · 2025-02-10T07:34:55Z

I have not checked the way the report works, however, I have a few comments:

Have you checked sampleresult.useNanoTime JMeter property? If so could you describe why the property is not enough for you?

Please refrain from modifying the existing tests. In most cases, if you modify an existing test it means you break backward compatibility which is unwanted.

Please create new tests for the feature you add

Please refrain from combining multiple unrelated changes into a single commit. I think the change of double -> BigDecimal should be discussed and measured separately

Yes I did tried sampleresult.useNanoTime, and finally find out that report was still in ms, this property will not help.
My PR do break backwark compatibility, as the former JTL file was stored in ms format, My PR will change it to ns format. So that the former tests wold not pass until updated.
The same as Q2
Changes in calculators from Long to BigDecimal is because, firstly every test timestamp is 1 million bigger than before so Long could be easily overflow during accumulation; secondly BigDecimal/BigInteger is more precise than Long in calculation.

vlsi · 2025-02-24T11:38:57Z

My PR do break backwark compatibility, as the former JTL file was stored in ms format, My PR will change it to ns format. So that the former tests wold not pass until updated.

I am afraid we can't merge such PRs then. We can't break everybody's workflows when releasing a new version.

junyejiang · 2025-02-26T03:24:22Z

My PR do break backwark compatibility, as the former JTL file was stored in ms format, My PR will change it to ns format. So that the former tests wold not pass until updated.

I am afraid we can't merge such PRs then. We can't break everybody's workflows when releasing a new version.

Understood. Thanks.

vlsi · 2025-02-26T04:21:13Z

It would be great if we could support both formats at the same time, however, keeping the old scripts workable and old files parseable is important

ra0077 · 2025-02-27T15:29:52Z

+1 to have the support of both formats

junyejiang · 2025-03-01T06:19:58Z

It would be great if we could support both formats at the same time, however, keeping the old scripts workable and old files parseable is important

I will try to make it. Reopen this PR

junyejiang · 2025-03-05T10:03:07Z

@vlsi
fd12dc3
Made following changes:

Introduced new metadatas in csv, jtl and xml, these metadatas are dedicated to store ns timestamp, old metadatas are untouched.
When Jmeter loads csv, jtl or xml, it tries to read and load ns timestamp first if possible, otherwise it goes the old fasioned way.
Added new setter functions in SampleResult.java, these functions are ns timestamp related only, original constructors and factory funcions are untouched.

Above 2 changes are for backward compability.

Renamed ns/ms key in reportgenerator.properies.
Added test to check my changes.

If my PR is not accepted finally some how, I still suggest to update in calculators from Long to BigInteger/Bigdecimal, which brings better precision and will not cause overflow.

vlsi requested changes Feb 10, 2025

View reviewed changes

Merge branch 'apache:master' into ns

173f05d

junyejiang closed this Feb 26, 2025

junyejiang reopened this Mar 1, 2025

junyejiang added 3 commits March 3, 2025 12:01

Merge branch 'apache:master' into ns

b763de1

update as CR

fd12dc3

Merge branch 'apache:master' into ns

71c084c

junyejiang requested a review from vlsi March 5, 2025 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make Jmeter able to support nanoseconds for testing and reporting #6406

Make Jmeter able to support nanoseconds for testing and reporting #6406

Uh oh!

junyejiang commented Jan 10, 2025 •

edited

Loading

Uh oh!

vlsi left a comment

Uh oh!

Uh oh!

vlsi Feb 10, 2025

Uh oh!

vlsi Feb 10, 2025

Uh oh!

vlsi Feb 10, 2025

Uh oh!

vlsi Feb 10, 2025

Uh oh!

vlsi Feb 10, 2025

Uh oh!

junyejiang commented Feb 10, 2025 •

edited

Loading

Uh oh!

vlsi commented Feb 24, 2025

Uh oh!

junyejiang commented Feb 26, 2025

Uh oh!

vlsi commented Feb 26, 2025

Uh oh!

ra0077 commented Feb 27, 2025

Uh oh!

junyejiang commented Mar 1, 2025

Uh oh!

junyejiang commented Mar 5, 2025 •

edited

Loading

Uh oh!

Uh oh!

Make Jmeter able to support nanoseconds for testing and reporting #6406

Are you sure you want to change the base?

Make Jmeter able to support nanoseconds for testing and reporting #6406

Uh oh!

Conversation

junyejiang commented Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

vlsi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vlsi Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

vlsi Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

vlsi Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

vlsi Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

vlsi Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

junyejiang commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vlsi commented Feb 24, 2025

Uh oh!

junyejiang commented Feb 26, 2025

Uh oh!

vlsi commented Feb 26, 2025

Uh oh!

ra0077 commented Feb 27, 2025

Uh oh!

junyejiang commented Mar 1, 2025

Uh oh!

junyejiang commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

junyejiang commented Jan 10, 2025 •

edited

Loading

junyejiang commented Feb 10, 2025 •

edited

Loading

junyejiang commented Mar 5, 2025 •

edited

Loading