arrivalCount and request rate do not correlate

[vibha-somashekhara]

Hi,

I am using artillery to run a few performance tests. I am using a phase with arrivalCount like below:

 - duration: 60
      arrivalCount: 10

But in summary I get back http.request_rate: .......................................... 7/sec

I have looked at your documentation here, which says The following example creates 20 virtual users in 60 seconds (one virtual user approximately every 3 seconds) and that makes sense.

• As per that, the phase I have in my test should have approx 1 request per 6 seconds. but why is the request rate 7/sec?
• Also, above test duration was only 20 secs - why did the test take 20sec when the phase was for a duration of 60 secs. (it is understandable if it was above 60secs considering requests took longer to complete)
• I ran the same test for a duration of 60 secs but by increasing arrivalCount to 20, 30, 40 and they all had request rate as 7/sec.
• When I increased the arrivalCount to 80, 90, 100 for a duration of 60secs, the request rate was zero.
  http.request_rate: .......................................... 0/sec

All above cases had no issues with requests and all requests were completed successfully (200)

Could you please help me understand this or let me know if I am missing anything with arrivalCount.

Thanks.

[hassy]

hi @vibha-somashekhara 👋 your understanding of arrivalCount phases is correct. What does your scenario look like - is there only one HTTP request?

[vibha-somashekhara]

Hi @hassy , Yes it is just one POST request.

[vibha-somashekhara]

May be it is worth mentioning I was testing this on v2.0.0-5

[tzuge]

I don't have an explanation for the observed issue, and I find the request rate calculation very confusing in general. It would be helpful if the documentation described the methodology behind the metrics shown in the report.

Regarding the request rate calculation (as far as I can tell from skimming the code):

1. The summary value appears to be a simple average of the rates calculated within the intervals and there's a TODO that seems to suggest recognition that improvements should be made:
   

   artillery/packages/core/lib/ssms.js

   Line 308 in 2a2adbb

   // TODO: retain first/last so that we have the duration

2. The rate calculation within the intervals is also confusing.

   artillery/packages/core/lib/ssms.js

   Line 552 in 2a2adbb

   this._aggregatedRates[ts][name] = round(

   
   a. Samples are aggregated every 5s but into 10s windows (?), and the denominator of the rate calculation is the difference between the first and last sample / request in the slice. I don't understand the denominator based on first and last samples. Perhaps the edges of the interval are trimmed because the intervals are not necessarily contiguous, but the rate samples are removed after calculation, so same sample in two overlapping intervals is not accounted for.
   b. A 1 second minimum is applied to the duration and the calculated rate is also rounded to 0 decimals. This seems lossy.