display stats/details before reset when --reset-stats is enabled

[jeremyandrews]

When enabled, the --reset-stats flag causes Goose to flush all statistics once all GooseUser's have launched. This ensures that the subsequent averages are accurate, and don't include the ramp up time.

This issue is to enhance this so we display all statistics collected before flushing them, if we are displaying statistics. When displaying these stats, it should also be visually clear that all GooseUser's have now launched, and these are the ramp-up statistics.

[LionsAd]

I think it should be default (can be follow up):

Maybe even say something like:

[Statistics]

All users hatched, resetting statistics (disable with —no-reset-stats)
——————

Reason for default: GUI in locust does it by default or at least IIRC there is a very prominent checkbox and in my career I always used the reset as it helped distinguish ramp up from sustained load.

In most cases warmup screws with max() and avg() statistics.

If we output all statistics and give easy feedback how to disable it, I don’t see why we should not have reset stats as default.

[jeremyandrews]

When --reset-stats is not enabled (the current default), the load test stops itself when run_time is hit, inclusive of the time it takes to spawn users. So, if you set --users 10 --run-time 1m the total run-time is 60 seconds, which includes 10 seconds of run-time. If you set --users 10 --run-time 5s the total run-time is 5 seconds, and we only launch 5 users.

A couple questions:

• should run-time include the time it takes to spawn users?
• does the answer change depending on if --reset-stats is enabled or disabled?
• also consider a more extreme example, --users 1000 --run-time 1m, which will take over 15 minutes to spawn all users -- currently we'd end the test after 1 minute, but without --reset-stats (or with my coming pr with --no-reset-stats) do we exit after 1 minute or after 16 minutes?

[LionsAd]

That's a hard question.

Does Locust have --run-time?

[jeremyandrews]

Yes:
https://docs.locust.io/en/stable/config-options.html

It seems they start counting from the start of the test, not after users are spawned:
https://github.com/locustio/locust/blob/629988b36c5ddf4a201168410c594f7539b38d9f/locust/main.py#L227

In our case, I assume we'd reset the timer when all user's are spawned if --reset-stats is enabled, as we want stats for that much time.

[LionsAd]

Yeah - probably cool to also allow specifying a —spawn-runtime.