Pacing and RPS trouble

[vokash3]

Prerequisites

• I am using the latest version of Locust
• I am reporting a bug, not asking a question

Description

Pacing problem

Hello!

I faced with the problem of a sharp RPS decrease for some time after reaching the maximum number of users for the test when using a wait_time = constant_pacing(30).

How to reproduce the problem:

1. there is a target RPS for a specific request (TEST in my case) equal to 100 requests/sec (6000 requests per minute);
2. load step = 30 seconds -> intensity = 2 scenarios per minute;
3. request response time = random value from 1 to 5 seconds;
4. required number of VU = 6000 / 2 = 3000.

Additionally:

• Locust 2.17.0;
• test duration = 10 minutes;
• use locust-plugins to send metrics to TimescaleDB.

I created a test locust script, launched it, and after reaching all 3000 VU in 30 seconds, I get a sharp drop in RPS and then the same quick return to the target intensity. In this test case, everything may not look so critical, given the return to the desired RPS, but in real tests with a large load step this causes problems due to the longer “hole” in RPS.

Expectations

I expect locust not to make such RPS drops after reaching needed VU value.

Example from JMeter

For example, the same test in JMeter doesn't have the same problem.

Command line

-m locust --timescale --pguser=postgres --pgpassword=password --class-picker -f

Locustfile contents

import random
import time

import locust
from locust import User, task, constant_pacing, LoadTestShape, run_single_user
import locust_plugins


class TestUser(User):
    # pacing = 30 seconds
    # target RPS = 100
    # required number of virtual users = 3000
    wait_time = constant_pacing(30)

    @task
    def task(self):
        request_start_time = time.time()
        time.sleep(random.randint(1, 5))  # to imitate respose time from 1 to 5 seconds
        processing_time = int((time.time() - request_start_time) * 1000)
        locust.events.request.fire(
            request_type="POST",
            name='TEST',
            response_time=processing_time,
            response_length=0,
            context=None,
            exception=None,
        )


class StagesShape(LoadTestShape):
    stages = [
        {"duration": 600, "users": 3000, "spawn_rate": 100, "user_classes": [TestUser]},
    ]

    def tick(self):
        run_time = self.get_run_time()

        for stage in self.stages:
            if run_time < stage["duration"]:
                try:
                    tick_data = (stage["users"], stage["spawn_rate"], stage["user_classes"])
                except:
                    tick_data = (stage["users"], stage["spawn_rate"])
                return tick_data

        return None


if __name__ == "__main__":
    run_single_user(TestUser)

Python version

3.11.0

Locust version

2.17.0

Operating system

macOS Ventura 13.4.1 (c) (22F770820d)

[cyberw]

Interesting. I can reproduce this, without --timescale, and at lower ramp up rates too. It is much less noticeable when users have time to run more than once before rampup is finished, but it is still there.

I think there is a bug in the logic for constant_pacing for the first time it sleeps. I think it implicitly assumes that the first run was instantaneous. It should definitely be fixable but needs some rethinking, because the wait_time function doesnt really get called until after the first iteration.

[cyberw]

Should be fixed now. Try running latest version from git, or pip install -U --pre locust in about 10 minutes.

[vokash3]

I've tried the new version (locust 2.17.1.dev60). The trouble had been solved. Thank you a lot! 🎉