Improve the Class Picker

[andrewbaldwin44]

Prerequisites

• I am using the latest version of Locust
• I am suggesting a new feature, not asking a question

Description

The issue

Currently the class picker only allows for selecting which user classes to run

Proposal

We should be able to configure settings on the user including the host, wait_time, weight, fixed_count, and list of tasks to run. The idea for the UI would be to have a modal that opens and allows the user to be configured.

[cyberw]

I’m not against this, but I think it is not super necessary either. When running tests you’re going to be making adjustments to the locustfile and restarting locust frequently anyway, so having things configurable in the UI doesnt add as much value as one might think.

It would be nice to be able to split Users and configuration though (maybe a yaml file containing user count, wait times, host, weights,…) Maybe after making such a feature it would be nice to also be able to set all those things in the UI. But we’d need to make that conceptual change first.

[andrewbaldwin44]

The idea behind this is that in the real world, the web UI is hosted, and then it makes a lot less sense to keep having to change the locust file and redeploy.

For me, there is a strong use case behind selecting the tasks that a user is ran with and the users weight (or fixed count). Depending on the requirements, we may want to test different tasks at different times, and there's currently no way to support this without commenting things out in the locust file and redeploying.

[cyberw]

"in the real world"? :)

I have never run locust in a "hosted" fashion , and I have almost never seen anyone else doing it. But I guess people can use it in a lot of different ways :) I find it more useful to set everything on command line (but I sometimes do miss the ability to set wait times and weights from there)

Btw, when you say "redeploying", is that something that is complicated for you? I just use locust-swarm for that (I've had some discussions with @DennisKrone about maybe distributing the locustfile from master->worker at startup, but it has not yet come to fruition - perhaps that would make it easier)

[andrewbaldwin44]

Well clearly I made some false assumptions about the "real world" :)

I thought the whole point of having the web UI and having a docker image was to deploy. I use the web UI to set everything about how I want the test to run. That's why something like this is useful to me. I want to run certain tasks one day and other tasks another day.

It's not that it's difficult to redeploy, it's the it doesn't make sense to change things and redeploy when I'll just need to change them back.

I had a very simple time deploying a kubernetes cluster for distributed load testing. There just needs to be a different env variable for the workers and the master and then a bash file runs different commands based on that env variable.

[cyberw]

Aha, interesting... do you have one big locustfile with lots of User definitions that you rarely change or what?

I dont mind adding features that make it possible to configure more stuff in the UI per se, but I prefer to make "redeploying" easier.

It should ideally be a single command and take <10 seconds (like it is with swarm today and maybe what it could be with automatically distributed locustfiles when using docker). If it is that fast then having access to configuration in the web ui is not needed because changing the locustfile is almost as fast.

But if you want to build it then I'm not completely against it, as long as it doesnt add a ton of complexity and is not a UI-only feature (the same settings should also be configurable on command line or using some sort of config file).

[cyberw]

I had a very simple time deploying a kubernetes cluster for distributed load testing. There just needs to be a different env variable for the workers and the master and then a bash file runs different commands based on that env variable.

That sounds interesting, but I dont understand exactly what you mean :) Can you share some more details on this? Could be good us as well (right now we use dedicated machines for load generation but would probably move to kubernetes if we could get a good enough solution working)

[andrewbaldwin44]

Yeah it's exactly that, and I have a whole bunch of tasks that I can toggle on or off depending on what I want to test

The reason I prefer having control over what tasks get run is because I think commenting out tasks in the locust file and redeploying can't really be considered a feature :) Not to mention this is more explicit about what is being ran, and it even allows for changing the configuration in the edit modal mid test run

It would be a very simple change for the UI, it would only involve exposing a new route and then updating the user. Something like:

user_class_name = request.args.get("user_class")
user_class = environment.available_user_classes[user_class_name]

 user_class.fixed_count = int(request.args.get("fixed_count", 0))

(the only slight complexity is we need to communicate with the workers in distributed mode)

I think if we wanted something like this to be configurable on the command line it could be done with some config file or inline config:

--user-config '{ "user_1": {"fixed_count": 1, "host": "http://localhost"}, "user_2": {"fixed_count": 3, "host": "http://localhost:5000"}}'

[andrewbaldwin44]

Yeah I have a Dockerfile that sets the entry point as a bash file:

FROM locustio/locust:2.16.1

ADD locust locust
ADD requirements.txt .
ADD run.sh .

RUN pip install -r requirements.txt

User root

# Set script to be executable
RUN chmod 755 run.sh

User locust

ENTRYPOINT ["bash", "./run.sh"]

The bash file runs locust based on the LOCUST_MODE env variable:

LOCUST="locust"
LOCUS_OPTS="-f ./locust/main.py --class-picker"
LOCUST_MODE=${LOCUST_MODE}

if [[ "$LOCUST_MODE" = "master" ]]; then
    LOCUS_OPTS="$LOCUS_OPTS --master --master-port=$LOCUST_MASTER_PORT"

elif [[ "$LOCUST_MODE" = "worker" ]]; then
    LOCUS_OPTS="$LOCUS_OPTS --worker --master-port=$LOCUST_MASTER_PORT --master-host=$LOCUST_MASTER_URL"
fi

$LOCUST $LOCUS_OPTS

Then you just create a Kubernetes deployment. One node just needs to have the env variable LOCUST_MODE=master. The all the workers have LOCUST_MODE=worker and the LOCUST_MASTER_URL

There's a pretty good article on this

The time it takes to deploy and the commands to deploy depend on where it is hosted right? :)

[cyberw]

Yea, commenting stuff out sucks, but disabling/enabling whole user classes already supported with the class picker, right?

Ok! If it is easy to do then go ahead and try it out!

[cyberw]

Yeah I have a Dockerfile that sets the entry point as a bash file:

Aha, interesting. But every time you need to update the locustfile you need to build and push the docker image? I dont like that workflow :)

[andrewbaldwin44]

Yeah exactly, but in an ideal world you would have no dependencies to install. You would just copy over your locust file and then the build would happen really quickly

[andrewbaldwin44]

Locust Swarm looks pretty interesting, I wasn't previously aware of this. But then you don't get a pretty web UI eh :)

[cyberw]

Sure, and the docker build cache should handle that. But it is still build + push + deploy. With swarm its all just one command :)

Actually swarm works nicely with the UI too, it just defaults to headless. You just need to pass the somewhat awkwardly named parameter --headful :)

[andrewbaldwin44]

Exactly, and anything can be a single command with a little bit of imagination :) Plus I would just have a Github action handle the build / push

Sounds pretty interesting, I'll have to try it. We really should have some documentation about all these plugins :)

[cyberw]

Yea, some day I want to merge swarm into locust. It is mentioned in the docs but people tend to miss it :)

Or maybe not. Maybe just add the docs to locust-core, so people find it.