fixed_count: ability to spawn a specific number of users (as opposed to just using weights) #1964

EzR1d3r · 2021-12-25T23:20:45Z

Issue #1939
Here is the overall working option, but noticed a problem with the display 'Ratio per User class' and 'Total ratio' on web UI.
I'm going to redo the logic a little bit, get_task_ratio_dict it will recalculate the ratio based on the number of users at the moment. I'll make it a separate commit in same branch. Maybe there are other ideas?

cyberw · 2021-12-26T08:21:53Z

We should probably mention it somewhere in the documentation.

@mboutet knows most about dispatching - what do you think?

EzR1d3r · 2021-12-29T22:55:01Z

I made the changes about which I wrote, it seems to me that it turned out quite appropriate. I made two separate functions for counting the ratio of classes and tasks, as can be seen from unit tests, the results of the calculation remained the same, only a little changed approach.

EzR1d3r · 2021-12-30T12:10:00Z

@cyberw Should I do something else or request ready to review now?

cyberw · 2021-12-30T16:13:08Z

Add a mention at some appropriate point in the docs (.rst files). I’m hoping @mboutet will review so I dont have to :)

mboutet · 2021-12-30T16:14:57Z

I'll try to look at this first week of January. Sorry for the delay.

locust/test/test_dispatch.py

cyberw · 2022-01-01T20:50:02Z

locust/test/test_dispatch.py

+
+        self.case_handler(
+            cases=[
+                self.Case(fixed_counts=(1, 1), weights=(1, 1, 1), target_user_count=5),


Do the tests validat the order in which users are spawned or only final distribution?

Only final distribution. Since the same algorithm is used as for weighted users, and it is tested above.

cyberw · 2022-01-01T20:52:54Z

locust/test/test_dispatch.py

+                                    user_class.fixed_count,
+                                )
+
+


Add an ”integration test” in test_main to test some basic functionality using headless.

I've looked into test_main and a little bit confused, what exactly you want to test? Basic spawning with fixed users? ie check the output is ok and corresponds to the users data?

locust/test/test_dispatch.py

mboutet · 2022-01-06T23:45:03Z

I think adding an integration test similar to

locust/locust/test/test_runners.py

Line 957 in cce0d29

def test_distributed_shape(self):

would be good.

mboutet · 2022-01-06T23:48:17Z

locust/dispatch.py

+        self._try_dispatch_fixed = True
+
+        self._no_user_to_spawn = False


Do you also need to set these in

locust/locust/dispatch.py

Line 145 in cce0d29

def new_dispatch(self, target_user_count: int, spawn_rate: float) -> None:

?

I guess no, self._no_user_to_spawn resets to False almost immediately after checking in _dispatcher
self._try_dispatch_fixed will be True if we stop to spawn somewhere on fixed users, so when we continue it is fine. If we spawned all fixed and it is False, it is also ok. The only case we need to reset it is ramp-down. Also I guess nothing changes if to set this fields, except a superfluous work in _try_dispatch_fixed if it was False. But I do not want to mislead about such a need.

mboutet · 2022-01-06T23:50:39Z

Can you also add a small stress test like

locust/locust/test/test_dispatch.py

Line 1934 in cce0d29

class TestLargeScale(unittest.TestCase):

? This is just to make sure there's no significant performance regression.

locust/dispatch.py

locust/test/test_dispatch.py

mboutet · 2022-01-07T00:12:19Z

I'll have to take another look later, but it looks good at first glance. The unit tests you added in test_dispatch.py seem to cover the new functionality adequately 👍🏼.

EzR1d3r · 2022-01-13T21:26:31Z

I made some fixups, also will add the tests @mboutet asked.

locust/test/test_dispatch.py

mboutet · 2022-01-18T02:10:56Z

@EzR1d3r, the PR looks good. The tests in test_dispath.py gives me confidence that the feature works as intended. Also, all the existing tests pass, so there should not be any regression 👍🏼.

I also tested your changes with the scenario in scripts/locustfile.py (I used UserA.fixed_count = 10) and the script in scripts/run-distributed-web.sh (13 workers). It works great!

After the last comments are resolved, I think this will be good to go.

EzR1d3r · 2022-01-18T22:47:02Z

It was a good idea to add TestLargeScale, so I found a bug with _distribute_users function and I had to remake the realisation of fixed users spawning. Within the _distribute_users iterating over the user_gen not add the users in _users_on_workers instantly and we cant monitore the count of the fixed users. Also I a little bit reduced the test code and added my test into existed. This changes in the new last commit.

EzR1d3r · 2022-01-18T23:56:15Z

I see the tests fail. But my locals tests are ok.
self.assertIn("Shutting down (exit code 0), bye.", output)

upd.: in some reason, is missed the tail of "Shutting down (exit code 0), bye."

cyberw · 2022-01-19T22:38:58Z

I see the tests fail. But my locals tests are ok. self.assertIn("Shutting down (exit code 0), bye.", output)

upd.: in some reason, is missed the tail of "Shutting down (exit code 0), bye.

I recently removed the ”, bye.” in master. Do thr same in your test and you should be fine. Worthless timing by me…

for spawning of each class with <fixed_count> class field. Fixed users spawn first. The weight parameter for them is ignored. Also added unit-tests for cases with fixed users.

cyberw · 2022-01-20T06:38:39Z

Looks pretty good to me.

Just needs an entry somewhere in the documentation (.rst files). One question, why did you add the _actual in the method names? Is that the right word? What would be the ”not actual” ratio? Maybe _current is more appropriate? (on mobile right now, sorry if it would have become obvious by reading the whole PR at once)

Anything left to do in your view, @mboutet ?

EzR1d3r · 2022-01-20T08:29:27Z

Looks pretty good to me.

Just needs an entry somewhere in the documentation (.rst files).

I will look.

One question, why did you add the _actual in the method names? Is that the right word? What would be the ”not actual” ratio? Maybe _current is more appropriate? (on mobile right now, sorry if it would have become obvious by reading the whole PR at once)

Anything left to do in your view, @mboutet ?

I guess, you right. 'Actual' in relation to previous 'static' behaviour. But if you dont know the previous, it has no sense, right?
I can change to 'current' or smth better or may be just remove at all?

cyberw · 2022-01-20T09:26:21Z

I guess, you right. 'Actual' in relation to previous 'static' behaviour. But if you dont know the previous, it has no sense, right? I can change to 'current' or smth better or may be just remove at all?

Remove sounds good!

…atio, now it is considered based on the real number of users. On the Task tab (web ui) the data is updated every 1 second, so it is possible to see the actual ratio changing. For command-line arguments --show-task-ratio, --show-task-ratio-json, the behavior is also changed - the ratio pre-calculation based on passed num_users. If there is no fixed_count users and num_users argument is None, the old behaviour occurs.

EzR1d3r · 2022-01-20T20:58:10Z

Added some docs (last commit) and removed 'actual'.
I've increased test_distribute_users time limit to 7 seconds - the last time some tests(py8 and py10) failed at this point, but I've changed nothing here.

Because in case then UserDispatcher._users_on_workers is not fills instantly (see UserDispatcher._distribute_users) we cant to monitore actual count of each user. Also added some additional tests, includes to check this behaviour.

…ation.

mboutet · 2022-01-20T23:36:16Z

@cyberw, this looks good!

@EzR1d3r, good job on that feature!

cyberw · 2022-01-21T08:46:07Z

Awesome stuff, thanks! This will automatically become part of the next prerelease build, and I'll do a real release in a couple days.

EzR1d3r · 2022-01-21T09:54:22Z

Thx you too for review and accepting this feature!

domik82 · 2022-02-08T09:06:15Z

@EzR1d3r thanks for wrapping it up.
@cyberw @mboutet - I guess work we did on user distribution controlled by the master is paying off.

Just to confirm the functionality we described here: #1583 should work now?

mboutet · 2022-02-08T15:47:55Z

I don't believe independant load test shapes has been implemented.

cyberw · 2022-02-08T19:57:49Z

No, independent load shapes is not implemented. That ticket is closed because it was stale. Dont know why it was referenced..

I guess in some cases fixed user count helps a little.

domik82 · 2022-02-10T16:25:40Z

I guess I assumed wrongly scope of this feature.

EzR1d3r force-pushed the issue-1939 branch from 2e9e4e1 to 8007d44 Compare December 25, 2021 23:21

EzR1d3r mentioned this pull request Dec 25, 2021

Possibility to set the exact number of users to spawn (instead weight) #1939

Closed

cyberw changed the title ~~Issue #1939~~ fixed_count: ability to spawn a specific number of users (as opposed to just using weights) Dec 28, 2021

cyberw reviewed Jan 1, 2022

View reviewed changes

EzR1d3r force-pushed the issue-1939 branch from 6fefd74 to 71d12cc Compare January 5, 2022 19:26

mboutet reviewed Jan 6, 2022

View reviewed changes