Avoid long task lists #1599
Avoid long task lists #1599
Conversation
|
Another thing I'd like to see is to ensure that the batch size is a multiple of the worker concurrency. For instance on the SMP VLTC tests, where I have a concurrency of 3, assigning a batch size of 7 or 8 is intensely silly and wasteful, not to mention the usual concerns about half-idle batches affecting frequency/hyperthreads and thus altering the effective TC. |
|
doesn't matter if the batch size is sufficiently long. The variations in game lengths are much larger. |
|
Applied by hand on PROD. |
|
On the other hand, as I stated, there are some cases where the batch size is not sufficiently long. |
|
This PR does have an effect on the itp balancing, significantly increasing the latency of response to changing itp, but that should be a small effect outweighed by the benefit to the server. But we should keep an eye on it. |
so, this works, larger tasks are being generated. Right now, as the tests have partially already >1000 tasks, the tasks generated are exceptionally large. For new tests, that will essentially never have that many tasks, the behavior will be OK. |
|
How about the 800,000 limit, it was preventing more cores on some jobs with this set to 1000. Is there anything we can do about that? |
|
I think it might need a bit of a refinement. The quadratic scaling with task number gets too much, tasks get very large. So, I'll damp that down a bit. It will allow more workers to join a task. The 800k, maybe we can indeed increase it a bit. Let me have a look. |
they make some operations in fishtest much slower, and the db bigger. Additionally larger batches means less idle time at the end of the batch for workers, which is good.
Let me know if there is a change to be reloaded on PROD (or this was only a squash) |
|
This is somewhat different code. It doesn't need testing on PROD, but it is different from what we had on PROD before. |
|
so, based now on the experience gathered, I limit the growth of the batch size with task_id to a maximum of 5. This will help when many workers land to keep tasks available. |
|
PROD updated, thank you @vondele :) |
they make some operations in fishtest much slower, and the db bigger. Additionally larger batches means less idle time at the end of the batch for workers, which is good.