Join GitHub today
test item scheduling does not schedule N items to N workers #40
I may just be misunderstanding how xdist works, but I can't seem to get the number of workers I requested to all accept tasks in parallel:
Watching the output as it comes, it looks like the second block of 4 tests doesn't start running until the first block of 4 completes.
Am I misunderstanding something? Why don't all 8 workers start running tests at the same time? Why doesn't
it clearly starts 8 workers, however due to a design problem thats sheduled to be fixed in the next bigger sprint it distributes tests in batches of 2 items, and thus sheduling is rather subpar
for your own case, the problem will elevae as the number of tests gows tho
AFAIR this problem was noted on the py.test issue tracker already
I only have 2 virtual cores. I don't think this is normal, as @RonnyPfannschmidt's comment confirms. I expect the 8 workers to all start accepting tasks at the same time. This should have nothing to do with my physical cores.
Of course, if I run 8 tasks on a machine with 2 cores, that means there will be a lot of context switching between the tasks. That's fine with me since my tests are I/O-heavy.
Do you have the py.test issue number?
Going forward, should xdist issues be reported here or on the main py.test issue tracker? I only searched here before reporting this issue. Apologies if it's a dup. I think it would be better if xdist issues were tracked exclusively in the xdist repo.