CUDA MPS

[adayton1]

We have over 1000 tiny regression tests. Because we currently have to do one rank per GPU, each GPU is extremely underutilized, and our throughput is terrible. I've heard about MPS allowing multiple ranks to share a GPU, and this sounds like it would improve our throughput tremendously. Have there been any plans to add MPS support to ATS?

[dawson6]

Hi @adayton1 That is a good suggestion. We have not put it in yet, but I understand how it would help.

We are currently working on a Flux scheduler, as that does seem to be the future. I can make a note of that request and see if we can somehow specify it when using Flux.

[adayton1]

Thanks, @dawson6! We are definitely going to need this at some point (our test suite is already too big to run all of it without MPS), so if you can work it in to the plan, that would be great!