Pulling the model from dual SSDs at once - small speedup

[tim-lab-57]

I realized that I had two NVMe SSDs, so I wondered if I was disk-bound, and had my local qwen3.6:27b attempt a change to pull from both drives at once, alternating between one and the next.

Before: ~1.0 tok/s
After: ~1.2 tok/s

Is this at all useful to anyone else? 😂 (if so, I can try polish and then submit a pull request)

Here's my current settings, plus the two new settings qwen made:
Optimized_GPU_MTP_web_chat.ps1.txt

Hardware:
AMD Ryzen Threadripper Pro 9965WX (24-core)
128 GB RAM
NVIDIA RTX PRO 6000 (blackwell) Max-Q edition

Windows 11 native, no WSL.

[Magnet-js]

Hello, I've got a quick question. I'm not an expert, but qwen3.6:27b seems really slow at 1 token per second on a Threadripper Pro with 128 GB and an RTX PRO 6000.

Normally, I would expect more than one token per second.
Could you explain this, just out of interest?

[uvreflective4]

I think he meant that he used qwen3.6:27b to modify the codebase, but the 1 tok/sec is from GLM 5.2.

[Magnet-js]

That make sense, thank you 🙏

[tim-lab-57]

Yes, that's what I meant, sorry for the confusion!

[bigjeff5]

This is almost certainly better handled by configuring your two drives as a RAID array in hardware than by attempting to do this in software. You should be seeing a closer to a 2x improvement, not 1.2x, which just shows how inefficient it is to try to implement this with both the OS and the disc controller getting in your way. Even software RAID should see a better than 20% improvement, IMO.

[DavidePapero]

Hardware: AMD Ryzen Threadripper Pro 9965WX (24-core) 128 GB RAM NVIDIA RTX PRO 6000 (blackwell) Max-Q edition

Windows 11 native, no WSL.

Could you do an I/O benchmark using one disk and using two in parallel? It's true that a hardware raid 0 would be ideal but the performance increase seems really low to me.