[Question] P40 support (best cost effective hardware)

[kripper]

❓ General Questions

What's the performance of the P40 using mlc-llm + CUDA?

mlc-llm is the fastest inference engine, since it compiles the LLM taking advantage of hardware specific optimizations, and the P40 is the best cost effective hardware.

This P40 has 3480 CUDA cores:
https://resources.nvidia.com/en-us-virtualization-and-gpus/p40-datasheet

Did you have difficulties using P40 via CUDA?

[Hzfengsy]

I think it should work if you turn off flashinfer and cutlass support. However, we do not have resource to optimize for such old device.

[Nero10578]

I think it should work if you turn off flashinfer and cutlass support. However, we do not have resource to optimize for such old device.

If it is reported to work on other Pascal generation cards like the GTX 1060 etc. Then it should work on the P40 right?

[tqchen]

we have seen seevral examples working on older cards, likely we just need to turnoff flash infer, cutlass, and also follow instruction to build tvm from source