Pipeline parallel not supported in benchmark?

[forrestjgq]

I built a model with tp 2 pp 2, that's world size 4, and benchmark it. It wars reported that expected world size is 2, and I check the code:
https://github.com/NVIDIA/TensorRT-LLM/blob/release/0.5.0/benchmarks/python/base_benchmark.py#L70C30-L70C30

It seems pipeline parallel is not supported in benchmark yet?

[jdemouth-nvidia]

PP support has been really enabled only for the C++ backend. There’s a “naive” implementation in Python to help us debug but it should not be used for performance studies.

[joostinyi]

Hello, is PP support roadmapped for the python benchmarks?

[jdemouth-nvidia]

For now, the Python runtime contains a simple but not efficient implementation of PP. It's been implemented only for debugging purposes. We do not plan to update it in a near future. We are working on wrappers of the C++ using Python to offer a fast implementation to our Python users.