Why use manual tensor parallelism implementation instead of using something like deepspeed?

[vikigenius]

I know it could have just been a design decision, but I would love to hear your rationale behind why you rolled your own implementation of model parallelism over using something like deepspeed as a dependency.

[zhuohan123]

Zero and FSDP communicate weights instead of activations. In inference, activations are small but weights are large. Therefore we choose tensor parallelism for efficiency.

[qingquansong]

For offline inference with prefill only case（such as embedding generation），this tp setting seems to be worse than deepspeed and fsdp.