Model EVO-2 takes a long time for generating moderately long sequences with Vortex (e.g. 16,384)

[shanikawm]

Model take a long time for generating moderately long sequences. How can I make use of multiple GPUs for this case?
from torch.nn.parallel import DistributedDataParallel also didn't help.

12 Minutes for generating 16,384 long sequence.

Is there any way to improve this using multiple GPUs?

[garykbrixi]

Hi @shanikawm, thanks for your interest. Currently, Vortex implements automatic depth-wise model partitioning across available GPUs. For optimal performance, we recommend using the minimum GPU count required to accommodate your model size and sequence length constraints. At present, alternative parallelization strategies (e.g., tensor, pipeline parallelism) are not supported but we welcome contributions to vortex in this direction.

Additionally, note that the BioNeMo team is incorporating Evo 2 generation capabilities to BioNeMo, which should include support for multiple parallelization strategies

[shanikawm]

Hi @garykbrixi noted and thanks for the information.