Can we use `torch.nn.DataParallel ` to speed encode_image ?

[mmmwhy]

It seems we cannot use encode_image in DataParallel

[jongwook]

This is because DataParallel works by overriding the forward() method to launch workload in parallel. The submodule model.visual contains the vision model, and I expect it to work if you wrap model.visual instead of model in nn.DataParallel.

Another workaround would be to define an nn.Module that has the CLIP module as its submodule and forwards the call to the CLIP module's encode_image().