Add a tutorial for distributed inference

[awaelchli]

🚀 Feature

Users are asking for examples how to predict with models in a distributed setting.

Motivation

We could link such a tutorial in the PL main docs.

Pitch

Add tutorial page for prediction on single GPU, multiple GPU / multiple nodes.
It should cover the PredictionWriterCallback and how to use it.

Alternatives

Additional context

Related PR #52

[gianscarpe]

Just pointing out a useful and elegant solution to multi-gpu prediction (does not involve creating new files)
https://github.com/open-mmlab/mmdetection/blob/482f60fe55c364e50e4fc4b50893a25d8cc261b0/mmdet/apis/test.py#L160

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[allanchan339]

@awaelchli
Any update on how to gather all validation_step_outputs to a single device {rank 0} at validation_epoch_end for further metrics calculation? I have redirected from discussion Lightning-AI/pytorch-lightning#5788 to here.