Skip to content

Commit

Permalink
Try spacing maybe?
Browse files Browse the repository at this point in the history
  • Loading branch information
muellerzr committed Feb 6, 2024
1 parent 31fcde4 commit 7c3d183
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion docs/source/usage_guides/distributed_inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,11 @@ model = prepare_pippy(model, example_args=(input,))
<Tip>

There are a variety of parameters you can pass through to `prepare_pippy`:
* `split_points` lets you determine what layers to split the model at. By default we use wherever `device_map="auto" declares, such as `fc` or `conv1`.

* `split_points` lets you determine what layers to split the model at. By default we use wherever `device_map="auto" declares, such as `fc` or `conv1`.

* `num_chunks` determines how the batch will be split and sent to the model itself (so `num_chunks=1` with four split points/four GPUs will have a naive MP where a single input gets passed between the four layer split points)

</Tip>

From here, all that's left is to actually perform the distributed inference!
Expand Down

0 comments on commit 7c3d183

Please sign in to comment.