Skip to content

Commit

Permalink
Describe what "partition" means
Browse files Browse the repository at this point in the history
  • Loading branch information
sublee authored and GitHub Enterprise committed Oct 21, 2019
1 parent 2f0001e commit c7391bf
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions docs/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,13 @@ Applying GPipe
~~~~~~~~~~~~~~

To train a module with GPipe, simply wrap it with :class:`torchgpipe.GPipe`.
Your module must be :class:`nn.Sequential <torch.nn.Sequential>` as GPipe will
automatically split the module into partitions with consecutive layers.
`balance` argument determines the number of layers in each partition. `chunks`
argument specifies the number of micro-batches. Input, output, and intermediate
tensors must be ``Tensor`` or ``Tuple[Tensor, ...]``. See also `Restrictions`_
for more details.
Your module must be a :class:`nn.Sequential <torch.nn.Sequential>` as GPipe
will automatically split the module into partitions. A partition is a group of
consecutive layers that run on a single device together. `balance` argument
determines the number of layers in each partition. `chunks` argument specifies
the number of micro-batches. Input, output, and intermediate tensors must be
``Tensor`` or ``Tuple[Tensor, ...]``. See also `Restrictions`_ for more
details.

The below example code shows how to split a module with four layers into two
partitions each having two layers. This code also splits a mini-batch into 8
Expand Down

0 comments on commit c7391bf

Please sign in to comment.