Skip to content

[parallel doc] dealing with layers larger than one gpu#13980

Merged
stas00 merged 1 commit intohuggingface:masterfrom
stas00:docs-parallel
Oct 12, 2021
Merged

[parallel doc] dealing with layers larger than one gpu#13980
stas00 merged 1 commit intohuggingface:masterfrom
stas00:docs-parallel

Conversation

@stas00
Copy link
Copy Markdown
Contributor

@stas00 stas00 commented Oct 12, 2021

This PR expands on the parallelism doc:

  • explains what to do when a layer is larger than one gpu
  • adds some more notes

@sgugger

Copy link
Copy Markdown
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding that bit!

@stas00 stas00 merged commit 61f6426 into huggingface:master Oct 12, 2021
@stas00 stas00 deleted the docs-parallel branch October 12, 2021 22:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants