Ddp2 fix #448
Merged
Ddp2 fix #448
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Borda
reviewed
Nov 3, 2019
| return loss, dict with metrics for tqdm | ||
| :param called with batch, batch_nb | ||
| additional: optimizer_i if multiple optimizers used | ||
| :return: |
Collaborator
There was a problem hiding this comment.
what s the return? if there is none, drop this like
| Must return model. | ||
| :param model: | ||
| :param device_ids: | ||
| :return: |
| output[k] = self.reduce_distributed_output(output[k], nb_gpus) | ||
|
|
||
| # do nothing when there's a scalar | ||
| elif isinstance(output[k], torch.Tensor) and output[k].dim() == 0: |
Collaborator
There was a problem hiding this comment.
in such case, you can skip this branching condition, right?
Contributor
Author
|
gpu tests passed. waiting on circle CI |
Contributor
|
It's probably too late, but it occurs to me that since DDP and AMP have nothing to do with the actual research code, it might be better to pass them as callbacks or something rather than bundling them with the model. Maybe something to think about for the future. |
Collaborator
|
Do we have a CircleCI? I do not see its config in the repo master... @williamFalcon |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR does the following: