Avoid graph break by removing redundant requires_grad attr change#7158
Merged
loadams merged 5 commits intodeepspeedai:masterfrom Mar 24, 2025
Merged
Avoid graph break by removing redundant requires_grad attr change#7158loadams merged 5 commits intodeepspeedai:masterfrom
loadams merged 5 commits intodeepspeedai:masterfrom
Conversation
This PR is a continuation of the efforts to improve Deepspeed performance when using PyTorch compile. Dynamo breaks the graph because flat_tensor.requires_grad = False * Is a side-effecting operation on tensor metadata * Occurs in a context where Dynamo expects static tensor properties for tracing flat_tensor.requires_grad is redundant and can be safely removed because: * _allgather_params function is already decorated with @torch.no_grad() which ensures the desired property * flat_tensor is created using the torch.empty(..) which sets the requires_grad=False by default. Signed-off-by: Max Kovalenko <mkovalenko@habana.ai>
This reverts commit 11612773b3d68aa5b8d72bad1de4b1714ea1193a. Signed-off-by: Max Kovalenko <mkovalenko@habana.ai>
tohtana
approved these changes
Mar 20, 2025
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
loadams
added a commit
that referenced
this pull request
Mar 25, 2025
) This PR is a continuation of the efforts to improve DeepSpeed performance when using PyTorch compile. Dynamo breaks the graph because `flat_tensor.requires_grad = False`: * Is a side-effecting operation on tensor metadata * Occurs in a context where Dynamo expects static tensor properties for tracing `flat_tensor.requires_grad` is redundant and can be safely removed because: * `_allgather_params()` function is already decorated with `@torch.no_grad()` which ensures the desired property * `flat_tensor` is created using the `torch.empty()` which sets the `requires_grad=False` by default. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com> Signed-off-by: Logan Adams <loadams@microsoft.com>
mauryaavinash95
pushed a commit
to DataStates/DeepSpeed
that referenced
this pull request
Mar 28, 2025
…epspeedai#7158) This PR is a continuation of the efforts to improve DeepSpeed performance when using PyTorch compile. Dynamo breaks the graph because `flat_tensor.requires_grad = False`: * Is a side-effecting operation on tensor metadata * Occurs in a context where Dynamo expects static tensor properties for tracing `flat_tensor.requires_grad` is redundant and can be safely removed because: * `_allgather_params()` function is already decorated with `@torch.no_grad()` which ensures the desired property * `flat_tensor` is created using the `torch.empty()` which sets the `requires_grad=False` by default. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com>
deepcharm
added a commit
to deepcharm/DeepSpeed
that referenced
this pull request
Apr 29, 2025
This PR is an follow-up to PR (deepspeedai#7158) handling the same issue. Signed-off-by: Max Kovalenko <mkovalenko@habana.ai>
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 19, 2025
…7263) This PR is an follow-up to [PR #7158](#7158) handling the same issue in another place. See [PR #7158](#7158) for details. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com>
ys950902
pushed a commit
to ys950902/DeepSpeed
that referenced
this pull request
May 21, 2025
…epspeedai#7158) This PR is a continuation of the efforts to improve DeepSpeed performance when using PyTorch compile. Dynamo breaks the graph because `flat_tensor.requires_grad = False`: * Is a side-effecting operation on tensor metadata * Occurs in a context where Dynamo expects static tensor properties for tracing `flat_tensor.requires_grad` is redundant and can be safely removed because: * `_allgather_params()` function is already decorated with `@torch.no_grad()` which ensures the desired property * `flat_tensor` is created using the `torch.empty()` which sets the `requires_grad=False` by default. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com> Signed-off-by: yisheng <yi.sheng@intel.com>
deepcharm
added a commit
to deepcharm/DeepSpeed
that referenced
this pull request
Jun 16, 2025
…eepspeedai#7263) This PR is an follow-up to [PR deepspeedai#7158](deepspeedai#7158) handling the same issue in another place. See [PR deepspeedai#7158](deepspeedai#7158) for details. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com> Signed-off-by: Max Kovalenko <mkovalenko@habana.ai>
mauryaavinash95
pushed a commit
to DataStates/DeepSpeed
that referenced
this pull request
Oct 4, 2025
…eepspeedai#7263) This PR is an follow-up to [PR deepspeedai#7158](deepspeedai#7158) handling the same issue in another place. See [PR deepspeedai#7158](deepspeedai#7158) for details. --------- Signed-off-by: Max Kovalenko <mkovalenko@habana.ai> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a continuation of the efforts to improve DeepSpeed performance when using PyTorch compile.
Dynamo breaks the graph because
flat_tensor.requires_grad = False:flat_tensor.requires_gradis redundant and can be safely removed because:_allgather_params()function is already decorated with@torch.no_grad()which ensures the desired propertyflat_tensoris created using thetorch.empty()which sets therequires_grad=Falseby default.