[Sigmoid][Delta Update][2/N] update delta update api to load original value first before casting to target dtype #167039
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167039
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 0c641c1 with merge base 9eebda9 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
henryoier
approved these changes
Nov 5, 2025
jijunyan
force-pushed
the
export-D86181685
branch
from
5aeacd5 to
5e531c4
Compare
jijunyan
added a commit
to jijunyan/pytorch
that referenced
this pull request
Nov 5, 2025
… value first before casting to target dtype (pytorch#167039) Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype. Test Plan: No more NaN values! {P2022339213} Reviewed By: henryoier, kqfu Differential Revision: D86181685
jijunyan
force-pushed
the
export-D86181685
branch
from
5e531c4 to
2222124
Compare
pytorch-bot bot
pushed a commit
that referenced
this pull request
Nov 6, 2025
… value first before casting to target dtype (#167039) Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype. Test Plan: No more NaN values! {P2025779492} Unit tests: buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_AOTInductorCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_InterpreterCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.TestLoadDeltaUpdates' 2>&1 | tee re_local_benchmark_log.txt Reviewed By: henryoier, kqfu Differential Revision: D86181685
|
@pytorchbot label "topic: not user facing" |
jijunyan
force-pushed
the
export-D86181685
branch
from
2222124 to
864f9c4
Compare
jijunyan
added a commit
to jijunyan/pytorch
that referenced
this pull request
Nov 6, 2025
… value first before casting to target dtype (pytorch#167039) Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype. Test Plan: No more NaN values! {P2025779492} Unit tests: buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_AOTInductorCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_InterpreterCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.TestLoadDeltaUpdates' 2>&1 | tee re_local_benchmark_log.txt Reviewed By: henryoier, kqfu Differential Revision: D86181685
… value first before casting to target dtype (pytorch#167039) Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype. Test Plan: No more NaN values! {P2025779492} Unit tests: buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_AOTInductorCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.Basic_InterpreterCuda_SwapWeights' 2>&1 | tee re_local_benchmark_log.txt buck test fbcode//mode/dev-nosan -c fbcode.nvcc_arch=a100,h100 fbcode//sigmoid/inference/test_gpu:model_runner_test -- --exact 'sigmoid/inference/test_gpu:model_runner_test - ModelRunnerTest.TestLoadDeltaUpdates' 2>&1 | tee re_local_benchmark_log.txt Reviewed By: henryoier, kqfu Differential Revision: D86181685
jijunyan
force-pushed
the
export-D86181685
branch
from
864f9c4 to
0c641c1
Compare
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
yarongmu-google
pushed a commit
to yarongmu-google/pytorch
that referenced
this pull request
Nov 7, 2025
… value first before casting to target dtype (pytorch#167039) Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype. Test Plan: No more NaN values! {P2022339213} Reviewed By: kqfu Differential Revision: D86181685 Pull Request resolved: pytorch#167039 Approved by: https://github.com/henryoier
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: The current delta update has a strong assumption that the non-lowered weights share the same tensor dtype from the lowered version. This is not true by design. When dtype mismatches the data loading will load the data into unexpected dtype which introduces undefined behavior. This diff aims to close the gap by always load tensor by its original dtype first then cast to desired dtype.
Test Plan:
No more NaN values!
{P2022339213}
Reviewed By: kqfu
Differential Revision: D86181685