Add cache update function for delta in-place update #1436
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for pytorch-fbgemm-docs canceled.
|
|
This pull request was exported from Phabricator. Differential Revision: D39158010 |
jianyuh
added a commit
to jianyuh/FBGEMM
that referenced
this pull request
Nov 2, 2022
Summary: Pull Request resolved: pytorch#1436 With inplace update, we need to update dev_weights, uvm_weights, as well as the cache_weights (note that during inference, cache is read-only; but in the inplace delta update, cache needs to be updated. Since we don't have "write back"/flushing cache (when inference finishes, or cache evicts) during inference, we need to update cache weight **and** the back storage (uvm weight) simultaneously during the inplace delta update (thanks yinghai for pointing this out!). Reviewed By: jspark1105 Differential Revision: D39158010 fbshipit-source-id: 3a50922640c7d511b5a6e0ddb443943c32d406af
jianyuh
force-pushed
the
export-D39158010
branch
from
1253233
to
d288505
Compare
|
This pull request was exported from Phabricator. Differential Revision: D39158010 |
jianyuh
added a commit
to jianyuh/FBGEMM
that referenced
this pull request
Nov 2, 2022
Summary: Pull Request resolved: pytorch#1436 With inplace update, we need to update dev_weights, uvm_weights, as well as the cache_weights (note that during inference, cache is read-only; but in the inplace delta update, cache needs to be updated. Since we don't have "write back"/flushing cache (when inference finishes, or cache evicts) during inference, we need to update cache weight **and** the back storage (uvm weight) simultaneously during the inplace delta update (thanks yinghai for pointing this out!). Reviewed By: jspark1105 Differential Revision: D39158010 fbshipit-source-id: d8ad76f420bd9beba724882bf363c8aeb83f86fa
jianyuh
force-pushed
the
export-D39158010
branch
from
d288505
to
de8f91a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D39158010 |
jianyuh
added a commit
to jianyuh/FBGEMM
that referenced
this pull request
Nov 2, 2022
Summary: Pull Request resolved: pytorch#1436 With inplace update, we need to update dev_weights, uvm_weights, as well as the cache_weights (note that during inference, cache is read-only; but in the inplace delta update, cache needs to be updated. Since we don't have "write back"/flushing cache (when inference finishes, or cache evicts) during inference, we need to update cache weight **and** the back storage (uvm weight) simultaneously during the inplace delta update (thanks yinghai for pointing this out!). Reviewed By: jspark1105 Differential Revision: D39158010 fbshipit-source-id: 2da0f5042062bce871dfed34a94359dd982e2fec
jianyuh
force-pushed
the
export-D39158010
branch
from
de8f91a
to
2f0d320
Compare
|
This pull request was exported from Phabricator. Differential Revision: D39158010 |
Summary: Pull Request resolved: pytorch#1436 With inplace update, we need to update dev_weights, uvm_weights, as well as the cache_weights (note that during inference, cache is read-only; but in the inplace delta update, cache needs to be updated. Since we don't have "write back"/flushing cache (when inference finishes, or cache evicts) during inference, we need to update cache weight **and** the back storage (uvm weight) simultaneously during the inplace delta update (thanks yinghai for pointing this out!). Reviewed By: jspark1105 Differential Revision: D39158010 fbshipit-source-id: a359c90ce66f58c141dd175ef572e38314456381
jianyuh
force-pushed
the
export-D39158010
branch
from
2f0d320
to
d64c490
Compare
|
This pull request was exported from Phabricator. Differential Revision: D39158010 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: With inplace update, we need to update dev_weights, uvm_weights, as well as the cache_weights (note that during inference, cache is read-only; but in the inplace delta update, cache needs to be updated. Since we don't have "write back"/flushing cache (when inference finishes, or cache evicts) during inference, we need to update cache weight and the back storage (uvm weight) simultaneously during the inplace delta update.
Differential Revision: D39158010