Add ability in SSDTBE to fetch weights from L1 and SP from outside of the module #3166

chouxi · 2025-07-07T19:15:02Z

X-link: https://github.com/facebookresearch/FBGEMM/pull/1513

Given row ids, fetch the updated weighs from L1 and SP.
register_backward_hook_before_eviction This function will be called in delta tracker

register the hook to be executed in backward before the sp eviction.
So the hooks are reordered to let the hook registered through this function to execute first.

fetch_from_l1_sp_w_row_ids will be called inside the backward hook registered using above function

to get updated weights for the given row ids already added offset to SSDTBE tables.
fetch from lxu_cache_weights (L1) and rest from inserted_rows (SP)

table_names is added to let the delta tracker aware of the table to TBE map.

the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map.
in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn.

Reviewed By: q10

Differential Revision: D72188513

facebook-github-bot · 2025-07-07T19:15:11Z

This pull request was exported from Phabricator. Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 Pull Request resolved: pytorch#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

facebook-github-bot · 2025-07-08T18:02:19Z

This pull request was exported from Phabricator. Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: Pull Request resolved: meta-pytorch#3166 X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 Pull Request resolved: pytorch#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

facebook-github-bot · 2025-07-08T18:17:31Z

This pull request was exported from Phabricator. Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: Pull Request resolved: meta-pytorch#3166 X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 Pull Request resolved: pytorch#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

facebook-github-bot · 2025-07-08T19:32:57Z

This pull request was exported from Phabricator. Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: Pull Request resolved: meta-pytorch#3166 X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (pytorch#4450) Summary: X-link: meta-pytorch/torchrec#3166 Pull Request resolved: pytorch#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

… the module (meta-pytorch#3166) Summary: Pull Request resolved: meta-pytorch#3166 X-link: pytorch/FBGEMM#4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513

facebook-github-bot · 2025-07-08T19:40:37Z

This pull request was exported from Phabricator. Differential Revision: D72188513

… the module (#4450) Summary: X-link: meta-pytorch/torchrec#3166 Pull Request resolved: #4450 X-link: facebookresearch/FBGEMM#1513 Given row ids, fetch the updated weighs from L1 and SP. `register_backward_hook_before_eviction` This function will be called in delta tracker - register the hook to be executed in backward before the sp eviction. - So the hooks are reordered to let the hook registered through this function to execute first. `fetch_from_l1_sp_w_row_ids` will be called inside the backward hook registered using above function - to get updated weights for the given row ids already added offset to SSDTBE tables. - fetch from `lxu_cache_weights (L1)` and rest from `inserted_rows (SP) ` `table_names` is added to let the delta tracker aware of the table to TBE map. - the delta tracker stores the updated ids in forward pass's lookup module, it stores ids as a 'fqn' -> 'ids' map. - in backward pass, per TBE module we need to know which ids we want to fetch for the updated weights, thus the table names is needed to infer fqn. Reviewed By: q10, duduyi2013 Differential Revision: D72188513 fbshipit-source-id: e35b86695877088829d359b5601561fcc51d2dcd

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 7, 2025

facebook-github-bot added the fb-exported label Jul 7, 2025

chouxi force-pushed the export-D72188513 branch from b7002ee to 9c5f960 Compare July 8, 2025 17:54

chouxi force-pushed the export-D72188513 branch from 9c5f960 to 2077c59 Compare July 8, 2025 17:55

chouxi force-pushed the export-D72188513 branch from 2077c59 to b6307da Compare July 8, 2025 18:02

chouxi force-pushed the export-D72188513 branch from b6307da to 01784db Compare July 8, 2025 18:17

chouxi force-pushed the export-D72188513 branch from 01784db to 43752d0 Compare July 8, 2025 19:24

chouxi force-pushed the export-D72188513 branch from 43752d0 to 08311a6 Compare July 8, 2025 19:25

chouxi force-pushed the export-D72188513 branch from 08311a6 to 28bd9bd Compare July 8, 2025 19:33

chouxi force-pushed the export-D72188513 branch from 28bd9bd to b2ea7bf Compare July 8, 2025 19:40

facebook-github-bot closed this in d7c7098 Jul 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add ability in SSDTBE to fetch weights from L1 and SP from outside of the module #3166

Add ability in SSDTBE to fetch weights from L1 and SP from outside of the module #3166

Uh oh!

chouxi commented Jul 7, 2025

Uh oh!

facebook-github-bot commented Jul 7, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add ability in SSDTBE to fetch weights from L1 and SP from outside of the module #3166

Add ability in SSDTBE to fetch weights from L1 and SP from outside of the module #3166

Uh oh!

Conversation

chouxi commented Jul 7, 2025

Uh oh!

facebook-github-bot commented Jul 7, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

facebook-github-bot commented Jul 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants