Support model buffers as pipeline postproc inputs (#2744) #2769
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
force-pushed
the
export-D69764721
branch
from
0a1bb33 to
7507deb
Compare
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Feb 27, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
force-pushed
the
export-D69764721
branch
from
7507deb to
1fe9ade
Compare
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Feb 27, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
force-pushed
the
export-D69764721
branch
from
1fe9ade to
aecdcf3
Compare
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Feb 27, 2025
Summary: Pull Request resolved: meta-pytorch#2769 Pull Request resolved: meta-pytorch#2744 Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
che-sh
force-pushed
the
export-D69764721
branch
from
aecdcf3 to
c1034b5
Compare
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Feb 27, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Mar 4, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Mar 4, 2025
Summary: Pull Request resolved: meta-pytorch#2769 Pull Request resolved: meta-pytorch#2744 Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
che-sh
force-pushed
the
export-D69764721
branch
2 times, most recently
from
81a42f5 to
2be5cbe
Compare
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Mar 4, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Mar 4, 2025
Summary: Pull Request resolved: meta-pytorch#2769 Pull Request resolved: meta-pytorch#2744 Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
added 4 commits
March 4, 2025 22:04
Differential Revision: D69438143
Differential Revision: D69461227
…Info, fold Differential Revision: D69461226
…ent operations (meta-pytorch#2744) Summary: Pull Request resolved: meta-pytorch#2744 Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy (**you are here**) Differential Revision: D69461228
che-sh
added a commit
to che-sh/torchrec
that referenced
this pull request
Mar 5, 2025
Summary: Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Summary: Pull Request resolved: meta-pytorch#2769 Pull Request resolved: meta-pytorch#2744 Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future. This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it). Internal Diff stack navigation: 1. D69292525 and below - before refactoring 2. D69438143 - Refactor get_node_args and friends into a class 3. D69461227 - refactor "joint lists" in ArgInfo into a list of ArgInfoStep 4. D69461226 - refactor `_build_args_kwargs` into instance methods on ArgInfo and ArgInfoStep 5. D69461228 - split monolithic `ArgInfoStep` into a class hierarchy 6. D69764721 - enable buffers as preproc arguments (**you are here**) Differential Revision: D69764721
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D69764721 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Torchrec rewriting logic got a bit hairy over the years, this sequence of changes aims to refactor the rewrite logic to be less convoluted and more maintainable in the future.
This change: Splits monolithic ArgInfoStep into multiple classes, each handling single potential operation (+minimum data necessary to perform it).
Internal
Diff stack navigation:
_build_args_kwargsinto instance methods on ArgInfo and ArgInfoStepArgInfoStepinto a class hierarchyDifferential Revision: D69764721