[export] Move serialized custom class objs to toplevel #114371
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
angelayi
requested review from
avikchaudhuri,
gmagogsfm,
tugsbayasgalan and
zhxchen17
as code owners
|
This pull request was exported from Phabricator. Differential Revision: D51479125 |
This PR needs a
|
angelayi
force-pushed
the
export-D51479125
branch
from
14ff806
to
8d1286c
Compare
|
This pull request was exported from Phabricator. Differential Revision: D51479125 |
ydwu4
approved these changes
Nov 22, 2023
Summary: Move the serialized CustomClassHolder objects to the toplevel SerializedArtifact instead of embedding the bytes in the graph. Currently the CustomClassHolder objects are embedded in the graph instead of being lifted to the ExportedProgram, so there's some logic introduced to lift it to the higher level of the serialized ExportedProgram. However, once that CustomClassHolder objects get lifted, we can remove the TODOs I added. Test Plan: CI Reviewed By: ydwu4, zhxchen17 Differential Revision: D51479125
angelayi
force-pushed
the
export-D51479125
branch
from
8d1286c
to
fa73ff8
Compare
|
This pull request was exported from Phabricator. Differential Revision: D51479125 |
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
pytorchmergebot
pushed a commit
that referenced
this pull request
Dec 15, 2023
Since pytorch 2.1 release we've made some BC breaking changes to the serialized schema. We should update it in time for the 2.2 release. Some of the changes include: * #114371 - custom class objects / pybinded objects are no longer saved directly to the `ExportedProgram` structure. Instead, the name is serialized inside of the program, and the actual bytes are stored. in a separate location from the exported program, allowing it to be saved to a different location. * #111204 - `GraphSignature` structure changed and `call_spec` is removed from the `GraphModule` schema * #111407 - `loss_outout` -> `loss_output` * #113075 - `example_inputs` removed from the `ExportedProgram` structure (this originally did not store anything), `dialect` added to the `ExportedProgram` structure. * #113689 - tensor constants are now lifted as inputs to the graph, and their locations are stored in the `GraphSignature` * #114172 - removed `equality_constraints` and added a `SymExprHint` for all symbolic expressions. Pull Request resolved: #115712 Approved by: https://github.com/gmagogsfm
angelayi
added a commit
that referenced
this pull request
Dec 15, 2023
Since pytorch 2.1 release we've made some BC breaking changes to the serialized schema. We should update it in time for the 2.2 release. Some of the changes include: * #114371 - custom class objects / pybinded objects are no longer saved directly to the `ExportedProgram` structure. Instead, the name is serialized inside of the program, and the actual bytes are stored. in a separate location from the exported program, allowing it to be saved to a different location. * #111204 - `GraphSignature` structure changed and `call_spec` is removed from the `GraphModule` schema * #111407 - `loss_outout` -> `loss_output` * #113075 - `example_inputs` removed from the `ExportedProgram` structure (this originally did not store anything), `dialect` added to the `ExportedProgram` structure. * #113689 - tensor constants are now lifted as inputs to the graph, and their locations are stored in the `GraphSignature` * #114172 - removed `equality_constraints` and added a `SymExprHint` for all symbolic expressions. Pull Request resolved: #115712 Approved by: https://github.com/gmagogsfm
guilhermeleobas
pushed a commit
to guilhermeleobas/pytorch
that referenced
this pull request
Dec 18, 2023
Since pytorch 2.1 release we've made some BC breaking changes to the serialized schema. We should update it in time for the 2.2 release. Some of the changes include: * pytorch#114371 - custom class objects / pybinded objects are no longer saved directly to the `ExportedProgram` structure. Instead, the name is serialized inside of the program, and the actual bytes are stored. in a separate location from the exported program, allowing it to be saved to a different location. * pytorch#111204 - `GraphSignature` structure changed and `call_spec` is removed from the `GraphModule` schema * pytorch#111407 - `loss_outout` -> `loss_output` * pytorch#113075 - `example_inputs` removed from the `ExportedProgram` structure (this originally did not store anything), `dialect` added to the `ExportedProgram` structure. * pytorch#113689 - tensor constants are now lifted as inputs to the graph, and their locations are stored in the `GraphSignature` * pytorch#114172 - removed `equality_constraints` and added a `SymExprHint` for all symbolic expressions. Pull Request resolved: pytorch#115712 Approved by: https://github.com/gmagogsfm
malfet
pushed a commit
that referenced
this pull request
Dec 18, 2023
Since pytorch 2.1 release we've made some BC breaking changes to the serialized schema. We should update it in time for the 2.2 release. Some of the changes include: * #114371 - custom class objects / pybinded objects are no longer saved directly to the `ExportedProgram` structure. Instead, the name is serialized inside of the program, and the actual bytes are stored. in a separate location from the exported program, allowing it to be saved to a different location. * #111204 - `GraphSignature` structure changed and `call_spec` is removed from the `GraphModule` schema * #111407 - `loss_outout` -> `loss_output` * #113075 - `example_inputs` removed from the `ExportedProgram` structure (this originally did not store anything), `dialect` added to the `ExportedProgram` structure. * #113689 - tensor constants are now lifted as inputs to the graph, and their locations are stored in the `GraphSignature` * #114172 - removed `equality_constraints` and added a `SymExprHint` for all symbolic expressions. Pull Request resolved: #115712 Approved by: https://github.com/gmagogsfm
dmenig
pushed a commit
to dmenig/pytorch
that referenced
this pull request
Dec 21, 2023
Since pytorch 2.1 release we've made some BC breaking changes to the serialized schema. We should update it in time for the 2.2 release. Some of the changes include: * pytorch#114371 - custom class objects / pybinded objects are no longer saved directly to the `ExportedProgram` structure. Instead, the name is serialized inside of the program, and the actual bytes are stored. in a separate location from the exported program, allowing it to be saved to a different location. * pytorch#111204 - `GraphSignature` structure changed and `call_spec` is removed from the `GraphModule` schema * pytorch#111407 - `loss_outout` -> `loss_output` * pytorch#113075 - `example_inputs` removed from the `ExportedProgram` structure (this originally did not store anything), `dialect` added to the `ExportedProgram` structure. * pytorch#113689 - tensor constants are now lifted as inputs to the graph, and their locations are stored in the `GraphSignature` * pytorch#114172 - removed `equality_constraints` and added a `SymExprHint` for all symbolic expressions. Pull Request resolved: pytorch#115712 Approved by: https://github.com/gmagogsfm
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Move the serialized CustomClassHolder objects to the toplevel SerializedArtifact instead of embedding the bytes in the graph.
Currently the CustomClassHolder objects are embedded in the graph instead of being lifted to the ExportedProgram, so there's some logic introduced to lift it to the higher level of the serialized ExportedProgram. However, once that CustomClassHolder objects get lifted, we can remove the TODOs I added.
Test Plan: CI
Reviewed By: zhxchen17
Differential Revision: D51479125
cc @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @suo @ydwu4