allow not memory planning mutable buffers #10071
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10071
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit d8fa917 with merge base d42676f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D72749868 |
Summary: Config option to not memory plan mutable buffers. This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr. Differential Revision: D72749868
JacobSzwejbka
force-pushed
the
export-D72749868
branch
from
1627860 to
cf917e8
Compare
Summary: Config option to not memory plan mutable buffers. This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr. Differential Revision: D72749868
JacobSzwejbka
force-pushed
the
export-D72749868
branch
from
cf917e8 to
4deaf94
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72749868 |
JacobSzwejbka
added
release notes: api
|
This pull request was exported from Phabricator. Differential Revision: D72749868 |
Summary: Pull Request resolved: pytorch#10071 Config option to not memory plan mutable buffers. This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr. Differential Revision: D72749868
JacobSzwejbka
force-pushed
the
export-D72749868
branch
from
4deaf94 to
3bf1cc2
Compare
Summary: Config option to not memory plan mutable buffers. This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr. Differential Revision: D72749868
JacobSzwejbka
force-pushed
the
export-D72749868
branch
from
3bf1cc2 to
c97559a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72749868 |
Summary: Config option to not memory plan mutable buffers. This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr. Differential Revision: D72749868
JacobSzwejbka
force-pushed
the
export-D72749868
branch
from
c97559a to
d8fa917
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72749868 |
| spec.extra_tensor_info = ExtraTensorInfo( | ||
| fully_qualified_name=fqn, location=TensorDataLocation.SEGMENT | ||
|
|
||
| if is_mutable_buffer: |
There was a problem hiding this comment.
This is the actual core logic change, the rest of the changes are mostly piping a flag around
| # Lower the graph to executorch. | ||
| ep = ep.to_executorch( | ||
| config=ExecutorchBackendConfig(emit_mutable_buffer_names=True) | ||
| config=ExecutorchBackendConfig( |
There was a problem hiding this comment.
Maybe also add a post_init in the ExecutorchBackendConfig dataclass that asserts if emit_mutable_buffer_names is False and alloc_mutable_buffers is also False.
There was a problem hiding this comment.
I considered this but since the MemoryPlanningPass is user configureable there is no guarantee memory_planning_pass.alloc_mutable_buffers exists. So I just check the end result in the emitter.
| if is_mutable_buffer: | ||
| # Emit names if we are supposed to. | ||
| if self.emitter_state.emit_mutable_buffer_names: | ||
| if spec.extra_tensor_info is None: |
There was a problem hiding this comment.
A little confused by this as i haven't kept track of the tensor info changes, what does it mean if extra_tensor_info is None and if it isn't we overwrite the fqn again? Maybe add a small comment here too.
There was a problem hiding this comment.
None just means no one has created one, if its not none they might not have populated the fqn so I populate it. I guess I could check what it is before overwriting it but the name is unique so it should always be safe to do this.
extra_tensor_info is where we put optional info in the flatbuffer to not regress the size of every tensor by too much in embedded cases that dont need it.
Differential Revision: D72749868 Pull Request resolved: pytorch#10071
3 participants
Summary:
Config option to not memory plan mutable buffers.
This when paired with a future runtime PR will allow users to retrieve buffers by name in the runtime and then set their dataptr.
Differential Revision: D72749868
cc @angelayi