Add support for rendering a summary of objects held by the graph (cherrypick of #14638) #14644
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As described in #12662, some use cases have surprising memory usage. To enable tracking those cases down, this change adds a
--memory-summary
option which summarizes the deep sizes of live objects in theGraph
.On the Python side, the deep size is calculated using a very basic deduping walk of
gc.get_referents
(after having investigated pympler, guppy3, objsize). On the Rust side, thedeepsize
crate is used, with sizes derived for all types reachable fromNodeKey
andNodeOutput
.Example output:
[ci skip-build-wheels]