Commits on Feb 27, 2022

1. Add support for rendering a summary of objects held by the graph (pan… …

   …tsbuild#14638)
   
   As described in pantsbuild#12662, some use cases have surprising memory usage. To enable tracking those cases down, this change adds a `--memory-summary` option which summarizes the deep sizes of live objects in the `Graph`.
   
   On the Python side, the deep size is calculated using a very basic deduping walk of `gc.get_referents` (after having investigated [pympler](https://pypi.org/project/Pympler/), [guppy3](https://pypi.org/project/guppy3/), [objsize](https://pypi.org/project/objsize/)). On the Rust side, the `deepsize` crate is used, with sizes derived for all types reachable from `NodeKey` and `NodeOutput`.
   
   Example output:
   ```
   Memory summary:
   64		1		pants.backend.docker.subsystems.dockerfile_parser.DockerfileParser
   64		1		pants.backend.docker.subsystems.dockerfile_parser.ParserSetup
   64		1		pants.backend.java.dependency_inference.java_parser_launcher.JavaParserCompiledClassfiles
   64		1		pants.backend.java.dependency_inference.symbol_mapper.FirstPartyJavaTargetsMappingRequest
   <snip>
   1588620		957		(native) pants.engine.internals.graph.hydrate_sources
   2317920		2195		(native) pants.backend.python.dependency_inference.module_mapper.map_module_to_address
   2774760		2434		(native) pants.engine.internals.graph.determine_explicitly_provided_dependencies
   4446900		1458		(native) pants.engine.internals.graph.resolve_dependencies
   ```
   
   [ci skip-build-wheels]

   

   stuhood committed Feb 27, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for d92de44
   • Browse repository at this point

   Copy the full SHA

   d92de44 View commit details

   Browse the repository at this point in the history