Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PropertyTree memory overuse #10534

Closed
rymsha opened this issue Apr 30, 2024 · 1 comment · Fixed by #10535
Closed

PropertyTree memory overuse #10534

rymsha opened this issue Apr 30, 2024 · 1 comment · Fixed by #10535
Assignees
Labels
Milestone

Comments

@rymsha
Copy link
Contributor

rymsha commented Apr 30, 2024

after #10015 was implemented we noticed extreme heap usage increase on some setups.
The consumption is very high for NodeVersion objects, specifically the ones with big amount of KV records in data.

A lot of memory is wasted because each property value is represented by an ArrayList with 1 element - wasting 9 reserved cells (This is how mutable ArrayList works).
Additionally PropertyArray object also wastes a lot of memory - because it uses HashMap. And When there are a bot more than 3000 elements in it 8192 elements are allocated.

@rymsha rymsha added the Bug label Apr 30, 2024
@rymsha rymsha added this to the 7.14.2 milestone Apr 30, 2024
@rymsha rymsha self-assigned this Apr 30, 2024
@alansemenov alansemenov changed the title PropertyTree momory overuse PropertyTree memory overuse Apr 30, 2024
@rymsha rymsha linked a pull request Apr 30, 2024 that will close this issue
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 10, 2024
rymsha added a commit that referenced this issue May 13, 2024
@rymsha
Copy link
Contributor Author

rymsha commented May 13, 2024

In order to save memory a separate representation of PropertyTree was introduced - specifically for cache.

  • PropertyTree is mutable ad must be copied anyway
  • PropertyTree is a complex data structure with internal HashMap for quick access by path/key

By storing single KV elements without a List (just a reference to a Value) and storing PropertySets in a List (or just value) we managed to shrink PropertyTree 3.8 MB object into ImputableProperty 1.9 MB object with a greater copy performance as well

rymsha added a commit that referenced this issue May 13, 2024
rymsha added a commit that referenced this issue May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant