Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory usage improvement #389

Closed
angelozerr opened this issue May 24, 2019 · 4 comments
Closed

Memory usage improvement #389

angelozerr opened this issue May 24, 2019 · 4 comments
Assignees
Milestone

Comments

@angelozerr
Copy link
Contributor

When XML large file is managed by the XML language server like https://github.com/angelozerr/lsp4xml/blob/master/org.eclipse.lsp4xml/src/test/resources/xml/nasa.xml (23.9MB), the memory grows up a lot (it can take 5GB!)

This issue is to improve memory problem. The first improvement that we can do is to improve memory with parse of DOM Document. It seems scanner is OK, but creating of DOM Document can be improved:

  • use int instead of Integer (ex: in DOMElement). Using Integer provides a problem with memory. Indeed autoboxing create a lot of Integer which are cached. Wen nasa.xml is parsed every time (without changes) we can see that there are a lot of memory used by Integer. To fix this problem, int should be used.

  • Memory Leak problem with DOMNode. DOMNode stores owner document, owner element, list of attributes with parent DOM node link. To help GC, we should have a dispose method in DOMNode to set parent, owner element, owncer document to null. It seems that with this fix, the memory keeps in 2GB after 10 minutes of parse instead of 5GB.

After thoses 2 fixes, we must continue to study problems (ex: execute symbol provider every time, etc) to try to display nasa.xml in vscode because today we cannot display it.

@angelozerr angelozerr self-assigned this May 24, 2019
@angelozerr angelozerr added this to the v0.7.0 milestone May 24, 2019
angelozerr pushed a commit that referenced this issue May 24, 2019
angelozerr pushed a commit that referenced this issue May 24, 2019
Signed-off-by: azerr <azerr@redhat.com>
@angelozerr
Copy link
Contributor Author

The XML scanner seems OK. If we parse the big file nasa.xml with scanner every time (see https://github.com/angelozerr/lsp4xml/blob/memory-improvement/org.eclipse.lsp4xml/src/test/java/org/eclipse/lsp4xml/performance/XMLScannerPerformance.java) we have this benchmark:

image

It takes around 100MB

angelozerr pushed a commit that referenced this issue May 24, 2019
Signed-off-by: azerr <azerr@redhat.com>
@angelozerr
Copy link
Contributor Author

angelozerr commented May 24, 2019

The DOM parser has problem with memory. If we parse the big file nasa.xml with DOM parser every time (see https://github.com/angelozerr/lsp4xml/blob/memory-improvement/org.eclipse.lsp4xml/src/test/java/org/eclipse/lsp4xml/performance/DOMParserPerformance.java) we have this benchmark:

image

It takes around 4-6GB!

@angelozerr
Copy link
Contributor Author

We can see that there are a lot of Integer instances which takes a lot memories:

image

This problem comes from autoboxing.

angelozerr pushed a commit that referenced this issue May 27, 2019
Add scanner & parser main test performance + use int instead of using
Integer.

Signed-off-by: azerr <azerr@redhat.com>
@angelozerr
Copy link
Contributor Author

The PR #392 improve memory with Integer to avoid autoboxing andwin around 1GB.

image

angelozerr pushed a commit that referenced this issue May 27, 2019
Add scanner & parser main test performance + use int instead of using
Integer.

Signed-off-by: azerr <azerr@redhat.com>
angelozerr pushed a commit that referenced this issue May 27, 2019
Add scanner & parser main test performance + use int instead of using
Integer.

Signed-off-by: azerr <azerr@redhat.com>
angelozerr added a commit that referenced this issue May 27, 2019
angelozerr pushed a commit that referenced this issue May 29, 2019
This PR adds a dispose method for DOM node to set as null the array list
fields and referenced fields (like owner element, owner attribute) to
help GC.
angelozerr pushed a commit that referenced this issue May 29, 2019
This PR adds a dispose method for DOM node to set as null the array list
fields and referenced fields (like owner element, owner attribute) to
help GC.
angelozerr pushed a commit that referenced this issue May 31, 2019
This PR adds a dispose method for DOM node to set as null the array list
fields and referenced fields (like owner element, owner attribute) to
help GC.
@fbricon fbricon changed the title Memory improvement Memory usage improvement Jun 5, 2019
angelozerr pushed a commit that referenced this issue Jun 6, 2019
This PR adds a dispose method for DOM node to set as null the array list
fields and referenced fields (like owner element, owner attribute) to
help GC.
angelozerr pushed a commit that referenced this issue Jun 28, 2019
This PR adds a dispose method for DOM node to set as null the array list
fields and referenced fields (like owner element, owner attribute) to
help GC.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant