New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse StringTree nodes on-the-fly #109

Closed
frizbog opened this Issue Jul 5, 2016 · 2 comments

Comments

Projects
None yet
1 participant
@frizbog
Owner

frizbog commented Jul 5, 2016

The GedcomParser still loads the entire file into memory before parsing it into the object model - it used to do it twice: once into an ArrayList, which was loaded into a StringTree hierarchy, which was then parsed into the object model. The ArrayList was removed recently, but the entire file is still loaded into the StringTree object...which is really unnecessary. Each 0-level item can be parsed as a root-level node as soon as the next 0-level item is encountered, and once it's parsed, it can be discarded (and garbage collected!), which should save memory and be faster.

@frizbog frizbog added this to the Release 3 milestone Jul 5, 2016

@frizbog frizbog self-assigned this Jul 5, 2016

@frizbog frizbog added the enhancement label Jul 5, 2016

frizbog added a commit that referenced this issue Jul 6, 2016

Issue #109 - GedcomParser/StringTreeBuilder do root-level node at a time
The GedcomParser and StringTreeBuilder classes now make a whole new
temporary StringTree for each root-level node and then parse it right
away into the object model, then discarding the StringTree.  It used to
build a huge StringTree for the entire file, then parse that.
@frizbog

This comment has been minimized.

Owner

frizbog commented Jul 6, 2016

This has made a significant improvement in peak memory usage, although no difference in memory usage once the file has been loaded into memory.

frizbog added a commit that referenced this issue Jul 7, 2016

Issue #109 - GedcomParser/StringTreeBuilder do root-level node at a time
The GedcomParser and StringTreeBuilder classes now make a whole new
temporary StringTree for each root-level node and then parse it right
away into the object model, then discarding the StringTree.  It used to
build a huge StringTree for the entire file, then parse that.
@frizbog

This comment has been minimized.

Owner

frizbog commented Jul 16, 2016

Released in v3.0.0

@frizbog frizbog closed this Jul 16, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment