Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory management for Flictionaries & prediction algorithm #405

Merged
merged 3 commits into from
Feb 28, 2021

Conversation

patrickgold
Copy link
Member

@patrickgold patrickgold commented Feb 28, 2021

This PR is a response to the reported memory crashes in #403 and attempts to fix them. In detail, this is what has been improved:

  • The Flictionary load function previously created a lot of unnecessary allocations which it does not need. Additionally, the input stream of the binary raw file is now read as a stream, instead of per-caching the whole file before working with it.
  • The NgramNode representation has also changed, so each node is much smaller in memory size than before. Also, the child nodes are now stored in an ArrayList rather than a LinkedHashMap, which vastly reduces the allocated memory size.

@patrickgold patrickgold changed the title Fix memory leaks Fix memory maangement for Flictionaries & prediction algorithm Feb 28, 2021
@patrickgold patrickgold changed the title Fix memory maangement for Flictionaries & prediction algorithm Fix memory management for Flictionaries & prediction algorithm Feb 28, 2021
@patrickgold patrickgold merged commit 637d7fe into master Feb 28, 2021
@patrickgold patrickgold deleted the fix-memory-leaks branch March 30, 2021 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant