-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make tictac more efficient by making level1 a map #441
Conversation
Pre-change (1M keys, tree size large): Generating Keys took 2513 milliseconds Memory footprint [{total,356732576},{processes,334051328},{processes_used,334044488},{system,22681248},{atom,540873},{atom_used,524383},{binary,1015120},{code,9692859},{ets,721496}] Generating new tree took 1 milliseconds Loading tree took 27967 milliseconds Memory footprint [{total,36733040},{processes,8875472},{processes_used,8875048},{system,27857568},{atom,540873},{atom_used,524449},{binary,6236480},{code,9692859},{ets,721496}] Exporting tree took 434 milliseconds Importing tree took 100 milliseconds Memory footprint [{total,155941512},{processes,123734808},{processes_used,123734384},{system,32206704},{atom,540873},{atom_used,524449},{binary,10401144},{code,9692859},{ets,721496}] Garbage collect Memory footprint [{total,39660504},{processes,8257520},{processes_used,8256968},{system,31402984},{atom,540873},{atom_used,524449},{binary,9781760},{code,9692859},{ets,721496}] Post change: Generating Keys took 2416 milliseconds Memory footprint [{total,284678120},{processes,258349528},{processes_used,257758568},{system,26328592},{atom,893161},{atom_used,878150},{binary,1013880},{code,11770188},{ets,774224}] Generating new tree took 0 milliseconds Loading tree took 2072 milliseconds Memory footprint [{total,49957448},{processes,17244856},{processes_used,16653896},{system,32712592},{atom,893161},{atom_used,878216},{binary,7397496},{code,11770188},{ets,774224}] Exporting tree took 448 milliseconds Importing tree took 108 milliseconds Memory footprint [{total,46504880},{processes,11197344},{processes_used,10606384},{system,35307536},{atom,893161},{atom_used,878216},{binary,9992112},{code,11770188},{ets,774224}] Garbage collect Memory footprint [{total,47394048},{processes,12223608},{processes_used,11632520},{system,35170440},{atom,893161},{atom_used,878216},{binary,9855008},{code,11770188},{ets,774224}]
Note the map version has a higher memory footprint. However, it should lead to less churn on the memory. The post change benefit in memory footprint post import of tree is because of a manual garbage_collect() call, not because of any efficiency in the load process. |
Size = SnkTree#tictactree.size, | ||
SizeSrc = SrcTree#tictactree.size, | ||
SizeSnk = SnkTree#tictactree.size, | ||
true = SizeSrc == SizeSnk, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here the assertion is clear enough
Co-authored-by: Thomas Arts <thomas.arts@quviq.com>
…rtinsumner/leveled into mas-d31-maskit.i116-level1map
* Make tictac more efficient by making level1 a map Pre-change (1M keys, tree size large): Generating Keys took 2513 milliseconds Memory footprint [{total,356732576},{processes,334051328},{processes_used,334044488},{system,22681248},{atom,540873},{atom_used,524383},{binary,1015120},{code,9692859},{ets,721496}] Generating new tree took 1 milliseconds Loading tree took 27967 milliseconds Memory footprint [{total,36733040},{processes,8875472},{processes_used,8875048},{system,27857568},{atom,540873},{atom_used,524449},{binary,6236480},{code,9692859},{ets,721496}] Exporting tree took 434 milliseconds Importing tree took 100 milliseconds Memory footprint [{total,155941512},{processes,123734808},{processes_used,123734384},{system,32206704},{atom,540873},{atom_used,524449},{binary,10401144},{code,9692859},{ets,721496}] Garbage collect Memory footprint [{total,39660504},{processes,8257520},{processes_used,8256968},{system,31402984},{atom,540873},{atom_used,524449},{binary,9781760},{code,9692859},{ets,721496}] Post change: Generating Keys took 2416 milliseconds Memory footprint [{total,284678120},{processes,258349528},{processes_used,257758568},{system,26328592},{atom,893161},{atom_used,878150},{binary,1013880},{code,11770188},{ets,774224}] Generating new tree took 0 milliseconds Loading tree took 2072 milliseconds Memory footprint [{total,49957448},{processes,17244856},{processes_used,16653896},{system,32712592},{atom,893161},{atom_used,878216},{binary,7397496},{code,11770188},{ets,774224}] Exporting tree took 448 milliseconds Importing tree took 108 milliseconds Memory footprint [{total,46504880},{processes,11197344},{processes_used,10606384},{system,35307536},{atom,893161},{atom_used,878216},{binary,9992112},{code,11770188},{ets,774224}] Garbage collect Memory footprint [{total,47394048},{processes,12223608},{processes_used,11632520},{system,35170440},{atom,893161},{atom_used,878216},{binary,9855008},{code,11770188},{ets,774224}] * Tidy-up * Add type * Remove ++ requiring copy of Acc Rely on mechanism producing a sorted result, not sorting * Update src/leveled_tictac.erl Co-authored-by: Thomas Arts <thomas.arts@quviq.com> * Update following review --------- Co-authored-by: Thomas Arts <thomas.arts@quviq.com>
Pre-change (1M keys, tree size large):
Post change: