You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For nmslib, we need to allocate objects that wrap vectors with metadata. This PR updates the objects to be allocated in one buffer as opposed to allocating objects individually. This was shown to prevent memory fragmentation.
For this change, I ran the same experiments as were run for #772 and they showed significant improvements in the RSS of the process after the indexing workload was run.
Configuration
RSS beginning (KB)
RSS after indexing (KB)
Vectors Indexed (vectors)
nmslib-default (baseline)
8984748
11125772
648500
nmslib with bulk update
8972132
9439540
628000
Specifically, the change in RSS decreases by about 78% with this change in the experiment above.
In addition to this, fix bug in CMakeLists.txt file preventing unit tests from being created.
For this change, I ran the same experiments as were run for https://github.com//issues/772 and they showed significant improvements in the RSS of the process after the indexing workload was run.
Configuration RSS beginning (KB) RSS after indexing (KB) Vectors Indexed (vectors)
nmslib-default (baseline) 8984748 11125772 648500
nmslib with bulk update 8972132 9439540 628000
Specifically, the change in RSS decreases by about 78% with this change in the experiment above.
In addition to this, fix bug in CMakeLists.txt file preventing unit tests from being created.
Meta issue: #772
Creating this issue for tracking purposes.
For nmslib, we need to allocate objects that wrap vectors with metadata. This PR updates the objects to be allocated in one buffer as opposed to allocating objects individually. This was shown to prevent memory fragmentation.
For this change, I ran the same experiments as were run for #772 and they showed significant improvements in the RSS of the process after the indexing workload was run.
Specifically, the change in RSS decreases by about 78% with this change in the experiment above.
In addition to this, fix bug in CMakeLists.txt file preventing unit tests from being created.
For this change, I ran the same experiments as were run for https://github.com//issues/772 and they showed significant improvements in the RSS of the process after the indexing workload was run. Configuration RSS beginning (KB) RSS after indexing (KB) Vectors Indexed (vectors) nmslib-default (baseline) 8984748 11125772 648500 nmslib with bulk update 8972132 9439540 628000Specifically, the change in RSS decreases by about 78% with this change in the experiment above.
In addition to this, fix bug in CMakeLists.txt file preventing unit tests from being created.
Related to this PR: #773
The text was updated successfully, but these errors were encountered: