-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug Fix: wrong listPageCount when listTotalBytes % pageSize==0 #5
Conversation
std::string dst{}; | ||
est_decomp_size *= 10; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does est_decomp_size multiple 10?
Configuration parameter may be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the reminder, that's in fact something i forgot to remove...
@@ -156,11 +156,10 @@ namespace SPTAG | |||
#endif | |||
|
|||
bool oneContext = (m_indexFiles.size() == 1); | |||
std::unique_ptr<Compressor> m_pCompressor = std::make_unique<Compressor>(); // no need compress level info for decompress | |||
auto m_pCompressor = std::make_unique<Compressor>(); // no need compress level to decompress |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you use auto to replace std::unique_ptr?
I think std::unique_ptr is more clear.
@@ -552,7 +550,7 @@ namespace SPTAG | |||
LOG(Helper::LogLevel::LL_Info, "EnableDeltaEncoding: %s\n", p_opt.m_enableDeltaEncoding ? "true" : "false"); | |||
LOG(Helper::LogLevel::LL_Info, "EnableDataCompression: %s, ZstdCompressLevel: %d\n", p_opt.m_enableDataCompression ? "true" : "false", p_opt.m_zstdCompressLevel); | |||
std::vector<size_t> postingListBytes(headVectorIDS.size()); | |||
std::unique_ptr<Compressor> m_pCompressor = std::make_unique<Compressor>(p_opt.m_zstdCompressLevel); | |||
auto m_pCompressor = std::make_unique<Compressor>(p_opt.m_zstdCompressLevel); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you use auto to replace std::unique_ptr?
I think std::unique_ptr is more clear.
…#297) * MVP version of Data compress (#1) * Support delta-encoding (#2) * support config of compress level (#3) * integrate zstd with cmake (#4) * Bug Fix: wrong listPageCount when listTotalBytes % pageSize==0 (#5) * train & share dictionary (#6) * rearrange posting list (#7) * config minDictTraingBufferSize and dictBufferCapacity (#8) * cmake with local installed zstd (#9) * refine visual studio config (#10) * parallel for get compressed size (#11) * fix check truth bug (#12) * change zstd branch (#13) * remove verbose log in truth analysis; refine dockerfile * check rvalue in Compressor.h * resolve back compatibility by add a search option: WithDataCompressionFeatures * remove redundant configs, change head info format only when compression enabled * reuse buffer when decompression * remove exit(1) from search index * bug fix: reuse compression buffer in workspace # with '#' will be ignored, and an empty message aborts the commit. * code refine * optimize mem usage * replace vcpkg zstd with nuget * config issue * fix solution build issue Co-authored-by: Philip Adams <35666630+PhilipBAdams@users.noreply.github.com>
No description provided.