New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

速度优化 #28

Closed

muyannian opened this issue Mar 29, 2013 · 1 comment

Labels

Owner

muyannian commented Mar 29, 2013

通过这次测试，暴露出很多问题,先前很多字段是我没有测试到的，总结如下：

Count有极大的优化空间
原先只有2台机器的情况下，内存资源是稀缺的，但现在有10台机器，内存富裕较多，故针对数值型的计算以及所有dist计算，
可以考虑不在像之前通过docid->termNum->（类似视频的关键帧压缩）->termValue的准换
而是直接采用 docid->termNum->termValue的转换，省去关键帧后，像creativeid这种重复值比较低的字段，dist，sum等速度提升不止是一倍两倍的关系
因硬盘空间富裕，frq文件不再采用zip压缩，测试过程中发现cpu使用率比较高，主要原因就是frq文件的zip解压引起

Owner Author

muyannian commented Apr 2, 2013

已经完成提升空间很大

muyannian closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment