New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

我觉得使用 SentenceTransformer 作为句子向量化不好 #28

Open

shguan2018 opened this issue Sep 29, 2021 · 1 comment

shguan2018 commented Sep 29, 2021

我觉得使用 SentenceTransformer 作为句子向量化不好,dimension 高达 512,句子向量化存储到 annoy 里导致空间特别大

Collaborator

xiaoyichao commented Oct 22, 2021

我觉得使用 SentenceTransformer 作为句子向量化不好,dimension 高达 512,句子向量化存储到 annoy 里导致空间特别大

如果觉得大。可以不使用向量召回，只使用Es召回，BERT类的应用都有高消耗的情况，硬件和效果之间总得做个平衡。一般的数据集向量化之后都是工业上可接受的范围。还有一个思路就是将向量在Es中召回，Es7支持向量召回。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment