Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate sentence-transformers/paraphrase-multilingual-mpnet-base-v2 model #10

Closed
danielbichuetti opened this issue Sep 22, 2023 · 3 comments · Fixed by #103
Closed

Investigate sentence-transformers/paraphrase-multilingual-mpnet-base-v2 model #10

danielbichuetti opened this issue Sep 22, 2023 · 3 comments · Fixed by #103
Assignees
@Anush008
Labels
model request request for supporting new models

Comments

@danielbichuetti
Copy link

It would be great, specially for users that need a language besides English to support multilingual-e5-large. It's the best model for plenty of non-mainstream languages.
@danielbichuetti danielbichuetti changed the title Add support for multilingual-e5-large Add intfloat/multilingual-e5-large model Sep 22, 2023
@NirantK NirantK self-assigned this Sep 25, 2023
@NirantK
Copy link
Contributor

NirantK commented Sep 28, 2023

Planning to support this latest by 2023-10-03.

In early experiments, I am not seeing enough performance improvements from quantizing this model. About 5% gain in throughput or so.

If you've suggestions more multi-lingual models, please do share!

@danielbichuetti
Copy link
Author

The current benchmarks for "default" multilingual models suggest this model to be the best. The other common model that is used a lot is paraphrase-multilingual-mpnet-base-v2.

Btw, thanks for the work on e5.

@NirantK NirantK changed the title Add intfloat/multilingual-e5-large model Investigate sentence-transformers/paraphrase-multilingual-mpnet-base-v2 model Oct 5, 2023
@generall generall added the model request request for supporting new models label Jan 5, 2024
@NirantK
Copy link
Contributor

NirantK commented Jan 30, 2024

We've now moved away from the idea of supporting only the fastest and quantized model. Hence, we should support both these models:

https://huggingface.co/Xenova/paraphrase-multilingual-mpnet-base-v2=
https://huggingface.co/Xenova/multilingual-e5-large

@NirantK NirantK assigned Anush008 and unassigned NirantK Jan 30, 2024
@NirantK NirantK mentioned this issue Jan 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Anush008 Anush008

Labels
model request request for supporting new models
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multili… qdrant/fastembed
4 participants
@generall @NirantK @Anush008 @danielbichuetti