Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using JAX models for asymmetric search & Answer Retrieval? #1764

Open
regstuff opened this issue Nov 23, 2022 · 2 comments
Open

Using JAX models for asymmetric search & Answer Retrieval? #1764

regstuff opened this issue Nov 23, 2022 · 2 comments

Comments

@regstuff
Copy link

Hi @nreimers
I've been using the msmarco-distilbert-dot-v5 model for asymmetric search and QnA retrieval.
I recently came across the models that were created during the Community Jax/Flax week.
Was wondering if these Jax models (for eg. the multi_qa_distilbert) give better results in asymmetric and answer retrieval compared to the msmarco-distilbert.
Or are the models listed in sbert's MSMarco page the best for this usecase?
My dataset is mostly non-technical, oriented more towards philosophy, religion etc.

@regstuff regstuff changed the title Using JAX models for asymmetric search? Using JAX models for asymmetric search & Answer Retrieval? Nov 23, 2022
@nreimers
Copy link
Member

Yes, multi-qa models perform better than ms marco models

@regstuff
Copy link
Author

regstuff commented Nov 26, 2022

Yes, multi-qa models perform better than ms marco models

Thanks for the info. Was wondering if there's any table comparing performance of these models, similar to what exists for the MS Marco models.

There are multiple JAX models, and I'm having a hard time figuring out which one I should use.

A related question: Is it alright to use the dot product models if my distance metric for similarity is euclidean distance? I'm using Scikit's BallTree to find knn, and it does not provide dotproduct as an option. So I am using Euclidean distance.

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants