New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CrossEncoder device #2463
CrossEncoder device #2463
Conversation
Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
Hello! This looks good at a glance! I also like moving I'll try it out locally in a minute
|
Very clean indeed, no comments here! Thanks a bunch for this.
|
Not a problem!
|
- Trigger SentenceTransformer Cross Encoder models now run fast on GPU enabled machines, including Mac ARM devices since UKPLab/sentence-transformers#2463 - Details - Use cross-encoder to rerank search results by default on GPU machines and when using an inference server - Only call search API when pause in typing search query on web, desktop apps
Pull Request Summary
This pull request focuses on enhancing device compatibility and improving code organization by relocating the
get_device_name
method and utilizing it to support the 'mps' device inCrossEncoder.py
.Key Changes
get_device_name
fromSentenceTransformer.py
toutil.py
for better modularity.CrossEncoder.py
to useget_device_name
fromutil.py
, enabling support for the 'mps' device.Testing
CrossEncoder
functions correctly with 'mps' device support.Conclusion
get_device_name
function toutil.py
.CrossEncoder
to support 'mps' devices, improving performance on macOS platforms.