import Embeddings takes too long time

[dmi]

from txtai import Embeddings

this takes about 3-5 seconds. Is there a way to speedup the process in case I wish to use only external embeddings?
It will be good for various services, like RAG applications, allowing to use txtai for it's fantastic simplicity, but use GPU calculations out of the box.

More precisely: I want to handle all database storage and search stuff at the service side with txtai, but want to handle all LLM stuff on the GPU server. So I can't start txtai as a service completely on a GPU side.

[davidmezzetti]

I'm not exactly sure why it is taking 3-5 seconds to load. Have you tried to narrow that down?

[davidmezzetti]

One additional note, the import time is related to the number of dependencies installed.

For example, if you ran pip install txtai[all] it will take longer to import than pip install txtai. Additionally, you can try installing the cpu-only version of torch.

pip install torch==2.3.0+cpu torchvision==0.18.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

[pyaji]

@davidmezzetti
Hi. Is it possible to minimize size of depences, and install only part of txtai[pipline] ? All i need is Textractor.

[davidmezzetti]

You can run:

pip install txtai[pipeline-data]

[davidmezzetti]

Closing this issue due to inactivity. Please re-open or open a new issue if there are further questions.