Repository navigation
[Feature Request] Settings for the LLM index queue and embedding retries/timeout #14767
Unanswered
dominatorstang
asked this question in
Feature Requests
Replies: 1 comment
|
Two corrections to my post. The exact version is Paperless-ngx 3.3.2 (Docker), and I've checked that 3.4.2 still has no setting for either item. Also, per the contributing guidelines: this request was researched and drafted with the help of an AI assistant on my behalf. I'm not planning to open a PR myself unless maintainers want one and agree on an approach here first. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Description
Two settings for the AI/LLM index, so a slow embedding backend does not hurt the rest of the system:
Problem today
With a self-hosted OpenAI-compatible embedding endpoint, embedding a large batch can take minutes and the endpoint is sometimes busy or cold. The LLM index tasks share a queue and workers with normal consumption, so a long index run can occupy workers that consumption needs. When the embedding call fails or times out, retries and timeout cannot be tuned, so a transient backend slowdown turns into repeated failed tasks.
Workaround we use
A Django settings override that monkeypatches task routing (to move the index tasks to their own queue served by a dedicated worker) and the embedding client constructor (to raise
max_retriesand the timeout). It works, but it depends on internal task names and the embedding client's__init__signature, so every upgrade needs re-checking.Proposal
PAPERLESS_AI_LLM_INDEX_QUEUE(default: the current queue, so nothing changes unless set).PAPERLESS_AI_EMBEDDING_MAX_RETRIESandPAPERLESS_AI_EMBEDDING_TIMEOUT.Happy to put up a PR if the maintainers think this fits.
I searched existing issues and discussions for embedding queue, retries and timeout and did not find an equivalent request (#13257 is a related support thread about slow AI classification, not a request for these settings).
Paperless-ngx 3.3.x, Docker install.
All reactions