v0.5.4

Local reranker concurrency

Local FastEmbed reranking now has two settings:

• RERANKING_THREADS controls inference threading in the shared ONNX model.
• RERANKING_WORKERS limits how many reranking jobs can run concurrently per adapter.

Both accept positive integers and default to 1, preserving the previous behaviour.
On a multi-core host, a starting point to benchmark is:

RERANKING_THREADS=4
RERANKING_WORKERS=2

Restart Forgetful after changing these settings. More threads or workers can increase CPU
contention, so measure with your own workload. Each server process has its own limits.
The worker limit bounds running jobs, not the waiting queue. Cancelling a request does not
stop inference that is already running.

These settings apply only to RERANKING_PROVIDER=FastEmbed. Cloud and other HTTP rerankers
are unchanged and continue to support overlapping requests.

Tests and documentation

• Add regression coverage for worker limits, failures, cancellation, simultaneous SQLite
  searches with the real model, and HTTP concurrency independent of local worker settings.
• Make SQLite reranking tests actually exercise reranking and isolate them from local
  configuration without changing settings files.
• Document the new settings, defaults and tuning guidance.

Upgrade notes

• No database migration or embedding rebuild is required.
• No dependency changes are included.
• Higher local reranking concurrency is opt-in; upgrading alone keeps the previous limits.

Configuration guide

Full changelog