Multi-Pass Re-Ranking - Add native support for a generalized, configurable multi-pass reranking pipeline rather than limiting the system to a single global provider.

[jasonpsimon]

Feature Description

Add native support for a generalized, configurable multi-pass reranking pipeline (e.g., Local/Pointwise -> Remote/Listwise/Generative LLM) rather than limiting the system to a single global provider.

Core Motivation

Currently, Hindsight requires selecting a single provider via HINDSIGHT_API_RERANKER_PROVIDER. However, modern retrieval architectures rely heavily on progressive filtering to optimize the tradeoff between accuracy, context window limitations, API costs, and latency.

For instance, a developer might want to perform an aggressive, cheap first pass on a massive candidate pool (30–100 chunks) and then use a highly precise, context-heavy model on a tightly refined subset (5–15 chunks). Different combinations serve different architectural needs:

1. Cost & Latency Optimization: local (Fast, free MiniLM pointwise) ──> typesafe (Jev listwise compression).
2. Enterprise Multilingual/Instruction Pipeline: zeroentropy (Pinpoint cross-encoder precision) ──> cohere or siliconflow (Qwen3 / Gemma2 generative reasoning heavy-lifters).
3. Hybrid Failover/Refinement: Running a hosted, cost-sensitive model first, followed by a heavy frontier model endpoint only on the highest-scoring candidate shortlists.

Having to choose only one provider forces a compromise between raw recall depth and downstream context window efficiency.

Proposed Architecture / UX

Introduce an array or ordered pipeline configuration string along with progressive limits for each tier:

# Example 1: Local first-pass to cheap Jev listwise compression
HINDSIGHT_API_RERANKER_PIPELINE=local,typesafe
HINDSIGHT_RERANKER_PASS1_LIMIT=20
HINDSIGHT_RERANKER_PASS2_LIMIT=5

# Example 2: Multilingual Cross-Encoder to massive Custom LLM reasoning window
HINDSIGHT_API_RERANKER_PIPELINE=zeroentropy,siliconflow
HINDSIGHT_RERANKER_PASS1_LIMIT=25
HINDSIGHT_RERANKER_PASS2_LIMIT=8

The framework would sequentially route the top N candidates from the prior pass into the input structure required by the next provider in the pipeline.

Alternative Approaches Tried

Implementing this currently requires developers to break out of the native framework orchestration. We must set Hindsight to return an intentionally inflated chunk count, intercept the output array via the SDK client, and manually handle the subsequent reranking steps via external API requests. Native pipeline support would drastically clean up production RAG and agent codebases.