PARC: Prediction-Free Adaptive Routing and Chunk Compression

English | 中文说明

PARC selects a compressed chunk-pool view for each retrieval-augmented generation query, using retrieval observations rather than generated answers. This repository releases the frozen router, selector-training code, complete retrieval/generation/official-scoring pipeline, benchmark adapters, synthetic examples, and score-only experiment summaries.

• Overview
• Installation
• Quick start
• Datasets
• Development split and selection protocol
• Run on your retrieval observations
• Train the selector
• Reported results
• Repository structure
• Troubleshooting
• Citation and license
• 中文说明

PARC combines a nine-route compression portfolio, raw and relative retrieval features, a frozen dual-view selector, confidence gates, and a compressed Dense-order anchor. At inference, routing does not consume test answers, generator outputs, quality scores, benchmark identities, or generator identities. "Prediction-free" means no trial-answer generation for routing at inference; the selector still predicts route utility. It is not training-free: development F1 differences supervise the selector, while Qwen and Llama are not fine-tuned.

The current paper scope contains 35 tasks from ETHIC, HELMET, and LongTableBench, with 11,997 scored units. The development inventory documents the 12 development tasks, source revisions, and selection rules. The selection protocol explains the retained pool's role, heuristic route budgets, and exact 144-policy development objective. Both have machine-readable metadata in configs/.

flowchart TD
    A[Upstream benchmark data] --> B[PARC adapters]
    B --> C[Question and chunks]
    C --> D[PARC retrieval materializer]
    D --> E[Dense + safe + nine candidate observations]
    E --> F[PARC features and dual-view selector]
    F --> G[Confidence gates and compressed anchor]
    G --> H[Selected route]
    H --> I[PARC generation and pinned official scoring]

The complete pipeline is public, while the quick start remains a CPU-only routing check that needs no dataset, API key, or generator checkpoint. An end-to-end run uses upstream datasets, model weights, and inventory validation. See the end-to-end guide and release boundaries.

Requirements: Git, Bash, Python 3.10.20, and pip. Use Linux or macOS; on Windows, run the shell scripts in WSL. GPU requirements depend on your external retrieval/generation stack, not the routing example.

git clone https://github.com/anning945/PARC.git
cd PARC
conda env create -f environment.yml
conda activate parc-routing

Alternatively, with Python 3.10.20 already installed:

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-benchmark.txt

Routing uses NumPy 2.2.6, joblib 1.5.3, and scikit-learn 1.7.2. Adapters also use pandas 2.2.3 and langchain-text-splitters 0.3.11. requirements-lock.txt pins the direct routing dependencies compatible with the bundled frozen selector.

For retrieval, generation, and scoring, install a CUDA-compatible PyTorch wheel and requirements-full.txt; details are in the end-to-end guide.

From the repository root, verify the release and run a synthetic routing query:

bash scripts/verify_parc_release.sh

This checks source and model hashes, runs the runtime self-test and unit tests, and executes the frozen selector. A successful run ends with:

PASS PARC release verification

Output: outputs/synthetic_routing_output.json, containing status: COMPLETE_RETRIEVAL_ONLY_ROUTING_V9, queries: 1, the selected route, and integrity metadata. It contains no generated answer or F1 score.

Run just the example, optionally choosing its output path:

bash scripts/run_parc_example.sh
bash scripts/run_parc_example.sh outputs/my_routing.json

With multiple Python installations, select the interpreter explicitly:

PYTHON=.venv/bin/python bash scripts/verify_parc_release.sh

The synthetic fixture exercises the routing interface. Generated outputs are ignored by Git; rerunning the example leaves source files unchanged.

Download data from the authors' sources; datasets are not redistributed here. Use the pinned versions and layout in the dataset guide.

LongTableBench is available from its official GitHub dataset directory. HELMET uses the pinned original v1 archive linked above.

The current paper inventory has 35 tasks and 11,997 scored units per method. The exact task names are in the current manifest. LongTableBench configurations reuse questions across formats and include separately scored conversation turns. See scope and aggregation for metric definitions.

For end-to-end execution, arrange the runner's required files according to the dataset guide, then run:

bash scripts/run_parc_schema_check.sh

Expected: status: PASS, with reports under outputs/parc_schema_check/. This is a schema/inventory check, not an F1 evaluation. The step-by-step workflow explains custom paths, selector-only export, and the complete execution stages. The end-to-end guide documents retrieval, generation, scoring, sharding, and resume behavior.

Run the full pipeline after filling the example configuration:

python scripts/parc_run_pipeline.py --config outputs/parc_pipeline.json

Then reaggregate the completed scores on the fixed 35-task manifest using the current-scope commands.

Supply a nonempty JSON list of query objects with exactly query_key, question, chunks, and methods. Each methods object must contain Dense, the safe compressed route, and all nine candidate routes with their chunk indices and retrieval scores. See the input contract, synthetic input, and route configuration. A selector stub alone still needs retrieval observations before routing.

python src/parc_router/parc_runtime.py \
  --model artifacts/parc_selector/parc_frozen_selector.joblib \
  --task-input /path/to/query_group_with_methods.json \
  --output outputs/routed.json

The output identifies the selected method for every query. The complete pipeline uses the same runtime internally and then reconstructs, generates, and scores paired Dense/PARC outputs.

PARC learns small tree-ensemble routing models from development F1 differences; it does not fine-tune an LLM. The training guide documents feature preparation, separate label files, exact hyperparameters, LOTO/LOFO policy selection, and loading a newly trained model.

Run a CPU training smoke test with artificial data:

python scripts/parc_make_training_example.py --output-dir outputs/parc_training_example
python scripts/parc_train_selector.py \
  --data outputs/parc_training_example/parc_development.npz \
  --selection published \
  --output-dir outputs/parc_trained_example

Use --selection cross-validated in a new output directory to run LOTO/LOFO and the 144-policy search. The synthetic labels exercise the training path. Training on development data uses materialized retrieval observations and development scores. New models have a separate manifest and use scripts/parc_route_trained.py; the paper's frozen model remains unchanged.

These point estimates come from verified task summaries. F1 is the unweighted macro over the named 35 tasks, on a 0-1 scale. Compression is weighted by scored units.

Every evaluated pool is compressed (11,997/11,997), with a mean compression of 56.43%. Compression measures the fraction of unique candidate chunks removed from each pool, averaged over scored units. The Qwen fixed CompressedDense R45-k8 baseline obtains 0.42739 F1 with 53.12% compression. See family and component results and the recomputed summary.

PARC/
  artifacts/parc_selector/    Frozen model, manifest, artifact hashes
  configs/                   Route policy and example dataset layout
  docs/                      Dataset sources, input contract, workflow
  examples/                  Synthetic retrieval-only input
  results/                   Verified scores and current 35-task summaries
  scripts/                   Retrieval, generation, scoring, training and checks
  src/parc_benchmarks/        Benchmark adapters and prompt renderers
  src/parc_pipeline/          End-to-end retrieval, generation and scoring
  src/parc_router/            Feature extraction, training and routing runtimes
  tests/                     Adapter, training, routing and integrity tests
  .github/workflows/          Clean-checkout continuous integration

For network restrictions, configure your own HTTPS_PROXY/HTTP_PROXY before downloading data. No proxy, token, or server address is hard-coded. File issues with the commit, environment versions and a minimal synthetic example, without private data or credentials. See contributing for tests and checksum maintenance.

Use CITATION.cff for software citation, record the exact commit, and cite each benchmark's original work.

PARC code uses the MIT License. Datasets and external checkpoints retain their own licenses. We do not include raw benchmark data, test answers, formal generations, private logs, credentials, or generator/retriever weights.

PARC 为每个 RAG 查询选择压缩后的 chunk 池视图，结合九种候选路由、 原始与相对检索特征、冻结的双视图选择器、置信度门控和压缩 Dense 顺序锚点。 推理时不读取测试答案、生成结果、F1、benchmark 身份或生成模型身份。 “Prediction-free”指推理时不生成试答来选择路由，不是无需训练或不做预测。 选择器用开发集 F1 差值训练，但不微调 Qwen 或 Llama。

已公开冻结模型、路由与训练源码、LOTO/LOFO 策略选择、数据适配器， 以及完整的检索、生成和固定官方评分流水线。原始数据、模型权重和上游评分源码 仍按各自许可证由使用者下载。

使用 Python 3.10.20。合成样例只需 CPU，不需数据集、LLM 或 API。 支持 Linux/macOS，Windows 建议使用 WSL。

git clone https://github.com/anning945/PARC.git
cd PARC
conda env create -f environment.yml
conda activate parc-routing
bash scripts/verify_parc_release.sh

成功时输出 PASS PARC release verification，结果写入 outputs/synthetic_routing_output.json，包含所选路由和校验信息，不包含答案或 F1。 只运行样例可执行 bash scripts/run_parc_example.sh。 也可用 Python 3.10.20 创建虚拟环境，安装 requirements-benchmark.txt； 多环境时用 PYTHON=.venv/bin/python 指定解释器。冻结模型使用上述固定依赖版本， 合成样例与自动测试用于检查路由接口和软件行为。

数据不放进仓库，直接从作者来源获取：

LongTableBench 使用官方 GitHub 数据目录，HELMET 使用表中固定的原始 v1 版本。

当前论文范围见固定清单。

1. 按数据与版本说明准备运行器所需文件。
2. 执行 bash scripts/run_parc_schema_check.sh 检查数据结构与运行清单。
3. 根据完整流水线说明填写配置。
4. 执行 python scripts/parc_run_pipeline.py --config outputs/parc_pipeline.json。
5. 保留各阶段 manifest、代码/数据/模型版本和哈希，完整评分必须覆盖全部清单。
6. 按当前结果说明用固定任务清单汇总 35 任务。

当前论文清单共 35 个任务、11,997 个评分单元。LongTableBench 的不同格式 会复用原始问题，多轮任务按轮计分。schema 检查核对数据结构、数量与输入格式； F1 由生成完成后的官方评分阶段计算。

先使用英文“Train the selector”中的两条命令运行 CPU 合成训练示例。 真实开发数据的准备方式、完整参数、分组验证和新模型使用方式见 选择器训练指南。published 复用公开策略， cross-validated 在开发集上重新执行 LOTO/LOFO 和 144 策略选择。 合成标签用于训练流程检查。新模型另存为 parc_trained_selector.joblib， 通过独立入口加载，并使用其对应的生成与评分结果进行评估。

当前 35 任务汇总中，Qwen 的宏平均 F1 从 0.42735 提高到 0.44106， Llama 从 0.26455 提高到 0.28381，分别增加 1.37 和 1.93 个百分点。 11,997 个单元的保留池均发生压缩，平均移除 56.43% 的候选 chunk。 压缩率按每个池中移除的唯一候选 chunk 比例计算，再对评分单元取平均。 Qwen 固定压缩 Dense 基线为 0.42739 F1、 53.12% 压缩率。详见当前汇总。

校验失败时检查本地文件改动，模型加载失败时核对依赖版本。 软件采用 MIT，数据与模型分别遵守上游许可证。引用见 CITATION.cff； 反馈问题请附提交版本、环境和合成样例。