Cross-Encoder Reranking

A Python package demonstrating cross-encoder reranking techniques for information retrieval. Companion code for the Medium article on cross-encoder reranking.

Quick Start

# Install
pip install -e ".[all]"

# Run the basic reranking example
python examples/01_basic_reranking.py

from cross_encoder_reranking import CrossEncoderReranker

reranker = CrossEncoderReranker("cross-encoder/ms-marco-MiniLM-L6-v2")
results, metrics = reranker.rank(
    query="How does photosynthesis work?",
    documents=[
        "Photosynthesis converts sunlight into chemical energy.",
        "The stock market rallied on Tuesday.",
        "Plants use chlorophyll to absorb light energy.",
    ],
)

for r in results:
    print(f"#{r.rank} (score: {r.score:.4f}): {r.text}")

Examples

Script	Description	Key Concepts
01_basic_reranking.py	Rerank 10 documents for a photosynthesis query	CrossEncoderReranker, before/after comparison
02_model_benchmarks.py	Compare 4 models on speed and quality (parallel)	BenchmarkRunner, ThreadPoolExecutor
03_fine_tuning_mse.py	MSE distillation with adversarial evaluation	MSEDistillationTrainer, seen/unseen topics
04_fine_tuning_bce.py	BCE fine-tuning on legal domain with adversarial eval	BCEFineTuner, domain adaptation, adversarial negatives
05_smart_query_caching.py	50-query traffic sim with precision eval	SmartQueryCache, Quora duplicate model, ground-truth validation
06_listwise_llm_reranking.py	50-doc funnel: bi-encoder → cross-encoder → LLM (OpenRouter)	ThreeStageReranker, structured output, rank movement
07_distillation.py	Cross-encoder → bi-encoder distillation on cybersecurity domain	CosineSimilarityLoss, SentenceTransformerTrainer, base vs distilled eval
08_colbert_late_interaction.py	ColBERT MaxSim vs cross-encoder with latency profiling	Late interaction, pre-indexing, QPS queue simulation, p50/p95/p99/p99.9

Fine-tuning and distillation examples include shared datasets in examples/data/ with adversarial distractors from closely related subtopics that test whether models understand true semantic relevance vs. keyword overlap.

Project Structure

src/cross_encoder_reranking/
├── __init__.py                     # Public API exports
├── models/
│   └── reranker.py                 # CrossEncoderReranker + RankedResult
├── pipelines/
│   ├── basic_reranking.py          # BasicRerankingPipeline
│   ├── fine_tuning.py              # MSEDistillationTrainer, BCEFineTuner, TrainingConfig
│   ├── query_caching.py            # SmartQueryCache
│   └── distillation.py             # CrossEncoderDistillation
└── utils/
    └── benchmarks.py               # BenchmarkRunner
examples/
├── 01-08_*.py                      # Runnable example scripts
└── data/                           # Shared training & eval datasets
    ├── mse_training.py             # 48 query-doc-score triples (12 topics)
    ├── bce_training.py             # 72 query-doc-label triples (12 legal topics)
    ├── eval_cases.py               # 20 adversarial eval cases (general domain)
    ├── bce_eval_cases.py           # 20 adversarial eval cases (legal domain)
    ├── retrieval_corpus.py         # 50-doc corpus with relevance tiers (example 06)
    ├── distillation_training.py    # 200 cybersecurity triples (20 topics × 10 pairs)
    ├── distillation_eval.py        # 30 cybersecurity eval cases (15 seen + 15 unseen)
    └── colbert_corpus.py           # 60-doc corpus + 10 queries (example 08)
├── outputs/                        # Saved stdout from each example run
tests/                              # pytest test suite (53 tests)
notebooks/                          # Jupyter demo notebook

API Reference

CrossEncoderReranker

Core reranker wrapping sentence_transformers.CrossEncoder. Methods: rank(), predict_scores().

BasicRerankingPipeline

Pipeline with formatted before/after display. Methods: rerank(), display().

MSEDistillationTrainer

Train a student model using MSE loss against teacher scores. Methods: train().

BCEFineTuner

Fine-tune a cross-encoder with binary cross-entropy loss. Methods: train().

SmartQueryCache

Semantic duplicate detection cache. Methods: store(), lookup(), find_duplicate().

CrossEncoderDistillation

Distill cross-encoder knowledge into a bi-encoder using CosineSimilarityLoss. Methods: train(), compare(), load_student(), display().

BenchmarkRunner

Multi-model parallel benchmark comparison. Methods: run(), display().

Development

# Create virtualenv
pyenv virtualenv 3.12.4 cross-encoder-reranking
pyenv local cross-encoder-reranking

# Install with dev dependencies
pip install -e ".[all]"

# Run tests
pytest

# Lint
ruff check src/ tests/ examples/

Contributing

See CONTRIBUTING.md for development setup and guidelines.

License

MIT