Redrob Candidate Ranking — Submission

Architecture

Two-phase pipeline, split deliberately so the timed/constrained step is as small as possible:

1. precompute.py (offline, NOT time-limited) — loads all 100k candidates, generates real sentence-transformers embeddings (BAAI/bge-small-en-v1.5) for three text zones per candidate (role summary, experience/career-history, skills), fits TF-IDF vectorizers for a sparse hybrid layer, and computes deterministic rule-based features (honeypot risk, consulting-only-career flag, title-chasing flag, ML-depth-from-career-text score). All of this is saved to artifacts/.

2. rank.py (the timed step — must complete in <=5 min, <=16GB RAM, CPU only, no network) — loads the precomputed artifacts and does only fast vectorized arithmetic: cosine similarity via matrix multiplication, a weighted blend of dense+sparse scores per zone, the official composite formula, and the rule-based penalty multiplier. No model inference and no network calls happen in this step.

This split is explicitly permitted by submission_spec.md Section 10.3:

"If your system requires pre-computation (e.g., generating embeddings), document this clearly — pre-computation may exceed the 5-minute window, but the ranking step that produces the CSV must complete within it."

Why this architecture (brief)

• No LLM / agent framework in the ranking path. Calling an LLM per candidate (locally or via API) cannot fit the 5-minute/CPU/no-network budget at 100k scale — confirmed by direct calculation (see methodology summary). All "judgment" happens via deterministic, schema-grounded rules, computed once at precompute time.
• Multi-zone embeddings, not one blob. Summary/experience/skills are embedded and scored separately, then weighted (experience weighted heaviest at 0.55) — directly addresses the JD's instruction that a well-written summary shouldn't mask thin actual experience.
• Hybrid dense+sparse, not pure semantic. bge-small-en-v1.5 (a retrieval-tuned model) handles paraphrase/meaning; TF-IDF corrects for exact-keyword cases dense embeddings can miss. Blended at alpha=0.7 dense / 0.3 sparse.
• No FAISS / vector DB. At 100k candidates x 384 dims (~150MB per zone), brute-force matrix multiplication is faster and simpler than building an approximate-nearest-neighbor index. FAISS/Qdrant/Weaviate would be the right call at 200k+ scale with live writes — not here.
• Honeypot/consulting-only/title-chasing detection is rule-based, not LLM-based. Implements the JD's explicit disqualifiers directly from candidate_schema.json fields: career-history-duration vs stated years-of-experience mismatches, expert-proficiency-with-zero-duration skill flags, impossible title jumps, and consulting-firm-only career history with no product-company experience.
• Reasoning column is template-generated from real fields, not LLM output. Every sentence references an actual field value (title, company, years, specific flag) — by construction this cannot hallucinate a skill or employer the candidate doesn't have.

Setup

pip install sentence-transformers scikit-learn pandas numpy scipy joblib

GPU is optional — sentence-transformers will use CUDA automatically if torch.cuda.is_available(), otherwise falls back to CPU. bge-small-en-v1.5 is small enough to encode 100k short texts on CPU in a few minutes.

Reproduction

# 1. Unpack the candidate pool (if not already done)
gunzip -k candidates.jsonl.gz

# 2. Precompute (offline step — takes several minutes, NOT subject to the
#    5-minute ranking budget; see Section 10.3 disclosure above)
python precompute.py --candidates candidates.jsonl --outdir artifacts

# 3. Rank (the timed step — completes in well under 5 minutes)
python rank.py --artifacts artifacts --out submission.csv

# 4. Validate
python validate_submission.py submission.csv

Precompute runtime on [YOUR MACHINE SPEC HERE]: ~X minutes. Ranking runtime: ~X seconds (fill in after you run it).

Sandbox demo

streamlit run app.py

Upload any <=100-candidate slice of candidates.jsonl (or sample_candidates.json) to see the full pipeline run live with real embeddings, end to end, in the browser.

Files

Redrob — Hackathon Submission v1

Short summary

• What: Redrob is a reproducible candidate-ranking pipeline designed for a hackathon submission. It ranks candidate profiles using a hybrid dense (sentence-transformer embeddings) + sparse (TF-IDF) signal, rule-based penalty adjustments, and lightweight behavioral signals. The repo includes an offline precompute step and a fast, sandbox-safe ranking step that produces the final submission CSV.
• Goal: Provide an auditable, deterministic ranking system that satisfies the hackathon constraints: the ranking step is CPU-only, network-free, completes quickly, and generates a zero-hallucination reasoning column.

Why this design (short)

• Embedding mode (sentence-transformers): Dense embeddings capture semantic similarity and paraphrase matching across career-history text—critical for matching job requirements to varied candidate descriptions. They generalize beyond exact keywords and are robust to phrasing differences.
• Why not only sparse (TF-IDF/BM25): Sparse TF-IDF is brittle to paraphrase and synonymy. It can miss relevant candidates who describe the same work differently. We keep a sparse TF-IDF layer in a hybrid blend to catch exact-keyword signals that dense models sometimes underweight.
• Why not rely on vector DBs for the hackathon submission: For the contest we need a fully local, reproducible pipeline. We precompute L2-normalized NumPy matrices (.npy) and use fast matrix multiplies for cosine similarity. This avoids external vector DB dependencies (network, credentials, infra) and keeps the fast ranking step self-contained and auditable.
• Why no LLMs in ranking: LLMs introduce latency, cost, network calls, and hallucination risk. The submission requires a deterministic, offline reproducible ranking step and a reasoning column grounded in factual fields. We therefore create template-driven reasoning strings from deterministic features (no generative model calls) to ensure verifiability.

Repository layout (what each file does)

• precompute.py: Offline artifact builder. Reads candidates.jsonl (or .gz), extracts three text zones per candidate (features.extract_zones), encodes dense embeddings with a sentence-transformers model (e.g., BAAI/bge-small-en-v1.5), fits zone-specific TF-IDF vectorizers, computes rule-based features (honeypot risk, consulting-only, title-chasing, ML-depth), and writes artifacts to artifacts/ (dense .npy, sparse .npz, tfidf_vectorizer_*.joblib, rule_features.csv, profile_facts.csv, candidate_ids.npy). This step may take minutes/hours and may use a GPU; it is explicitly allowed to be slow for this pipeline.
• rank.py: The fast, sandbox-safe ranking step. Loads precomputed artifacts from artifacts/, computes blended dense+sparse per-zone similarity, composes a final score from semantic, skill, experience, and behavioral terms, applies rule-based penalty multipliers, and writes a 100-row submission.csv. This script is CPU-only and avoids network/LLM calls.
• features.py: Pure deterministic feature engineering. Extracts text zones used for embeddings, defines honeypot and career-quality heuristics, computes ml_depth_text_score, and computes a vectorized behavioral modifier for activity/open-to-work signals. No model or network calls here.
• jd_config.py: Single source of truth for job-description text, zone weights, and explicit JD-derived lists (consulting firms, non-technical titles, ML-depth keywords, thresholds). Keeps policy and hyperparameters explicit and auditable.
• app.py: Streamlit sandbox demo for small samples (<=100 candidates). Uses the same model as precompute but runs on a small sample in-memory for an interactive demo. Useful for manual inspection and lightweight demos — not required for the final evaluation but helpful for reviewers.
• validate_submission.py: Utility to verify the final submission.csv meets the hackathon rules (100 rows, correct header, score monotonicity, ID format, tie-break rules).

Contents of artifacts/ (what we store and why)

• candidate_ids.npy — array of candidate IDs in the artifact ordering (ensures stable alignment).
• dense_<zone>.npy — (N, D) L2-normalized float32 dense embeddings per zone.
• jd_dense_<zone>.npy — (1, D) L2-normalized JD query vector per zone.
• sparse_<zone>.npz — TF-IDF sparse matrix per zone (scipy format).
• jd_sparse_<zone>.npz — JD TF-IDF vector for sparse retrieval.
• tfidf_vectorizer_<zone>.joblib — fitted sklearn vectorizer to reproduce sparse transforms.
• rule_features.csv — per-candidate rule outputs (honeypot risk, penalties, ml_depth scores) used for penalty multipliers and reasoning.
• profile_facts.csv — lightweight candidate facts (title, years_of_experience, activity signals) used to craft deterministic reasoning strings.

Why NumPy .npy instead of external vector DBs

• Reproducibility: .npy files are deterministic, version-controllable (or attachable as release artifacts), and readable without external services.
• Simplicity: for a single ranking job we only need to compute a handful of matrix-vector multiplies. NumPy/BLAS is fast enough and avoids the operational overhead of vector DBs.
• Auditability: reviewers can open .npy artifacts and confirm numeric values used in ranking; vector DBs are opaque and require extra infra.

How to install

1. Create a virtualenv (recommended):

python -m venv .venv
# Windows (PowerShell)
.venv\\Scripts\\Activate.ps1
# macOS / Linux
source .venv/bin/activate

2. Upgrade pip and install dependencies:

python -m pip install --upgrade pip
pip install -r requirements.txt

3. Optional GPU notes: if you want GPU-accelerated embedding encoding, install a CUDA build of torch appropriate for your GPU (see PyTorch install guide). The default requirements.txt pins a CPU-friendly set; install GPU torch manually if needed.

Typical workflows

• Use provided artifacts (recommended for reproducible submission):

python rank.py --artifacts artifacts --out submission.csv
python validate_submission.py submission.csv

• If you need to regenerate artifacts locally (slow, may use GPU):

python precompute.py --candidates candidates.jsonl.gz --outdir artifacts

• Quick sandbox demo (<=100 candidates):

streamlit run app.py

Git/GitHub instructions (create a new repo and push)

• Create .gitignore (this repo includes one). Ensure you DO NOT push virtual environments, caches, or sensitive files.

Suggested commands (replace <your-repo-url> with the GitHub URL):

git init
git add .
git commit -m "Initial commit: Redrob hackathon submission v1 — ranking pipeline, precompute artifacts optional"
# Create a remote on GitHub via the website or use `gh repo create` (GitHub CLI):
# gh repo create my-org/redrob --public --source=. --remote=origin
git remote add origin <your-repo-url>
git branch -M main
git push -u origin main

Notes on large artifacts

• Artifacts can be large. For reproducibility, either:
  • upload the artifacts/ directory as a GitHub release or separate download link, or
  • use Git LFS for large binary files, or
  • provide a pre-seeded public cloud URL and document it in the README.

Security and privacy

• Candidate data may be sensitive in real settings. This dataset is for a hackathon submission, but treat personal data responsibly and remove or anonymize real personal PII before sharing broadly.

Acknowledgements & reproducibility

• The pipeline uses sentence-transformers models and sklearn TF-IDF. All ranking logic is deterministic given the artifacts and random seeds are not needed in the ranking step.

Contact

• If you want me to (optionally) prepare a requirements-locked.txt, CI workflow, or a small Makefile for the common flows, ask and I will add them.