RDPRASSY RAG System

An evaluation-first RAG system by Devi Prasad that exposes the evidence,
scores, latency, and quality gates behind every answer.

Live demo · 90-second walkthrough · Architecture · Evaluation · API

---

Built by Devi Prasad Choudhary under the rdprassy portfolio brand.

Most RAG demos stop at a chat box. RDPRASSY RAG System is designed as an engineering case study: it lets you ask a question, inspect the query rewrite, compare lexical and semantic scores, see reciprocal-rank fusion and reranking, open the exact cited passages, and understand why the system answered or abstained.

The hosted version is fully deterministic and needs no paid model key. That makes the public demo reproducible, safe to fork, and honest about what is running. The architecture documents where production embedding, reranking, vector-store, and generation adapters fit.

What this project demonstrates

Capability	Implementation
Multi-format ingestion	PDF, Markdown, and Notion pipeline design
Structure-aware chunking	420-token target, 64-token overlap, heading preservation
Hybrid retrieval	BM25-style lexical rank + semantic concept matching
Rank fusion	Reciprocal-rank fusion across both candidate lists
Reranking	relevance, coverage, title affinity, and freshness
Grounded generation	claim-level citations and source excerpts
Safe failure	confidence-gated abstention for weak or out-of-domain evidence
Observability	rewrite, scores, candidates, context tokens, and latency by stage
Evaluation	MRR, precision@4, recall@4, citation coverage, abstention accuracy
Production posture	health API, tests, CI, Vercel config, edge-compatible build

System flow

flowchart LR
    S["Sources"] --> P["Parse"]
    P --> C["Chunk + dedupe"]
    C --> I1["Lexical index"]
    C --> I2["Vector index"]
    Q["Query"] --> R["Rewrite"]
    R --> I1
    R --> I2
    I1 --> F["RRF"]
    I2 --> F
    F --> X["Rerank"]
    X --> G{"Confidence gate"}
    G -- enough evidence --> A["Answer + citations"]
    G -- weak evidence --> B["Abstain"]
    A --> T["Trace + evaluation"]
    B --> T

Loading

Read the complete architecture and failure-mode analysis.

Live workbench

Try questions such as:

How does the system prevent unsupported answers?
How are changed documents re-indexed?
What metrics are used to evaluate RAG quality?
How does the system defend against prompt injection?
How are cost and latency controlled?

Every response includes:

• a grounded answer with numbered inline citations;
• confidence and explicit abstention state;
• source document, section, chunk ID, and excerpt;
• lexical, semantic, fused, and rerank scores;
• rewritten query and candidate counts;
• context-token volume and per-stage latency.

Quality baseline

The UI documents a representative 50-query synthetic baseline used for the project narrative:

Metric	Baseline
Faithfulness	0.96
Answer relevance	0.91
Context precision	0.88
Context recall	0.92
Mean reciprocal rank	0.95
Retrieval p95	42 ms

The repository also includes an executable golden set. It calculates retrieval, citation, and abstention metrics directly from the current code:

npm run eval

The generated report is written to reports/evaluation-latest.json. See evaluation methodology for definitions, release gates, and the important distinction between deterministic checks and judged generation metrics.

Quick start

Requirements: Node.js 22.13 or newer.

git clone https://github.com/rdprassy/rag-studio.git
cd rag-studio
npm install
npm run dev

Open http://localhost:3000.

No environment variables are required for the public demo.

Validate the complete system

npm run lint       # static checks
npm run test:unit  # retrieval, ranking, citation, abstention behavior
npm run eval       # golden-set metrics and JSON report
npm test           # unit + production worker build + rendered integration tests

The GitHub Actions workflow runs the quality gate on pull requests and uploads the evaluation report as a build artifact.

API

curl -X POST http://localhost:3000/api/query \
  -H 'content-type: application/json' \
  -d '{"query":"How does hybrid retrieval work?"}'

{
  "answer": "Retrieval starts with BM25 plus semantic similarity… [1]",
  "confidence": 0.91,
  "abstained": false,
  "citations": [],
  "retrieval": [],
  "trace": {
    "queryRewrite": "hybrid retrieval search candidate ranking",
    "candidatesScanned": 12,
    "candidatesReranked": 8,
    "contextTokens": 271,
    "totalMs": 49
  }
}

See the complete API contract.

Productionizing it

The deterministic engine is intentionally small enough to inspect. For a production workload:

1. replace semantic concept matching with an embedding provider;
2. move chunks to pgvector, Qdrant, or Pinecone;
3. add a cross-encoder or hosted reranker;
4. connect an LLM behind the grounded generation interface;
5. add tenant-aware prefilters and an authenticated ingestion surface;
6. send traces to OpenTelemetry, Langfuse, or Phoenix;
7. extend the golden set with real, reviewed domain questions.

The source metadata, scoring contract, confidence gate, evaluation loop, and UI remain valid across those substitutions.

Deploy

Vercel

Import the repository into Vercel. The included vercel.json selects the Next.js production build. The demo requires no secrets.

vercel

Container or other Node platform

npm run vercel-build
npm start

The health probe is available at GET /api/health.

Repository map

app/
├── api/query/          # query endpoint
├── api/health/         # readiness endpoint
├── RagStudio.tsx       # interactive product experience
└── page.tsx
lib/
├── corpus.ts           # synthetic public corpus
├── rag.ts              # retrieval and answer pipeline
└── types.ts
scripts/
└── evaluate.ts         # executable golden-set evaluation
tests/
├── rag.test.ts         # pipeline behavior
└── rendered-html.test.mjs
docs/
├── ARCHITECTURE.md
├── EVALUATION.md
├── API.md
└── DEMO_SCRIPT.md

Engineering observations

• Reranking earned its latency. It adds a small amount of time but removes high-level topical matches that do not answer the query.
• Chunk boundaries matter. Keeping headings attached to procedural content produced more useful evidence than increasing overlap alone.
• Hybrid retrieval is complementary. Lexical search anchors exact product and metric names; semantic matching recovers paraphrased intent.
• Abstention is a feature. A visible confidence gate makes the cost of weak evidence explicit and prevents the interface from rewarding confident prose.
• A trace changes debugging. Bad answers become a specific retrieval, ranking, context, or generation problem rather than “the model was weird.”

Walkthrough

The repository contains the finished demo video and the editable 90-second narration script.

License

MIT © 2026 Devi Prasad Choudhary