Skip to content

Usage Guide

Anjana Tiha edited this page Mar 31, 2026 · 3 revisions

Usage Guide

Streamlit Chat

Streamlit Pipeline


Chat Mode

Chat mode runs the full RAG pipeline: classify → expand → hybrid retrieve → confidence check → rerank → synthesise → answer.

Responses stream to the UI token by token — words appear as they are generated rather than waiting for the complete answer.

Starting a Chat Session

streamlit run app.py      # web UI (with streaming)
python main.py            # terminal

Query Types

The system automatically classifies each query and adjusts retrieval depth:

Query Type Example Retrieval Budget
Factual "What is the candidate's GPA?" 5 chunks
Comparison "Compare Python vs Java performance" 15 chunks
Summarise "Summarise this document" Full document scope
General Any other query 10 chunks

Multi-Turn Conversation

The system maintains conversation history within a session. You can ask follow-up questions:

You: What is Alice's role?
Assistant: Alice is a Senior Software Engineer in the Platform team.

You: What department is she in?
Assistant: Alice is in the Platform team, as mentioned previously.

Click Clear conversation in the sidebar to start a new session.


URL Ingestion

Paste any public URL into the Add a URL field. Supported URL types:

Type How it is handled
Webpage (HTML) Tags stripped, sentence-windowed, indexed
PDF Downloaded to temp file, page-level chunking
Word (.docx) Downloaded, paragraph and table extraction
Excel (.xlsx, .xls) Downloaded, row-level key=value pairs
CSV Downloaded, row-level key=value pairs
PowerPoint (.pptx) Downloaded, text shapes per slide

The URL type is detected in priority order: Content-Type header → file extension → PDF magic bytes → HTML fallback.


Recursive URL Crawling

Follow all links from a seed URL up to a configurable depth.

  1. Paste the seed URL in the Add a URL field
  2. Enable Recursive crawl in the sidebar
  3. Set the depth (1–3) and optionally a keyword filter
  4. Click Add

Crawling is limited to the same domain as the seed URL to prevent unintended indexing of external sites.


Topic Search

Search the web for a topic and index the top results automatically — no API key required.

  1. Type a topic into the Search a topic field (e.g. "transformer architecture")
  2. Click Search
  3. The top DuckDuckGo results are crawled and indexed; a live log is shown in the sidebar

File Upload

Upload files directly through the web UI without placing them in ./docs/:

  1. Use the Upload a file panel in the sidebar
  2. Select one or more files
  3. Files are indexed immediately — no restart needed

Uploaded files are added to the live index alongside any documents already loaded from ./docs/.


Pipeline Panel

After each query, the sidebar shows:

Panel item What it shows
Query type Detected classification (factual / comparison / summarise / general)
Confidence Whether the best retrieved chunk cleared the similarity threshold
Best score Cosine similarity of the top retrieved chunk
Pre-rerank chunks All chunks retrieved before reranking
Post-rerank chunks Final ranked chunks used to generate the answer
Session stats Total queries answered in this session

Clearing the Index

To remove URL and file-upload chunks while keeping locally loaded documents:

Click Clear added content in the sidebar.

To rebuild the entire index from scratch (e.g. after adding new files to ./docs/):

rm -rf ./chroma_db/
streamlit run app.py

Clone this wiki locally