Repository navigation
Testing Checklist
Anjana Tiha edited this page Mar 28, 2026
·
1 revision
Work through each item top to bottom. Check off as you go. Report anything that fails — include what you did and what happened.
- Factual question → answer with source citation
- Comparison question ("compare X vs Y") → covers multiple sources
- Summarise question ("summarise this document") → longer answer covering key points
- Follow-up with pronoun ("what else about her?") → uses prior context correctly
- Unrelated question → "not found" message, no hallucinated answer
- PDF (.pdf)
- Word (.docx)
- Excel (.xlsx)
- PowerPoint (.pptx)
- CSV (.csv)
- Markdown (.md)
- Plain text (.txt)
- HTML (.html)
- Legacy Excel (.xls)
- Webpage URL → indexed and queryable
- Direct PDF URL → downloaded and indexed
- Recursive crawl → multiple pages indexed
- Topic search → DuckDuckGo results crawled and indexed
-
rag_search→ searches documents and returns answer -
calculator→ "what is 15% of 85000?" -
summarise→ "summarise the uploaded document" -
sentiment→ "what is the sentiment of this paragraph?" -
translate→ "translate to Spanish: the sky is blue" -
finish→ agent completes task and returns final answer
- Query type badge shown (factual / comparison / summarise / general)
- Confidence badge shown (high / low)
- Pre-rerank chunk count shown
- Post-rerank chunk count shown
- Session stats increment after each query
- Clear conversation resets chat
- Tokens appear word by word as they generate (not all at once)
-
python main.py→ chat mode works -
python main.py --agent→ agent mode works -
python main.py --benchmark→ benchmark runs and prints scores
- Factual question → answer with source citation
- Comparison question → covers multiple sources
- Summarise question → longer answer covering key points
- Follow-up with pronoun → uses prior context correctly
- Unrelated question → "not found" message, no hallucination
- PDF (.pdf)
- Word (.docx)
- Excel (.xlsx)
- PowerPoint (.pptx)
- CSV (.csv)
- Markdown (.md)
- Plain text (.txt)
- HTML (.html)
- Legacy Excel (.xls)
- Webpage URL → indexed and queryable
- Direct PDF URL → downloaded and indexed
- Topic search → results crawled and indexed
-
rag_search→ searches documents -
calculator→ "what is 15% of 85000?" -
summarise→ "summarise the uploaded document" -
sentiment→ "what is the sentiment of this paragraph?" -
translate→ "translate to Spanish: the sky is blue" -
finish→ agent returns final answer
- Query type badge shown
- Confidence badge shown
- Pre/post rerank chunk counts shown
- Session stats increment
- Clear conversation resets chat
- Tokens appear word by word as they generate