[Feature] Reduce Response Verbosity by Improving System Prompt Precision

[GovindhKishore]

Problem

As more data sources are integrated into the retrieval pipeline, the chatbot responses are becoming increasingly long and noisy. This is a known concern raised in the project and is caused by two compounding factors:

1. Retrieval side - multiple data sources return large numbers of documents, many of which are tangentially relevant (partially addressed by the FlashRank reranking PR Add FlashRank reranker to HybridRetriever to improve retrieval quality #116)
2. Generation side - the current system prompts in both reactome/prompt.py and uniprot/prompt.py explicitly instruct the LLM to be comprehensive and capture all mechanistically relevant details, which causes the LLM to include everything it finds regardless of how directly relevant it is to the question

Root Cause in Current Prompts

reactome/prompt.py:

"comprehensively, mechanistically, and with precision"
"Capture ALL mechanistically relevant details"
"information-rich narrative"
"presenting background, mechanisms, and significance"

uniprot/prompt.py:

"comprehensively, accurately, and in an engaging manner"
"Answer the question comprehensively and accurately"
"Provide any useful background information"

These instructions tell the LLM to maximise coverage - which worked fine with one data source but causes response bloat as more sources are added.

Proposed Fix

Modify both system prompts to shift from exhaustive coverage to relevance-first precision:

• Replace "comprehensively" with "accurately and precisely"
• Replace "Capture ALL details" with "Include only the most directly relevant details"
• Add explicit instructions: "Answer ONLY what was asked", "Do NOT repeat information", "Do NOT add background unless essential"
• Add stop condition: "stops when the question is fully answered"
• Retain all citation formatting, source discipline, and tone guidelines unchanged

Files to Change

src/retrievers/reactome/prompt.py
src/retrievers/uniprot/prompt.py

Status

Implementation is ready. Currently testing the changes locally to gather before/after response comparisons as proof before raising a PR.

Relation to Other Work

This complements PR #116 (FlashRank reranking) which addresses the retrieval side of the same problem. Together they form a complete solution:

• PR Add FlashRank reranker to HybridRetriever to improve retrieval quality #116 ensures better documents go to the LLM.
• This PR ensures LLM generates most relevant content.

[mithil27360]

This is a great direction.

One additional issue I observed during testing is that verbosity is often accompanied by weak grounding. The model tends to include generic biological explanations instead of referencing specific Reactome pathways or stable identifiers.

Along with reducing verbosity, it may help to explicitly enforce:

• Inclusion of Reactome stable identifiers (R-HSA IDs) where applicable
• Avoidance of generic textbook-style explanations
• Preference for directly retrieved pathway data over background knowledge

This could further improve both precision and trustworthiness of responses.

[GovindhKishore]

Thanks a lot, @mithil27360, for testing the PR. I really appreciate the detailed feedback. I understand your point about verbosity being linked with weak grounding, especially the need to prioritize Reactome specific identifiers and avoid generic explanations.

I’ll start working on these improvements and will update you once I push the changes.

[GovindhKishore]

Hi @mithil27360, I have incorporated both points into PR #121. I think the changes work well but feel free to test it when you find the time and let me know if you think anything else needs adjusting.