feat: add scientific research context to RAG

[ridzkyyy]

Summary

• Add a scientific RAG helper layer for section-aware chunk metadata, stable local citation keys, Chroma result formatting, and Semantic Scholar paper normalization.
• Add POST /api/fetch-research to retrieve external research context from Semantic Scholar with bounded results, timeout handling, optional API-key support, and no-fail behavior for the chat flow.
• Harden local document retrieval with method/query validation, configurable CHROMA_PATH, bounded nResults, and preformatted citation-bearing evidence.
• Update RAG chat to use same-origin API calls instead of hard-coded localhost:3000, combine uploaded-document evidence with external research evidence, and require citation keys in answers.
• Add focused Vitest coverage for scientific section detection, citation key generation, Chroma formatting, Semantic Scholar URL building, and paper formatting.

Bounty context

Related to the Algora ISAAC-497 bounty for an enhanced RAG pipeline for scientific/research workflows. Algora currently links that bounty to aietal/isaac#45, but I could not access a public aietal/isaac repository. This repo contains the existing AimenGPT RAG implementation and already has related RAG bounty work under review, so I am submitting this here for maintainers to confirm fit.

/claim #45

Why this approach

The current RAG flow only retrieves uploaded Chroma chunks and asks the model to cite them, but it does not provide stable citation keys or broader research context. This patch keeps uploaded documents as the primary evidence source and adds external paper context as a secondary source, which is useful for scientific workflows where users often ask about a concept before uploading every relevant paper locally.

The Semantic Scholar fetch is intentionally bounded and non-fatal. If the external API is slow, unavailable, or rate-limited, the chat still proceeds with local retrieved documents instead of failing the whole answer.

Testing

From ui/:

PATH=/Users/ridzkyguntur/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin:$PATH npm test -- --run
PATH=/Users/ridzkyguntur/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin:$PATH npx tsc --noEmit
PATH=/Users/ridzkyguntur/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin:$PATH npm run lint

Results:

• Vitest passed: 17 tests
• TypeScript check passed
• Lint passed with pre-existing React hook dependency warnings in unrelated files

[nscarjr]

I've been analyzing the current RAG challenges in ISAAC-497. While PR #4 and this PR address basic citations and external fetching, I'm working on a more robust architecture using Llama Index that unifies Semantic Scholar data with local PDFs through a Hierarchical Indexing strategy.

My approach includes:

1. Scientific Section Awareness: Automated detection of Methodology/Results for better retrieval weighting.
2. 2. Stable Citation Anchoring: Content-hash based keys for immutable references.
3. 3. Unified Source Routing: Seamless merge of API data and local uploads.
      I have the core logic ready and would like to submit a comprehensive PR that aligns with the 'complete overhaul' requested. Is the team open to a Llama Index-based implementation?