Feat/inline sources in content upstream#347
Draft
EnjoyBacon7 wants to merge 4 commits into
Draft
Conversation
… `extra`
Open WebUI 0.9.2 (and 0.8.x) silently drops the non-standard `extra`
field that OpenRAG returns at the chat-completion top-level — verified
by inspecting `backend/open_webui/utils/middleware.py` (the
`non_streaming_chat_response_handler` reads `choices`, `output`, and
`usage` only) and the Svelte/TS frontend (no `response.extra` ever
referenced for message rendering). Same behavior in LibreChat,
Continue, Cursor, etc. — the OpenAI contract has no notion of citations.
This means our deployed RAG aliases on Mirai-OWUI (PR mirai-open-webui#118)
display the LLM answer correctly but render no sources at all, even
though OpenRAG retrieves them and embeds them in `extra.sources`. The
end-user has the conclusion without the audit trail — a hard regression
for legal/compliance use cases.
Approach: opt-in flag `INLINE_SOURCES_IN_CONTENT` that appends a
markdown source block to the assistant `content` after stripping the
LLM's `[Sources: ...]` tag. The `extra` field is unchanged so clients
that already consume it (the Mirai pipe_openrag, MyRAG bridge,
programmatic agents) keep their structured access.
Default is `false` — no breaking change for existing deployments.
Implementation
--------------
- `components/utils.py`:
- `inline_sources_enabled()` — env-var read, lazy.
- `format_sources_as_markdown(sources)` — render a deduplicated,
score-filtered, ranked markdown source list. Dedup on file_url
(OpenRAG returns several chunks of the same file with different
scores; we show each file once with its best chunk). Ranks by
relevance_score desc and caps at INLINE_SOURCES_TOP_K (default 5).
- Three knobs: `INLINE_SOURCES_IN_CONTENT`, `INLINE_SOURCES_TOP_K`,
`INLINE_SOURCES_MIN_SCORE`.
- `routers/openai.py` — three call sites, each appends the markdown
block to the cleaned content right after `extract_and_strip_sources_block`:
- non-streaming `/v1/chat/completions`
- non-streaming `/v1/completions`
- streaming via `stream_with_source_filtering` (in `components/utils.py`),
which now emits one extra delta chunk carrying the markdown block
before the finish-reason chunk. Sent before finish so clients that
buffer until finish (most do) see it as part of the content.
- Tests:
- `TestFormatSourcesAsMarkdown`: dedup, ranking, min_score, top_k,
enabled/disabled, empty input.
- `TestStreamWithInlineSources`: inline block appears when enabled,
is omitted when disabled, `extra.sources` still reaches the finish
chunk regardless.
- `deploy/.env.example.vm`: documents the three knobs and recommends
enabling on the Mirai deployment (OWUI 0.9.2 ignores `extra`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
format_sources_as_markdown silently dropped web sources because their dedup key resolved to "" (no file_url/filename/source). Add url as a fallback in both the key and the link URL lookup. Also escape [ and ] in labels — only | was escaped before, so a title like "Report [draft]" would break the markdown link at the first ]. Add two tests to lock both behaviors in.
components.utils -> components.indexer.__init__ -> Indexer -> chunker imports back into components.utils, which fails when utils is imported standalone (e.g. from test_source_filtering). sanitize_text is only used inside format_web_context, so import it there instead.
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR adds opt-in functionality to embed markdown "Sources :" blocks directly into OpenAI-compatible API response content. The feature is controlled by environment variables ( Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Router as /chat/completions
participant Extractor as Source Extractor
participant Filter as Source Filter
participant Formatter as format_sources_as_markdown
participant Response
Client->>Router: Request with query
Router->>Extractor: Extract sources from context
Extractor-->>Router: List of sources with scores
Router->>Filter: Filter sources (min_score, citations)
Filter-->>Router: Filtered sources
alt INLINE_SOURCES_IN_CONTENT enabled
Router->>Formatter: Format sources as markdown
Note over Formatter: Deduplicate by URL<br/>Rank by score<br/>Apply top_k limit
Formatter-->>Router: "**Sources :**" block
Router->>Response: Append markdown to content
else INLINE_SOURCES_IN_CONTENT disabled
Router->>Response: Use content as-is
end
Router->>Response: Include filtered sources in extra
Response-->>Client: OpenAI-compatible response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4 tasks
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
openrag/components/test_source_filtering.py (1)
205-245:⚠️ Potential issue | 🟡 MinorPin the baseline streaming tests to inline-sources off.
TestStreamWithSourceFilteringcurrently inherits the ambientINLINE_SOURCES_IN_CONTENTvalue, so these assertions can start failing in CI or local runs where the feature is enabled. Make the off-by-default behavior explicit for this class.Suggested fix
class TestStreamWithSourceFiltering: SOURCES = [{"file": "a.pdf"}, {"file": "b.pdf"}, {"file": "c.pdf"}] + + `@pytest.fixture`(autouse=True) + def _disable_inline_sources(self, monkeypatch): + monkeypatch.setenv("INLINE_SOURCES_IN_CONTENT", "false")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@openrag/components/test_source_filtering.py` around lines 205 - 245, Pin the inline-sources feature off for this test class by explicitly overriding INLINE_SOURCES_IN_CONTENT within TestStreamWithSourceFiltering so it does not inherit ambient state; locate the TestStreamWithSourceFiltering class and set INLINE_SOURCES_IN_CONTENT = False at class scope (or ensure the equivalent environment flag is set to disable inline sources for the duration of these tests) so stream_with_source_filtering assertions remain stable.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@openrag/components/utils.py`:
- Around line 245-304: The markdown link destination isn't being sanitized in
format_sources_as_markdown: build a link-safe destination before interpolating
url (variable url used in the loop that appends to lines) by wrapping the raw
url in angle brackets and escaping any '>' (or otherwise percent-encoding unsafe
characters) so characters like spaces or parentheses don't break the
[label](destination) syntax; update the loop that appends f"{i}.
[{label}]({url}){score_suffix}" to instead produce a sanitized destination
(e.g., "<sanitized_url>") and keep using label and score_suffix as before.
---
Outside diff comments:
In `@openrag/components/test_source_filtering.py`:
- Around line 205-245: Pin the inline-sources feature off for this test class by
explicitly overriding INLINE_SOURCES_IN_CONTENT within
TestStreamWithSourceFiltering so it does not inherit ambient state; locate the
TestStreamWithSourceFiltering class and set INLINE_SOURCES_IN_CONTENT = False at
class scope (or ensure the equivalent environment flag is set to disable inline
sources for the duration of these tests) so stream_with_source_filtering
assertions remain stable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 7b32f740-b652-433e-b1bf-0eaff81a6b42
📒 Files selected for processing (4)
.env.exampleopenrag/components/test_source_filtering.pyopenrag/components/utils.pyopenrag/routers/openai.py
…or spaces don't break Source URLs like https://en.wikipedia.org/wiki/Foo_(bar) or any path containing a space terminate the bare-URL link destination prematurely, rendering as malformed markdown. Wrap the URL in <...> instead — that form tolerates spaces, parens, and any other character except < > and unescaped newlines. Percent-encode the only two that would still close the destination.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #345 with two follow-up fixes on top of etiquet's commit.
OpenRAG returns sources in a non-standard
extrafield. Most OpenAI-compat clients (Open WebUI, LibreChat, Continue) drop it, so users see the answer without sources. This adds an opt-inINLINE_SOURCES_IN_CONTENTflag that also writes a markdown source block into the assistantcontent. Default off, no breaking change.Knobs
INLINE_SOURCES_IN_CONTENT(default false): enable the content blockINLINE_SOURCES_TOP_K(default 5): cap on rendered sourcesINLINE_SOURCES_MIN_SCORE(default -inf): filter weak chunksChanges vs #345
cfea199- include web sources in the inline block, escape\,[,],|in labels (review feedback from feat(openai): inline sources in content (opt-in) for clients ignoringextra#345)f5e59e1- movesanitize_textimport insideformat_web_contextto break a circular import that surfaced whencomponents.utilsis imported from a test