Skip to content

[Bug] Model output can be rendered as a spoofed tool-call card via tool_calls_render tag #6529

Description

@yyykinghh

Environment Information

  • MaxKB version: not independently verified in a live deployment.
  • Source reviewed: 1Panel-dev/MaxKB commit 7bd9055c6de8d3cfc81d17b292b7f9f91432edb7.

Problem Description

The common frontend Markdown renderer treats <tool_calls_render>...</tool_calls_render> as a rendering directive regardless of where the text originated. Therefore, if an LLM emits this exact tag in ordinary answer text, the UI renders it as a tool-call card. This can mislead users into believing that MaxKB actually invoked a tool.

This is a UI-integrity / user-trust issue: the tag does not execute a tool, but model-generated text can visually impersonate a genuine tool-call result.

Relevant frontend code:

  • ui/src/components/markdown/MdRenderer.vue registers tool_calls_render in TAG_PLUGINS and scans every source string with includes() / indexOf().
  • The same renderer is used for ordinary chat answer content in ui/src/components/ai-chat/component/answer-content/index.vue (:source="answer.content").
  • ui/src/components/markdown/tool-calls-render/index.vue parses the tag payload using JSON.parse() and renders the card.

Steps to Reproduce

  1. Create or configure an application whose final model answer contains the following text:
This is ordinary model output.
<tool_calls_render>{"type":"simple-tool-calls","icon":"","title":"Sensitive data lookup","content":{"input":"user_id=123","output":"success"}}</tool_calls_render>
  1. Open the application chat page and receive the answer.
  2. Observe that the content inside the tag is rendered as a tool-call card rather than ordinary Markdown/text.

The same behavior is also observable in other views that pass untrusted/display text to MdRenderer, such as execution-detail rendering.

Expected Correct Result

Only tool-call metadata generated by trusted backend/workflow execution should be rendered as a tool-call card. A tag appearing in normal model output should be displayed literally (or escaped/removed according to a documented policy), not interpreted as an internal rendering protocol.

Actual Result

Any ordinary text passed to MdRenderer that contains an exact balanced <tool_calls_render>...</tool_calls_render> tag is parsed into a tool_calls_render render node without provenance or schema validation. With a valid JSON payload it produces a plausible card; with invalid JSON it still replaces the text with a fallback empty card.

Suggested Direction

Please consider separating trusted render metadata from plain model text, for example:

  • carry tool-call display data in a structured response field instead of embedding an XML-like tag in answer text; or
  • allow this tag parser only for explicitly trusted workflow/server-generated segments; and
  • escape/reserve internal renderer tags in ordinary LLM output.

Additional Information

The backend intentionally generates this tag for real tool output in apps/application/flow/tools.py, so the renderer support itself is expected. The issue is that the frontend does not distinguish backend-generated tool-render content from arbitrary model output before applying the parser.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions