Skip to content

feat(api): pass app_id to model plugins for provider-side cost attribution#35859

Open
ryuta-kobayashi-ug wants to merge 2 commits into
langgenius:mainfrom
ryuta-kobayashi-ug:feat/pass-app-id-to-model-plugins
Open

feat(api): pass app_id to model plugins for provider-side cost attribution#35859
ryuta-kobayashi-ug wants to merge 2 commits into
langgenius:mainfrom
ryuta-kobayashi-ug:feat/pass-app-id-to-model-plugins

Conversation

@ryuta-kobayashi-ug
Copy link
Copy Markdown

Fixes #35772

Summary

Pass app_id to model plugins so they can tag LLM requests with the originating Dify app for provider-side cost attribution (e.g., Bedrock requestMetadata, OpenAI metadata, Vertex AI labels).

Changes

  • Add app_context.py with ContextVar-based get_current_app_id() / set_current_app_id()
  • Set app_id in all app runners (chat, completion, agent_chat, advanced_chat, workflow) before LLM invocation
  • PluginModelRuntime.invoke_llm() reads get_current_app_id() and passes it to PluginModelClient
  • PluginModelClient includes app_id in the HTTP payload to plugin daemon
  • app_id is None when called outside app context (RAG routing, title generation, etc.)

Design

Uses contextvars.ContextVar because PluginModelRuntime is created at tenant scope (before app execution), and graphon sits between the app runner and PluginModelRuntime with a fixed interface that cannot pass app_id as an argument.

Testing

  • Unit tests for _dispatch_payload with/without app_id
  • E2E tested all 5 app types (chatbot, chatflow, workflow, text generator, agent) — all pass app_id correctly
  • Regression tested: official plugin, RAG, blocking API mode — no degradation

Checklist

  • This change requires a documentation update
  • I've added a test for each change that was introduced
  • I ran dev/reformat(backend) to appease the lint gods

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label May 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

Pyrefly Diff

base → PR
--- /tmp/pyrefly_base.txt	2026-05-06 13:52:40.434110858 +0000
+++ /tmp/pyrefly_pr.txt	2026-05-06 13:52:26.808039981 +0000
@@ -4126,33 +4126,33 @@
 ERROR Argument `None` is not assignable to parameter `project` with type `str` in function `core.ops.utils.validate_project_name` [bad-argument-type]
    --> tests/unit_tests/core/ops/test_utils.py:136:40
 ERROR Missing required key `content` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:340:19
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:392:19
 ERROR Missing required key `content_type` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:340:19
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:392:19
 ERROR Key `type` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:340:20
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:392:20
 ERROR Key `value` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:340:36
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:392:36
 ERROR Missing required key `content` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:341:19
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:393:19
 ERROR Missing required key `content_type` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:341:19
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:393:19
 ERROR Key `type` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:341:20
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:393:20
 ERROR Key `value` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:341:37
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:393:37
 ERROR Missing required key `content` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:360:17
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:412:17
 ERROR Missing required key `content_type` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:360:17
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:412:17
 ERROR Key `type` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:360:18
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:412:18
 ERROR Missing required key `content` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:361:18
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:413:18
 ERROR Missing required key `content_type` for TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:361:18
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:413:18
 ERROR Key `type` is not defined in TypedDict `MultimodalRerankInput` [bad-typed-dict-key]
-   --> tests/unit_tests/core/plugin/impl/test_model_client.py:361:19
+   --> tests/unit_tests/core/plugin/impl/test_model_client.py:413:19
 ERROR Argument `werkzeug.wrappers.request.Request` is not assignable to parameter `request` with type `flask.wrappers.Request` in function `core.plugin.impl.trigger.PluginTriggerClient.invoke_trigger_event` [bad-argument-type]
    --> tests/unit_tests/core/plugin/impl/test_trigger_client.py:126:21
 ERROR Argument `werkzeug.wrappers.request.Request` is not assignable to parameter `request` with type `flask.wrappers.Request` in function `core.plugin.impl.trigger.PluginTriggerClient.invoke_trigger_event` [bad-argument-type]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pass app_id to model plugins for provider-side cost attribution

1 participant