Skip to content

refactor(api): extract shared RAG domain entities into core/rag/entity#34685

Merged
asukaminato0721 merged 1 commit intolanggenius:mainfrom
corevibe555:refactor/rag-domain-entities-core
Apr 7, 2026
Merged

refactor(api): extract shared RAG domain entities into core/rag/entity#34685
asukaminato0721 merged 1 commit intolanggenius:mainfrom
corevibe555:refactor/rag-domain-entities-core

Conversation

@corevibe555
Copy link
Copy Markdown
Contributor

@corevibe555 corevibe555 commented Apr 7, 2026 •
edited
Loading

Part of #32385, Follow up PRs are coming.
The deletion of duplicates will be a separate PR.

Summary

  • Add processing_entities.py with ParentMode, PreProcessingRule, Segmentation, and Rule models
  • Add retrieval_settings.py with VectorSetting and KeywordSetting models
  • Add __init__.py re-exporting all RAG domain entities for convenient from core.rag.entities import ... usage

Why this change

VectorSetting, KeywordSetting, Rule, Segmentation, and related processing types are duplicated across knowledge_retrieval/entities.py, knowledge_index/entities.py, and rerank/entity/weight.py. Extracting them into a single shared location (core/rag/entities/) removes the duplication and establishes a canonical import path.

This PR only introduces the shared definitions. Consumer migration to the new import paths follows in a stacked PR.

Changes

  • core/rag/entities/__init__.py: New file — re-exports all entities
  • core/rag/entities/processing_entities.py: New file — ParentMode, PreProcessingRule, Segmentation, Rule
  • core/rag/entities/retrieval_settings.py: New file — VectorSetting, KeywordSetting

Test plan

  • ruff check passes

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. refactor labels Apr 7, 2026
@corevibe555 corevibe555 changed the title refactor(api): extract shared RAG domain entities into core/rag/entit… refactor(api): extract shared RAG domain entities into core/rag/entity Apr 7, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 14:57:09.815773790 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 14:57:01.283752433 +0000
@@ -299,6 +299,8 @@
    --> core/tools/workflow_as_tool/provider.py:237:26
 ERROR Returned type `dict[object, str]` is not assignable to declared return type `dict[str, Any] | None` [bad-return]
    --> core/workflow/human_input_compat.py:183:16
+ERROR Could not import `Condition` from `core.workflow.nodes.knowledge_retrieval.entities` [missing-module-attribute]
+  --> core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py:37:5
 ERROR No matching overload found for function `redis.client.Redis.__init__` called with arguments: (connection_pool=ConnectionPool) [no-matching-overload]
    --> extensions/ext_redis.py:244:38
 ERROR Cannot index into `Literal['']` [bad-index]
@@ -4952,6 +4954,8 @@
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1579:16
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
+ERROR Could not import `Condition` from `core.app.app_config.entities` [missing-module-attribute]
+  --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:14:5
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
 ERROR Could not find name `metadata_name` [unknown-name]
@@ -4968,13 +4972,13 @@
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
@@ -5767,6 +5771,8 @@
    --> tests/unit_tests/core/workflow/nodes/knowledge_index/test_knowledge_index_node.py:120:20
 ERROR Argument `dict[str, bool]` is not assignable to parameter `summary_index_setting` with type `SummaryIndexSettingDict | None` in function `core.workflow.nodes.knowledge_index.knowledge_index_node.KnowledgeIndexNode._invoke_knowledge_index` [bad-argument-type]
    --> tests/unit_tests/core/workflow/nodes/knowledge_index/test_knowledge_index_node.py:641:35
+ERROR Could not import `Condition` from `core.workflow.nodes.knowledge_retrieval.entities` [missing-module-attribute]
+  --> tests/unit_tests/core/workflow/nodes/knowledge_retrieval/test_knowledge_retrieval_node.py:13:5
 ERROR Argument `dict[str, dict[str, list[str] | str] | str]` is not assignable to parameter `config` with type `NodeConfigDict` in function `core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node.KnowledgeRetrievalNode.__init__` [bad-argument-type]
    --> tests/unit_tests/core/workflow/nodes/knowledge_retrieval/test_knowledge_retrieval_node.py:110:20
 ERROR Argument `dict[str, dict[str, Any] | str]` is not assignable to parameter `config` with type `NodeConfigDict` in function `core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node.KnowledgeRetrievalNode.__init__` [bad-argument-type]
@@ -6281,7 +6287,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

@corevibe555 corevibe555 force-pushed the refactor/rag-domain-entities-core branch from 9c541f6 to d6e7e25 Compare April 7, 2026 15:03
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 15:04:19.963923240 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 15:04:11.403966508 +0000
@@ -299,6 +299,8 @@
    --> core/tools/workflow_as_tool/provider.py:237:26
 ERROR Returned type `dict[object, str]` is not assignable to declared return type `dict[str, Any] | None` [bad-return]
    --> core/workflow/human_input_compat.py:183:16
+ERROR Could not import `Condition` from `core.workflow.nodes.knowledge_retrieval.entities` [missing-module-attribute]
+  --> core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py:37:5
 ERROR No matching overload found for function `redis.client.Redis.__init__` called with arguments: (connection_pool=ConnectionPool) [no-matching-overload]
    --> extensions/ext_redis.py:244:38
 ERROR Cannot index into `Literal['']` [bad-index]
@@ -4952,6 +4954,8 @@
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1579:16
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
+ERROR Could not import `Condition` from `core.app.app_config.entities` [missing-module-attribute]
+  --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:14:5
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
 ERROR Could not find name `metadata_name` [unknown-name]
@@ -4968,13 +4972,13 @@
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
@@ -5767,6 +5771,8 @@
    --> tests/unit_tests/core/workflow/nodes/knowledge_index/test_knowledge_index_node.py:120:20
 ERROR Argument `dict[str, bool]` is not assignable to parameter `summary_index_setting` with type `SummaryIndexSettingDict | None` in function `core.workflow.nodes.knowledge_index.knowledge_index_node.KnowledgeIndexNode._invoke_knowledge_index` [bad-argument-type]
    --> tests/unit_tests/core/workflow/nodes/knowledge_index/test_knowledge_index_node.py:641:35
+ERROR Could not import `Condition` from `core.workflow.nodes.knowledge_retrieval.entities` [missing-module-attribute]
+  --> tests/unit_tests/core/workflow/nodes/knowledge_retrieval/test_knowledge_retrieval_node.py:13:5
 ERROR Argument `dict[str, dict[str, list[str] | str] | str]` is not assignable to parameter `config` with type `NodeConfigDict` in function `core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node.KnowledgeRetrievalNode.__init__` [bad-argument-type]
    --> tests/unit_tests/core/workflow/nodes/knowledge_retrieval/test_knowledge_retrieval_node.py:110:20
 ERROR Argument `dict[str, dict[str, Any] | str]` is not assignable to parameter `config` with type `NodeConfigDict` in function `core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node.KnowledgeRetrievalNode.__init__` [bad-argument-type]
@@ -6281,7 +6287,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

@corevibe555 corevibe555 force-pushed the refactor/rag-domain-entities-core branch from d6e7e25 to 8c00c67 Compare April 7, 2026 15:28
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 15:30:03.124909517 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 15:29:54.998971881 +0000
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

This was referenced Apr 7, 2026
Merged
Merged
@corevibe555
Copy link
Copy Markdown
Contributor Author

corevibe555 commented Apr 7, 2026

@asukaminato0721 Here is the follow up PR. Would you review when you have a chance?
Thanks

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 17:01:47.001487254 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 17:01:38.221497881 +0000
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

@corevibe555 corevibe555 force-pushed the refactor/rag-domain-entities-core branch 2 times, most recently from 8c00c67 to f6f3889 Compare April 7, 2026 17:11
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 17:12:10.852635212 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 17:12:00.918594377 +0000
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 17:12:31.922984179 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 17:12:22.074852534 +0000
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

@corevibe555 corevibe555 force-pushed the refactor/rag-domain-entities-core branch from f6f3889 to e098652 Compare April 7, 2026 18:28
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Apr 7, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

base → PR 
--- /tmp/pyrefly_base.txt	2026-04-07 18:29:15.418148763 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 18:29:05.616168053 +0000
@@ -62,6 +62,8 @@
    --> core/rag/datasource/keyword/jieba/jieba.py:157:29
 ERROR Argument `object` is not assignable to parameter `iterable` with type `Iterable[@_]` in function `list.__init__` [bad-argument-type]
   --> core/rag/datasource/keyword/jieba/jieba_keyword_table_handler.py:88:35
+ERROR Could not import `MetadataCondition` from `core.rag.entities.metadata_entities` [missing-module-attribute]
+  --> core/rag/datasource/retrieval_service.py:18:49
 ERROR Class member `AnalyticdbVector.add_texts` overrides parent class `BaseVector` in an inconsistent manner [bad-override]
   --> core/rag/datasource/vdb/analyticdb/analyticdb_vector.py:43:9
 ERROR Argument `str | None` is not assignable to parameter `namespace_password` with type `str` in function `alibabacloud_gpdb20160503.models._create_namespace_request.CreateNamespaceRequest.__init__` [bad-argument-type]
@@ -287,6 +289,8 @@
    --> core/rag/index_processor/processor/qa_index_processor.py:204:33
 ERROR Object of class `Document` has no attribute `score` [missing-attribute]
    --> core/rag/index_processor/processor/qa_index_processor.py:205:16
+ERROR Could not import `MetadataCondition` from `core.rag.entities.metadata_entities` [missing-module-attribute]
+  --> core/rag/retrieval/dataset_retrieval.py:44:60
 ERROR No matching overload found for function `core.model_manager.ModelInstance.invoke_llm` called with arguments: (prompt_messages=list[SystemPromptMessage | UserPromptMessage], tools=list[PromptMessageTool], stream=Literal[False], model_parameters=dict[str, float | int]) [no-matching-overload]
   --> core/rag/retrieval/router/multi_dataset_function_call_router.py:32:58
 ERROR Class member `MCPToolProviderController.entity` overrides parent class `ToolProviderController` in an inconsistent manner [bad-override]
@@ -335,6 +339,8 @@
   --> services/document_indexing_proxy/duplicate_document_indexing_task_proxy.py:14:5
 ERROR Class member `DuplicateDocumentIndexingTaskProxy.PRIORITY_TASK_FUNC` overrides parent class `BatchDocumentIndexingProxy` in an inconsistent manner [bad-override]
   --> services/document_indexing_proxy/duplicate_document_indexing_task_proxy.py:15:5
+ERROR Could not import `MetadataCondition` from `core.rag.entities.metadata_entities` [missing-module-attribute]
+  --> services/external_knowledge_service.py:12:49
 ERROR Argument `RetrievalMethod | bool | dict[str, str] | int | Unknown` is not assignable to parameter `top_k` with type `int` in function `core.rag.datasource.retrieval_service.RetrievalService.retrieve` [bad-argument-type]
   --> services/hit_testing_service.py:86:19
 ERROR Argument `RetrievalMethod | bool | dict[str, str] | float | int | Unknown` is not assignable to parameter `score_threshold` with type `float | None` in function `core.rag.datasource.retrieval_service.RetrievalService.retrieve` [bad-argument-type]
@@ -4962,20 +4968,14 @@
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
-ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
     --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
@@ -6281,8 +6281,6 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
-   --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20
 ERROR Cannot index into `object` [bad-index]

@corevibe555 corevibe555 force-pushed the refactor/rag-domain-entities-core branch from e098652 to 3c9ff80 Compare April 7, 2026 18:39
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

No changes detected.

@corevibe555 corevibe555 mentioned this pull request Apr 7, 2026
Merged
1 task
@asukaminato0721 asukaminato0721 added this pull request to the merge queue Apr 7, 2026
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 7, 2026
Copilot AI reviewed Apr 7, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extracts shared RAG domain Pydantic models into core.rag.entities to establish a canonical import path and reduce duplication across retrieval/index/rerank components (consumer migration follows in stacked PRs).

Changes:

  • Added shared retrieval settings models (VectorSetting, KeywordSetting).
  • Added shared processing models (ParentMode, PreProcessingRule, Segmentation, Rule).
  • Added core.rag.entities package __init__.py to re-export selected entities.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
api/core/rag/entities/retrieval_settings.py Introduces shared retrieval weight/config models.
api/core/rag/entities/processing_entities.py Introduces shared preprocessing/segmentation/rule models for indexing.
api/core/rag/entities/init.py Adds a re-export surface for convenient imports from core.rag.entities.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread api/core/rag/entities/processing_entities.py
Comment on lines +23 to +27
class Rule(BaseModel):
pre_processing_rules: list[PreProcessingRule] | None = None
segmentation: Segmentation | None = None
parent_mode: Literal["full-doc", "paragraph"] | None = None
subchunk_segmentation: Segmentation | None = None
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ParentMode is defined as a StrEnum, but Rule.parent_mode is typed as Literal["full-doc", "paragraph"]. This duplicates the allowed values in two places and risks drift (e.g., updating the enum without updating the Literal). Consider typing parent_mode as ParentMode | None (or otherwise reusing ParentMode as the single source of truth) so validation/schema and typing stay consistent.

Copilot uses AI. Check for mistakes.
Comment thread api/core/rag/entities/__init__.py
Comment on lines +1 to +5
from core.rag.entities.citation_metadata import RetrievalSourceMetadata
from core.rag.entities.context_entities import DocumentContext
from core.rag.entities.processing_entities import ParentMode, PreProcessingRule, Rule, Segmentation
from core.rag.entities.retrieval_settings import KeywordSetting, VectorSetting

Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says this __init__.py re-exports “all RAG domain entities”, but this module only exports a subset (e.g., it does not re-export entities from event.py or metadata_entities.py which live in the same package). Either expand the re-exports to match the stated intent, or adjust the PR description/expectations so consumers don’t assume core.rag.entities is comprehensive.

Copilot uses AI. Check for mistakes.
Merged via the queue into langgenius:main with commit d2ee486 Apr 7, 2026
37 checks passed
HanqingZ pushed a commit to HanqingZ/dify that referenced this pull request Apr 23, 2026
@corevibe555 @HanqingZ
refactor(api): extract shared RAG domain entities into core/rag/entity (
0bc5165 
langgenius#34685)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@asukaminato0721 asukaminato0721 asukaminato0721 approved these changes

@JohnJyong JohnJyong Awaiting requested review from JohnJyong JohnJyong is a code owner

@laipz8200 laipz8200 Awaiting requested review from laipz8200

@QuantumGhost QuantumGhost Awaiting requested review from QuantumGhost

Assignees

No one assigned

Labels

lgtm This PR has been approved by a maintainer refactor size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@corevibe555 @asukaminato0721