refactor(api): remove duplicated RAG entities from services layer by corevibe555 · Pull Request #34689 · langgenius/dify

corevibe555 · 2026-04-07T16:08:03Z

Part of issue #32385.
Note: This PR can only be merged after following PRs: #34685
#34692

I will ask one question for all Pydantic models: what concept does the model describe(Not who imports it)?
Based on answer, I will put models in proper place.

Summary

Delete duplicated ParentMode, PreProcessingRule, Rule, Segmentation from knowledge_entities.py
Delete duplicated VectorSetting, KeywordSetting from rag_pipeline_entities.py
Migrate all consumers to import from core.rag.entities (canonical source from refactor(api): extract shared RAG domain entities into core/rag/entity #34685)

Why this change

These models were defined in the services/ layer but consumed by core/ and models/ — an upward dependency violation. #34685 extracted them into core/rag/entities/. This PR removes the now-redundant copies from services/ and updates all remaining consumers.

Changes

services/entities/knowledge_entities/knowledge_entities.py: Remove ParentMode, PreProcessingRule, Segmentation, Rule definitions, import Rule from canonical source
services/entities/knowledge_entities/rag_pipeline_entities.py: Remove VectorSetting, KeywordSetting definitions, import from canonical source
models/dataset.py: Import ParentMode, Rule from core.rag.entities
core/rag/index_processor/processor/*.py: Import Rule, ParentMode from core.rag.entities
services/vector_service.py: Import ParentMode from core.rag.entities
services/hit_testing_service.py: Import MetadataFilteringCondition from core.rag.entities
controllers/console/app/app.py: Import PreProcessingRule, Rule, Segmentation from core.rag.entities
controllers/service_api/dataset/document.py: Import PreProcessingRule, Rule, Segmentation from core.rag.entities
tests/.../dataset_service_test_helpers.py: Update imports
tests/.../document_service_validation.py: Update imports
tests/.../test_parent_child_index_processor.py: Update imports

Test plan

ruff check passes
Unit tests pass
Zero behavioral changes — only import paths updated

github-actions · 2026-04-07T16:11:12Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-07 16:10:52.691370000 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 16:10:44.085373548 +0000
@@ -1,5 +1,5 @@
 ERROR Object of class `App` has no attribute `access_mode` [missing-attribute]
-   --> controllers/console/app/app.py:562:13
+   --> controllers/console/app/app.py:560:13
 ERROR Argument `str | None` is not assignable to parameter `language` with type `str` in function `services.account_service.AccountService.send_email_register_email` [bad-argument-type]
   --> controllers/console/auth/email_register.py:78:108
 ERROR Object of class `MissingRouter` has no attribute `get` [missing-attribute]
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6264,7 +6264,7 @@
 ERROR Could not import `MetadataValue` from `services.entities.knowledge_entities.knowledge_entities` [missing-module-attribute]
   --> tests/unit_tests/services/dataset_metadata.py:95:5
 ERROR Argument `str` is not assignable to parameter `indexing_technique` with type `Literal['economy', 'high_quality']` in function `services.entities.knowledge_entities.rag_pipeline_entities.KnowledgeConfiguration.__init__` [bad-argument-type]
-   --> tests/unit_tests/services/dataset_service_test_helpers.py:449:28
+   --> tests/unit_tests/services/dataset_service_test_helpers.py:447:28
 ERROR Argument `Literal['']` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
    --> tests/unit_tests/services/document_indexing_task_proxy.py:831:114
 ERROR Argument `None` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

…ies/ fix fix refactor(api): extract shared RAG domain entities into core/rag/entities/ fix fix fix

github-actions · 2026-04-07T16:31:01Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-07 16:30:43.169326782 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 16:30:34.673523741 +0000
@@ -1,5 +1,5 @@
 ERROR Object of class `App` has no attribute `access_mode` [missing-attribute]
-   --> controllers/console/app/app.py:562:13
+   --> controllers/console/app/app.py:560:13
 ERROR Argument `str | None` is not assignable to parameter `language` with type `str` in function `services.account_service.AccountService.send_email_register_email` [bad-argument-type]
   --> controllers/console/auth/email_register.py:78:108
 ERROR Object of class `MissingRouter` has no attribute `get` [missing-attribute]
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6264,7 +6264,7 @@
 ERROR Could not import `MetadataValue` from `services.entities.knowledge_entities.knowledge_entities` [missing-module-attribute]
   --> tests/unit_tests/services/dataset_metadata.py:95:5
 ERROR Argument `str` is not assignable to parameter `indexing_technique` with type `Literal['economy', 'high_quality']` in function `services.entities.knowledge_entities.rag_pipeline_entities.KnowledgeConfiguration.__init__` [bad-argument-type]
-   --> tests/unit_tests/services/dataset_service_test_helpers.py:449:28
+   --> tests/unit_tests/services/dataset_service_test_helpers.py:447:28
 ERROR Argument `Literal['']` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
    --> tests/unit_tests/services/document_indexing_task_proxy.py:831:114
 ERROR Argument `None` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

github-actions · 2026-04-07T23:27:19Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-07 23:27:04.338256065 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-07 23:26:55.098251284 +0000
@@ -1,5 +1,5 @@
 ERROR Object of class `App` has no attribute `access_mode` [missing-attribute]
-   --> controllers/console/app/app.py:562:13
+   --> controllers/console/app/app.py:560:13
 ERROR Argument `str | None` is not assignable to parameter `language` with type `str` in function `services.account_service.AccountService.send_email_register_email` [bad-argument-type]
   --> controllers/console/auth/email_register.py:78:108
 ERROR Object of class `MissingRouter` has no attribute `get` [missing-attribute]
@@ -4953,39 +4953,39 @@
 ERROR Object of class `FunctionType` has no attribute `call_count` [missing-attribute]
     --> tests/unit_tests/core/rag/rerank/test_reranker.py:1630:16
 ERROR Argument `list[float] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1951:20
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:1949:20
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2771:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2769:29
 ERROR Could not find name `metadata_name` [unknown-name]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2772:29
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:2770:29
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3784:64
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3782:64
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval._handle_invoke_result` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3788:67
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:3786:67
 ERROR `None` is not subscriptable [unsupported-operation]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4027:16
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4025:16
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4561:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4559:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4613:40
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4618:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4611:40
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4616:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4634:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4632:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4664:36
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4669:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4662:36
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4667:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.single_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4677:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4675:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4713:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4711:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4741:36
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4739:36
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4799:40
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4797:40
 ERROR Argument `list[SimpleNamespace]` is not assignable to parameter `available_datasets` with type `list[Dataset]` in function `core.rag.retrieval.dataset_retrieval.DatasetRetrieval.multiple_retrieve` [bad-argument-type]
-    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4843:44
+    --> tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py:4841:44
 ERROR Argument `Iterator[Any | Unknown] | Iterator[Any]` is not assignable to parameter `invoke_result` with type `Generator[Unknown]` in function `core.rag.retrieval.router.multi_dataset_react_route.ReactMultiDatasetRouter._handle_invoke_result` [bad-argument-type]
    --> tests/unit_tests/core/rag/retrieval/test_multi_dataset_react_route.py:199:52
 ERROR Argument `None` is not assignable to parameter `text` with type `str` in function `core.rag.splitter.text_splitter.RecursiveCharacterTextSplitter.split_text` [bad-argument-type]
@@ -6264,7 +6264,7 @@
 ERROR Could not import `MetadataValue` from `services.entities.knowledge_entities.knowledge_entities` [missing-module-attribute]
   --> tests/unit_tests/services/dataset_metadata.py:95:5
 ERROR Argument `str` is not assignable to parameter `indexing_technique` with type `Literal['economy', 'high_quality']` in function `services.entities.knowledge_entities.rag_pipeline_entities.KnowledgeConfiguration.__init__` [bad-argument-type]
-   --> tests/unit_tests/services/dataset_service_test_helpers.py:449:28
+   --> tests/unit_tests/services/dataset_service_test_helpers.py:447:28
 ERROR Argument `Literal['']` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
    --> tests/unit_tests/services/document_indexing_task_proxy.py:831:114
 ERROR Argument `None` is not assignable to parameter `plan` with type `CloudPlan` in function `DocumentIndexingTaskProxyTestDataFactory.create_mock_features` [bad-argument-type]
@@ -6281,7 +6281,7 @@
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:60:25
 ERROR Argument `None` is not assignable to parameter `response` with type `Response` in function `httpx._exceptions.HTTPStatusError.__init__` [bad-argument-type]
   --> tests/unit_tests/services/enterprise/test_plugin_manager_service.py:61:26
-ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
+ERROR Argument `SimpleNamespace` is not assignable to parameter `metadata_condition` with type `MetadataFilteringCondition | None` in function `services.external_knowledge_service.ExternalDatasetService.fetch_external_knowledge_retrieval` [bad-argument-type]
    --> tests/unit_tests/services/external_dataset_service.py:851:36
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/unit_tests/services/hit_service.py:430:20

…nggenius#34689)

corevibe555 requested review from JohnJyong, QuantumGhost and laipz8200 as code owners April 7, 2026 16:08

dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. refactor labels Apr 7, 2026

corevibe555 changed the title ~~Refactor/rag domain entities services~~ refactor(api): remove duplicated RAG entities from services layer Apr 7, 2026

refactor(api): extract shared RAG domain entities into core/rag/entit…

fbfa410

…ies/ fix fix refactor(api): extract shared RAG domain entities into core/rag/entities/ fix fix fix

corevibe555 force-pushed the refactor/rag-domain-entities-services branch from 2332f41 to fbfa410 Compare April 7, 2026 16:29

corevibe555 mentioned this pull request Apr 7, 2026

refactor(api): deduplicate RAG index entities and consolidate import paths #34690

Merged

1 task

asukaminato0721 enabled auto-merge April 7, 2026 23:20

asukaminato0721 previously approved these changes Apr 7, 2026

View reviewed changes

dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 7, 2026

Merge branch 'main' into refactor/rag-domain-entities-services

a570e32

corevibe555 dismissed asukaminato0721’s stale review via a570e32 April 7, 2026 23:26

asukaminato0721 self-requested a review April 7, 2026 23:36

asukaminato0721 approved these changes Apr 7, 2026

View reviewed changes

asukaminato0721 added this pull request to the merge queue Apr 7, 2026

Merged via the queue into langgenius:main with commit 624db69 Apr 7, 2026
27 checks passed

HanqingZ pushed a commit to HanqingZ/dify that referenced this pull request Apr 23, 2026

refactor(api): remove duplicated RAG entities from services layer (la…

bd7b05f

…nggenius#34689)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(api): remove duplicated RAG entities from services layer#34689

refactor(api): remove duplicated RAG entities from services layer#34689
asukaminato0721 merged 2 commits intolanggenius:mainfrom
corevibe555:refactor/rag-domain-entities-services

corevibe555 commented Apr 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 7, 2026

Uh oh!

github-actions Bot commented Apr 7, 2026

Uh oh!

github-actions Bot commented Apr 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

corevibe555 commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why this change

Changes

Test plan

Uh oh!

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

Uh oh!

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

Uh oh!

github-actions Bot commented Apr 7, 2026

Pyrefly Diff

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

corevibe555 commented Apr 7, 2026 •

edited

Loading