chore: DocumentSegment to Typebase by asukaminato0721 · Pull Request #35635 · langgenius/dify

asukaminato0721 · 2026-04-28T06:36:40Z

Important

Make sure you have read our contribution guidelines
Ensure there is an associated issue and you have been assigned to it
Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

Screenshots

Before	After
...	...

Checklist

This change requires a documentation update, included: Dify Document
I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.
I ran make lint && make type-check (backend) and cd web && pnpm exec vp staged (frontend) to appease the lint gods

github-actions · 2026-05-07T18:32:18Z

Pyrefly Type Coverage

Metric	Base	PR	Delta
Type coverage	0.00%	43.67%	+43.67%
Strict coverage	0.00%	43.20%	+43.20%
Typed symbols	0	22,040	+22,040
Untyped symbols	0	28,742	+28,742
Modules	0	2551	+2,551

github-actions · 2026-05-11T05:08:35Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-05-12 06:26:21.175751462 +0000
+++ /tmp/pyrefly_pr.txt	2026-05-12 06:26:13.024860963 +0000
@@ -1043,10 +1043,6 @@
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/services/test_dataset_service_delete_dataset.py:32:20
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
-  --> tests/test_containers_integration_tests/services/test_dataset_service_get_segments.py:38:20
-ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
-  --> tests/test_containers_integration_tests/services/test_dataset_service_get_segments.py:44:62
-ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/services/test_dataset_service_permissions.py:38:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/services/test_dataset_service_permissions.py:40:58
@@ -1506,10 +1502,6 @@
   --> tests/test_containers_integration_tests/tasks/test_batch_clean_document_task.py:73:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/tasks/test_batch_clean_document_task.py:82:20
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_batch_clean_document_task.py:297:23
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_batch_clean_document_task.py:698:27
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
    --> tests/test_containers_integration_tests/tasks/test_batch_create_segment_to_index_task.py:101:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
@@ -1523,131 +1515,111 @@
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:123:20
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:298:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:295:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:301:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:298:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:422:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:419:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:425:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:422:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:498:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:495:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:501:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:498:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:555:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:552:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:558:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:555:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:644:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:638:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:647:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:641:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:761:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:751:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:764:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:754:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:855:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:845:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:858:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:848:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1012:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:999:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1015:22
+    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1002:22
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:100:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:117:27
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:140:31
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:222:23
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:271:27
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:283:27
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:304:27
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:354:23
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:366:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:221:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:388:27
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:270:27
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:437:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:282:27
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:454:27
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:352:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:476:31
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:364:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:539:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:434:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:551:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:451:27
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:576:27
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:535:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:640:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:547:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:652:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:635:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:672:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:647:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:733:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:727:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:751:27
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:774:31
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:745:27
 ERROR `num_segments_per_doc` may be uninitialized [unbound-name]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:795:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:788:32
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:839:27
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:832:27
 ERROR `>=` is not supported between `None` and `Literal[3]` [unsupported-operation]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:898:16
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:890:16
 ERROR `>=` is not supported between `None` and `Literal[9]` [unsupported-operation]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:899:16
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:944:23
-ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:970:27
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:891:16
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:992:31
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:936:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:1050:23
+   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:962:27
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:1063:23
+    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:1041:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:1100:27
+    --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:1054:23
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:229:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:226:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:332:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:327:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:443:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:436:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:578:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:568:16
 ERROR Object of class `NoneType` has no attribute `error` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:579:52
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:569:52
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:651:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:639:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:729:16
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:715:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:835:20
+   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:922:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:939:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1032:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1051:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1038:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1057:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1153:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1174:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1159:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1180:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1274:16
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1297:16
-ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1303:16
+    --> tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py:1280:16
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/tasks/test_disable_segment_from_index_task.py:56:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
@@ -1711,55 +1683,55 @@
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:93:20
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:225:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:224:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:232:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:231:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:327:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:326:20
 ERROR Object of class `NoneType` has no attribute `processing_started_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:328:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:327:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:379:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:378:20
 ERROR Object of class `NoneType` has no attribute `processing_started_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:380:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:379:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:448:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:447:20
 ERROR Object of class `NoneType` has no attribute `processing_started_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:449:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:448:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:495:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:494:20
 ERROR Object of class `NoneType` has no attribute `processing_started_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:496:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:495:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:553:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:552:20
 ERROR Object of class `NoneType` has no attribute `error` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:554:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:553:20
 ERROR Object of class `object` has no attribute `lower` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:555:38
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:554:38
 ERROR Object of class `NoneType` has no attribute `stopped_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:556:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:555:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:595:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:594:20
 ERROR Object of class `NoneType` has no attribute `error` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:596:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:595:20
 ERROR Object of class `object` has no attribute `lower` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:597:31
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:596:31
 ERROR Object of class `NoneType` has no attribute `stopped_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:598:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:597:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:659:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:658:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:702:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:701:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:746:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:745:20
 ERROR Object of class `NoneType` has no attribute `is_paused` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:861:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:860:20
 ERROR Object of class `NoneType` has no attribute `indexing_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:862:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:861:20
 ERROR Object of class `NoneType` has no attribute `display_status` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:863:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:862:20
 ERROR Object of class `NoneType` has no attribute `processing_started_at` [missing-attribute]
-   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:864:20
+   --> tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py:863:20
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
   --> tests/test_containers_integration_tests/tasks/test_enable_segments_to_index_task.py:55:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
@@ -6043,18 +6015,6 @@
    --> tests/unit_tests/services/test_dataset_service_segment.py:182:80
 ERROR Argument `SimpleNamespace` is not assignable to parameter `dataset` with type `Dataset` in function `services.dataset_service.SegmentService.delete_child_chunk` [bad-argument-type]
    --> tests/unit_tests/services/test_dataset_service_segment.py:205:64
-ERROR Argument `list[Unknown] | None` is not assignable to parameter `obj` with type `Sized` in function `len` [bad-argument-type]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:465:20
-ERROR Type `None` is not iterable [not-iterable]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:466:49
-ERROR Type `None` is not iterable [not-iterable]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:467:47
-ERROR Type `None` is not iterable [not-iterable]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:468:61
-ERROR Type `None` is not iterable [not-iterable]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:469:60
-ERROR Type `None` is not iterable [not-iterable]
-   --> tests/unit_tests/services/test_dataset_service_segment.py:470:68
 ERROR Argument `SimpleNamespace` is not assignable to parameter `segment` with type `DocumentSegment` in function `services.dataset_service.SegmentService.update_child_chunk` [bad-argument-type]
    --> tests/unit_tests/services/test_dataset_service_segment.py:811:49
 ERROR Argument `SimpleNamespace` is not assignable to parameter `document` with type `Document` in function `services.dataset_service.SegmentService.update_child_chunk` [bad-argument-type]

Copilot

Pull request overview

This PR migrates the DocumentSegment ORM model to TypeBase (SQLAlchemy MappedAsDataclass) and updates tasks/tests to accommodate the new typed/dataclass initialization patterns, including safer handling of nullable index_node_id when cleaning vector indexes.

Changes:

Convert DocumentSegment to TypeBase and retype key columns (including status as SegmentStatus).
Update indexing/cleanup tasks to ignore None index_node_id values when building index_node_ids.
Adjust unit/integration tests to stop passing id= during DocumentSegment construction and switch status strings to enums in several places.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
api/tests/unit_tests/services/test_dataset_service_segment.py	Updates unit tests to work with `DocumentSegment` dataclass-style init (no `id=` in ctor).
api/tests/test_containers_integration_tests/tasks/test_enable_segments_to_index_task.py	Removes explicit `id=` in segment factory to align with new model defaults.
api/tests/test_containers_integration_tests/tasks/test_duplicate_document_indexing_task.py	Removes explicit `id=` for segments created in integration tests.
api/tests/test_containers_integration_tests/tasks/test_disable_segments_from_index_task.py	Refactors segment creation to constructor-based init; minor type-related adjustments.
api/tests/test_containers_integration_tests/tasks/test_disable_segment_from_index_task.py	Changes helper signature to use `SegmentStatus` enum instead of string.
api/tests/test_containers_integration_tests/tasks/test_delete_segment_from_index_task.py	Removes explicit `id=` during segment creation; adjusts test data setup.
api/tests/test_containers_integration_tests/tasks/test_deal_dataset_vector_index_task.py	Removes explicit `id=` from `Dataset`/`DocumentSegment` constructors where defaults exist.
api/tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py	Removes explicit `id=` on segments and adds some assertions in test setup.
api/tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py	Removes explicit `id=` on segments; test setup adjusted for new model init.
api/tests/test_containers_integration_tests/tasks/test_batch_clean_document_task.py	Removes explicit `id=` on segments and adds tenant assertions for setup typing.
api/tests/test_containers_integration_tests/tasks/test_add_document_to_index_task.py	Removes explicit `id=` on segments created in integration tests.
api/tests/test_containers_integration_tests/services/test_dataset_service_get_segments.py	Switches various status fields from strings to enums; expands imports accordingly.
api/tasks/sync_website_document_indexing_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/tasks/duplicate_document_indexing_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/tasks/document_indexing_sync_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/tasks/disable_segments_from_index_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/tasks/clean_document_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/tasks/batch_create_segment_to_index_task.py	Uses `SegmentStatus.COMPLETED` instead of `"completed"` string for new segments.
api/tasks/batch_clean_document_task.py	Filters out `None` `index_node_id` values before cleaning index.
api/services/vector_service.py	Adds assertions around `segment.index_node_id` before vector/keyword operations and child chunk generation.
api/models/dataset.py	Converts `DocumentSegment` to `TypeBase` and redefines columns with dataclass-friendly typing/defaults.
api/core/rag/index_processor/index_processor.py	Filters out `None` `index_node_id` values before cleaning index.
api/core/rag/docstore/dataset_docstore.py	Adds an assertion for `_document_id` prior to creating `DocumentSegment`.
api/core/rag/datasource/retrieval_service.py	Tweaks typing around `index_node_segments` and adds assertions; imports `Sequence`.
api/core/rag/datasource/keyword/jieba/jieba.py	Adds assertions ensuring `segment.index_node_id` exists before keyword table updates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kick off

b62d1ee

asukaminato0721 changed the title ~~chore: DocumentSegment to Tyoebase~~ chore: DocumentSegment to Typebase Apr 28, 2026

autofix-ci Bot and others added 13 commits April 28, 2026 06:38

[autofix.ci] apply automated fixes

bfda33f

.

7da5955

[autofix.ci] apply automated fixes

76cba4e

assert some

007fb48

mv

023479a

[autofix.ci] apply automated fixes

470dd92

.

820c3f2

default

12fbac3

.

33b0e16

[autofix.ci] apply automated fixes

6255c0a

.

e714480

[autofix.ci] apply automated fixes

0931d24

Merge branch 'main' into typebase-DocumentSegment

6e36ec7

asukaminato0721 force-pushed the typebase-DocumentSegment branch from b0a44dd to 6e36ec7 Compare May 7, 2026 18:29

langgenius deleted a comment from github-actions Bot May 11, 2026

Merge branch 'main' into typebase-DocumentSegment

c9bdc31

asukaminato0721 and others added 4 commits May 12, 2026 14:18

?

73fcdcc

.

8d8181c

.

f83b6eb

[autofix.ci] apply automated fixes

9c3d9c3

asukaminato0721 marked this pull request as ready for review May 12, 2026 05:32

Copilot AI review requested due to automatic review settings May 12, 2026 05:32

asukaminato0721 requested review from JohnJyong, QuantumGhost and laipz8200 as code owners May 12, 2026 05:32

dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. refactor labels May 12, 2026

asukaminato0721 enabled auto-merge May 12, 2026 05:32

Copilot started reviewing on behalf of asukaminato0721 May 12, 2026 05:32 View session

Copilot AI reviewed May 12, 2026

View reviewed changes

asukaminato0721 and others added 3 commits May 12, 2026 14:45

.

df0c146

[autofix.ci] apply automated fixes

fcb7d8b

.

1356574

dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels May 12, 2026

WH-2099 approved these changes May 12, 2026

View reviewed changes

asukaminato0721 added this pull request to the merge queue May 12, 2026

dosubot Bot added the lgtm This PR has been approved by a maintainer label May 12, 2026

Merged via the queue into langgenius:main with commit 51a8f79 May 12, 2026
27 checks passed

asukaminato0721 deleted the typebase-DocumentSegment branch May 12, 2026 07:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: DocumentSegment to Typebase#35635

chore: DocumentSegment to Typebase#35635
asukaminato0721 merged 22 commits into
langgenius:mainfrom
asukaminato0721:typebase-DocumentSegment

asukaminato0721 commented Apr 28, 2026

Uh oh!

github-actions Bot commented May 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

asukaminato0721 commented Apr 28, 2026

Summary

Screenshots

Checklist

Uh oh!

github-actions Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pyrefly Type Coverage

Uh oh!

github-actions Bot commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pyrefly Diff

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented May 7, 2026 •

edited

Loading

github-actions Bot commented May 11, 2026 •

edited

Loading