test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs by bohdansolovie · Pull Request #35146 · langgenius/dify

bohdansolovie · 2026-04-14T07:18:17Z

Summary

Migrate test_clean_dataset_task.py from legacy SQLAlchemy Query API usage to SQLAlchemy 2.0 statement APIs.
Replace fixture cleanup deletes from Session.query(...).delete() to Session.execute(delete(...)).
Replace read assertions from query(...).filter_by(...).all() / query(...).where(...).all() to Session.scalars(select(...).where(...)).all().

Changes

Updated file:
- api/tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py
Added imports:
- from sqlalchemy import delete, select
Refactored:
- Database cleanup fixture delete operations.
- Dataset/document/segment/metadata/upload-file verification queries across multiple test cases.
- Read-only test assertions to use select/scalars patterns consistently.

Related Issue

Part of: use select style api in orm #22668

Replace remaining legacy Session.query usage in clean_dataset task integration tests with Session.execute(delete(...)) and select-based reads to align with SQLAlchemy 2.x migration work (langgenius#22668).

github-actions · 2026-04-14T07:19:55Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-14 07:19:40.830125314 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-14 07:19:31.072150059 +0000
@@ -1655,41 +1655,41 @@
 ERROR `>` is not supported between `None` and `Literal[0]` [unsupported-operation]
    --> tests/test_containers_integration_tests/tasks/test_batch_create_segment_to_index_task.py:708:16
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:112:20
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:113:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:122:20
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:123:20
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:297:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:298:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:300:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:301:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:409:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:420:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:412:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:423:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:477:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:494:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:480:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:497:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:532:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:551:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:535:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:554:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:617:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:640:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:620:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:643:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:730:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:757:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:733:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:760:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:816:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:849:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:819:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:852:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:971:32
+    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1004:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:974:22
+    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1007:22
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:90:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]

github-actions · 2026-04-14T07:21:39Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-14 07:21:27.631076732 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-14 07:21:17.550892653 +0000
@@ -1655,41 +1655,41 @@
 ERROR `>` is not supported between `None` and `Literal[0]` [unsupported-operation]
    --> tests/test_containers_integration_tests/tasks/test_batch_create_segment_to_index_task.py:708:16
 ERROR Argument `Literal['active']` is not assignable to parameter `status` with type `AccountStatus | SQLCoreOperations[AccountStatus]` in function `models.account.Account.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:112:20
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:113:20
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `SQLCoreOperations[TenantStatus] | TenantStatus` in function `models.account.Tenant.__init__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:122:20
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:123:20
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:297:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:298:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:300:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:301:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:409:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:422:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:412:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:425:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:477:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:498:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:480:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:501:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:532:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:555:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:535:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:558:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:617:36
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:644:36
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:620:26
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:647:26
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:730:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:761:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:733:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:764:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:816:32
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:855:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:819:22
+   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:858:22
 ERROR Argument `IndexTechniqueType | None` is not assignable to parameter `indexing_technique` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:971:32
+    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1012:32
 ERROR Argument `str | None` is not assignable to parameter `doc_form` with type `str` in function `celery.app.task.Task.__call__` [bad-argument-type]
-   --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:974:22
+    --> tests/test_containers_integration_tests/tasks/test_clean_dataset_task.py:1015:22
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]
   --> tests/test_containers_integration_tests/tasks/test_clean_notion_document_task.py:90:23
 ERROR Object of class `NoneType` has no attribute `id` [missing-attribute]

…35146) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

…anggenius#35146) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs

8fd37ec

Replace remaining legacy Session.query usage in clean_dataset task integration tests with Session.execute(delete(...)) and select-based reads to align with SQLAlchemy 2.x migration work (langgenius#22668).

dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Apr 14, 2026

[autofix.ci] apply automated fixes

934d709

autofix-ci Bot requested review from QuantumGhost and laipz8200 as code owners April 14, 2026 07:20

asukaminato0721 enabled auto-merge April 14, 2026 07:24

asukaminato0721 disabled auto-merge April 14, 2026 07:24

asukaminato0721 enabled auto-merge April 14, 2026 07:24

asukaminato0721 approved these changes Apr 14, 2026

View reviewed changes

asukaminato0721 added this pull request to the merge queue Apr 14, 2026

dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 14, 2026

Merged via the queue into langgenius:main with commit 173e0d6 Apr 14, 2026
28 checks passed

CodingOnStar pushed a commit that referenced this pull request Apr 14, 2026

test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs (#…

6c382dd

…35146) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

HanqingZ pushed a commit to HanqingZ/dify that referenced this pull request Apr 23, 2026

test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs (l…

279e48d

…anggenius#35146) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs#35146

test: migrate clean_dataset integration tests to SQLAlchemy 2.0 APIs#35146
asukaminato0721 merged 2 commits intolanggenius:mainfrom
bohdansolovie:chore/sqlalchemy2-clean-dataset-large-slice

bohdansolovie commented Apr 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 14, 2026

Uh oh!

github-actions Bot commented Apr 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bohdansolovie commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Related Issue

Uh oh!

github-actions Bot commented Apr 14, 2026

Pyrefly Diff

Uh oh!

github-actions Bot commented Apr 14, 2026

Pyrefly Diff

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bohdansolovie commented Apr 14, 2026 •

edited

Loading