refactor(api): add null safety to extract_processor and firecrawl retry method by tmimmanuel · Pull Request #34809 · langgenius/dify

tmimmanuel · 2026-04-09T04:51:33Z

Summary

Fix upload_file possibly-unbound variable in extract_processor.py by
binding it before the file_path check and adding assert guards at PDF/DOCX
extractor call sites
Fix response possibly-unbound variable in firecrawl_app.py by initializing
before retry loops with assert after loop completion

Test plan

make type-check-core passes with 0 errors
All 194 extractor unit tests pass (no test changes needed)
No runtime behavior changes — null safety assertions only

Part of #26412

…ry method

github-actions · 2026-04-09T04:53:09Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-09 04:52:52.931814192 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-09 04:52:43.049788766 +0000
@@ -247,18 +247,6 @@
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:47
 ERROR Argument `list[float] | list[list[float]] | None` is not assignable to parameter `vector` with type `Iterable[LaxFloat] | None` in function `core.rag.models.document.Document.__init__` [bad-argument-type]
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:60
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:116:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:126:62
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:152:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:158:62
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:183:16
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:192:16
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.post` [bad-argument-type]
    --> core/rag/extractor/notion_extractor.py:106:25
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.request` [bad-argument-type]

tmimmanuel · 2026-04-10T05:02:15Z

@asukaminato0721. Please review.
(#34796 #34702 also)

github-actions · 2026-04-10T17:49:38Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-10 17:49:21.726426926 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-10 17:49:13.049322650 +0000
@@ -259,18 +259,6 @@
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:47
 ERROR Argument `list[float] | list[list[float]] | None` is not assignable to parameter `vector` with type `Iterable[LaxFloat] | None` in function `core.rag.models.document.Document.__init__` [bad-argument-type]
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:60
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:116:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:126:62
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:152:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:158:62
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:183:16
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:192:16
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.post` [bad-argument-type]
    --> core/rag/extractor/notion_extractor.py:106:25
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.request` [bad-argument-type]

github-actions · 2026-04-10T17:54:04Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-10 17:53:51.263669690 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-10 17:53:42.871714294 +0000
@@ -259,18 +259,6 @@
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:47
 ERROR Argument `list[float] | list[list[float]] | None` is not assignable to parameter `vector` with type `Iterable[LaxFloat] | None` in function `core.rag.models.document.Document.__init__` [bad-argument-type]
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:60
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:116:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:126:62
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:152:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:158:62
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:183:16
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:192:16
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.post` [bad-argument-type]
    --> core/rag/extractor/notion_extractor.py:106:25
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.request` [bad-argument-type]

github-actions · 2026-04-13T03:07:02Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-13 03:06:48.115322433 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-13 03:06:39.979375721 +0000
@@ -259,18 +259,6 @@
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:47
 ERROR Argument `list[float] | list[list[float]] | None` is not assignable to parameter `vector` with type `Iterable[LaxFloat] | None` in function `core.rag.models.document.Document.__init__` [bad-argument-type]
    --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:467:60
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:116:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:126:62
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:152:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:158:62
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:183:16
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:192:16
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.post` [bad-argument-type]
    --> core/rag/extractor/notion_extractor.py:106:25
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.request` [bad-argument-type]

github-actions · 2026-04-14T05:20:09Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-04-14 05:19:55.064381593 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-14 05:19:44.457377414 +0000
@@ -78,18 +78,6 @@
    --> core/rag/datasource/keyword/jieba/jieba.py:157:29
 ERROR Argument `object` is not assignable to parameter `iterable` with type `Iterable[@_]` in function `list.__init__` [bad-argument-type]
   --> core/rag/datasource/keyword/jieba/jieba_keyword_table_handler.py:88:35
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:116:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:126:62
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:152:61
-ERROR `upload_file` may be uninitialized [unbound-name]
-   --> core/rag/extractor/extract_processor.py:158:62
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:183:16
-ERROR `response` may be uninitialized [unbound-name]
-   --> core/rag/extractor/firecrawl/firecrawl_app.py:192:16
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.post` [bad-argument-type]
    --> core/rag/extractor/notion_extractor.py:106:25
 ERROR Argument `dict[str, bytes | str]` is not assignable to parameter `headers` with type `Headers | Mapping[bytes, bytes] | Mapping[str, str] | Sequence[tuple[bytes, bytes]] | Sequence[tuple[str, str]] | None` in function `httpx._api.request` [bad-argument-type]

refactor(api): add null safety to extract_processor and firecrawl ret…

2dc81d0

…ry method

tmimmanuel requested a review from JohnJyong as a code owner April 9, 2026 04:51

dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Apr 9, 2026

Merge branch 'main' into refactor/pyright-core-rag-extractor-null-safety

fd7a570

Merge branch 'main' into refactor/pyright-core-rag-extractor-null-safety

6dc9b42

Merge branch 'main' into refactor/pyright-core-rag-extractor-null-safety

e14bf8d

Merge branch 'main' into refactor/pyright-core-rag-extractor-null-safety

6a2ed09

This was referenced Apr 14, 2026

refactor(api): fix pyright errors in jieba, milvus, couchbase, oracle, and router #34938

Open

refactor(api): add null safety to extractor_processor and firecrawl #35209

Merged

refactor(api): tighten core rag typing batch 1 #35210

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(api): add null safety to extract_processor and firecrawl retry method#34809

refactor(api): add null safety to extract_processor and firecrawl retry method#34809
tmimmanuel wants to merge 5 commits intolanggenius:mainfrom
tmimmanuel:refactor/pyright-core-rag-extractor-null-safety

tmimmanuel commented Apr 9, 2026

Uh oh!

github-actions bot commented Apr 9, 2026

Uh oh!

tmimmanuel commented Apr 10, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 10, 2026

Uh oh!

github-actions bot commented Apr 10, 2026

Uh oh!

github-actions bot commented Apr 13, 2026

Uh oh!

github-actions bot commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

tmimmanuel commented Apr 9, 2026

Summary

Test plan

Uh oh!

github-actions bot commented Apr 9, 2026

Pyrefly Diff

Uh oh!

tmimmanuel commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 10, 2026

Pyrefly Diff

Uh oh!

github-actions bot commented Apr 10, 2026

Pyrefly Diff

Uh oh!

github-actions bot commented Apr 13, 2026

Pyrefly Diff

Uh oh!

github-actions bot commented Apr 14, 2026

Pyrefly Diff

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tmimmanuel commented Apr 10, 2026 •

edited

Loading