Skip to content

fix(rss): detect empty or HTML responses feedparser treats as non-bozo (#36) - #69

Open
1012839419a-alt wants to merge 1 commit into
2233admin:mainfrom
1012839419a-alt:fix/rss-empty-html-detection
Open

fix(rss): detect empty or HTML responses feedparser treats as non-bozo (#36)#69
1012839419a-alt wants to merge 1 commit into
2233admin:mainfrom
1012839419a-alt:fix/rss-empty-html-detection

Conversation

@1012839419a-alt

Copy link
Copy Markdown
Contributor

Closes #36

问题

feedparser 对空响应体 / HTML 错误页返回 bozo=False, entries=[],与合法空 feed 无法区分(已用 feedparser 6.0.12 实测复现:空 body → bozo=False, version=None;HTML 错误页 → bozo=False, version='')。采集表现为"成功的空 feed",无结构化 error_type 到达 control recorder,SCHEMA_DRIFT 传感器不触发。

修复

backend/channels/rss_channel.py

  • 新增 _parsed_version() / _non_feed_response() 辅助函数
  • collect()fetch() 在既有 W1 bozo 分支之后,检测"零条目且 feedparser 未识别任何 feed 格式(version 为空/None)"的响应,显式失败并携带 error_type="ParseError"error_kinds.py 映射 SCHEMA_DRIFT)
  • 保守设计:合法空 feed(rss20/atom10,version 非空)不受影响;部分解析出条目的 feed 不受影响
  • body-shape 启发式(feedparser version 字段作为判据)已在辅助函数 docstring 记录

与 WIRING_GAP_LEDGER W1(#35 / #60 已修)互补:W1 覆盖 bozo=True 的解析失败,本 PR 覆盖 bozo=False 的响应质量问题,两者都是 SCHEMA_DRIFT 链路的输入。

测试

  • tests/unit/channels/test_rss_channel_schema_drift.py +4:HTML 错误页 / 空 body / 非 feed 合法 XML → ParseError→SCHEMA_DRIFT;合法空 feed → 成功
  • tests/unit/channels/test_rss_fetch.py +3:fetch 路径 HTML / 空 body → ChannelFetchError(ParseError);合法空 feed → 成功
  • pytest tests/unit/channels/ tests/unit/test_workflow_rss_source_executor.py:285 passed
  • ruff check:通过

2233admin#36)

feedparser reports bozo=False with entries=[] for an empty HTTP 200 body
or an HTML error page, making them look like a healthy empty feed (no
structured error reaches control, SCHEMA_DRIFT never fires). Fail
explicitly with a structured ParseError (maps to SCHEMA_DRIFT in
error_kinds.py) when no feed version was detected, on both collect() and
fetch() paths. Conservative: a real zero-entry feed still carries a
detected version ('rss20'/'atom10') and passes.
@repowise-bot

repowise-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

⚠️ Health of changed files: 6.1 → 6.0 (-0.1)
⚠️ Change risk: moderate, riskier than 58% of this repo's commits.

📋 At a glance
2 files changed health · 3 hotspots touched · 4 new findings introduced · 2 files with recent fix history.

Files & modules (2)
  • backend (1 file)
    • backend/channels/rss_channel.py
  • tests (2 files)
    • .../channels/test_rss_channel_schema_drift.py
    • .../channels/test_rss_fetch.py

✅ Health gate: passed

📌 Before you merge

  • Run .../channels/test_rss_channel.py, .../channels/test_rss_channel_entries.py, .../channels/test_rss_channel_errors.py: they import the changed files
🔎 More signals (2)

🗺️ Change map

flowchart LR
  subgraph PR ["Changed in this PR (1 with dependents)"]
    f_backend_channels_rss_channel_py["backend/channels/rss_channel.py 🔥"]:::changed
  end
  f_backend_channels_api_channel_py["backend/channels/api_channel.py"]
  f_backend_channels_rss_channel_py --> f_backend_channels_api_channel_py
  f_backend_channels_registry_py["backend/channels/registry.py"]
  f_backend_channels_rss_channel_py --> f_backend_channels_registry_py
  f_backend_workflow_rss_source_executor_py["backend/workflow/rss_source_executor.py"]
  f_backend_channels_rss_channel_py --> f_backend_workflow_rss_source_executor_py
  t_tests_unit_channels_test_rss_channel_py(["✅ .../channels/test_rss_channel.py"]):::guard
  t_tests_unit_channels_test_rss_channel_py -.-> f_backend_channels_rss_channel_py
  classDef changed fill:#dbeafe,stroke:#1d4ed8,color:#1e3a5f
  classDef warn fill:#fef3c7,stroke:#b45309,color:#78350f
  classDef guard fill:#dcfce7,stroke:#15803d,color:#14532d
Loading

Solid arrows: code that imports the changed files (3 direct dependents, from the last indexed snapshot). Dashed: history/tests.

🔥 Hotspots touched (3)

  • .../channels/test_rss_channel_schema_drift.py: 1 commits/90d, 0 dependents · primary owner: 2233admin (100%)
  • .../channels/test_rss_fetch.py: 7 commits/90d, 0 dependents · primary owner: Curry (90%)
  • backend/channels/rss_channel.py: 9 commits/90d, 11 dependents · primary owner: Curry (61%)

👀 Suggested reviewers @2233admin


📊 See the full report for this PR
Your repo map with this PR's blast radius lit up, every caller of the contracts it changes, and health before and after. No sign-in. · ⭐ Star Repowise · 📥 Install bot · Silence on a single PR with [skip repowise] in the title · Per-repo toggle on repowise.dev/settings?tab=bot · Updated 2026-08-07 16:54 UTC

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@1012839419a-alt, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7f1975a2-83ca-4ae8-aa06-6c354a26be02

📥 Commits

Reviewing files that changed from the base of the PR and between 94ab53d and c2487ea.

📒 Files selected for processing (3)
  • backend/channels/rss_channel.py
  • tests/unit/channels/test_rss_channel_schema_drift.py
  • tests/unit/channels/test_rss_fetch.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(rss): detect empty or HTML responses that feedparser treats as non-bozo

1 participant