Skip to content

fix(docker): preserve single-url crawl failure details - #2117

Open
nightcityblade wants to merge 1 commit into
unclecode:mainfrom
nightcityblade:fix/issue-2116
Open

fix(docker): preserve single-url crawl failure details#2117
nightcityblade wants to merge 1 commit into
unclecode:mainfrom
nightcityblade:fix/issue-2116

Conversation

@nightcityblade

Copy link
Copy Markdown
Contributor

Summary

Fixes #2116

Treat unsuccessful crawls from the single-URL /md and /llm/{url} handlers as upstream failures (502 Bad Gateway) so the existing exception handler preserves error_message instead of replacing it with an opaque internal-server-error response. The LLM handler now also preserves deliberate HTTPException statuses.

List of files changed and why

  • deploy/docker/api.py - centralize unsuccessful crawl handling and return a detail-preserving 502 from both single-URL handlers.
  • deploy/docker/tests/test_api_crawl_failures.py - verify the 502 mapping, preserved anti-bot reason, both handler call sites, and LLM exception propagation.

How Has This Been Tested?

  • .venv/bin/python -m pytest deploy/docker/tests/test_api_crawl_failures.py -q (3 passed)
  • .venv/bin/python -m pytest deploy/docker/tests/test_security_headers_xss.py::TestErrorSanitization -q (2 passed)
  • .venv/bin/python -m ruff check --select E9,F63,F7,F82 deploy/docker/api.py deploy/docker/tests/test_api_crawl_failures.py (passed)
  • Direct exception-handler smoke check confirmed a 502 response retains the anti-bot detail.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas (N/A: no complex logic was introduced)
  • I have made corresponding changes to the documentation (N/A: no configuration or usage change)
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@nightcityblade

Copy link
Copy Markdown
Contributor Author

CI note: the code-related posture-gate and security-offline checks both pass. The only failing job is the Discord notification workflow; its log shows an empty DISCORD_WEBHOOK and exits before exercising this change. This is repository/fork-secret configuration rather than a patch failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: /md and /llm/{url} return HTTP 500 when anti-bot detection marks a result failed

1 participant