Fix 403 link warnings#2558
Merged
DerekMelchin merged 2 commits intoJun 22, 2026
Merged
Conversation
Many third-party sites (FINRA, Coinbase, Bitfinex, CoinAPI, BLS, ByBit, ...) return 401/403 to plain HTTP clients based on TLS/header fingerprinting even though the links are valid. Re-check those responses with curl_cffi (Chrome TLS impersonation) before warning, so bot-blocked-but-valid links no longer generate noise. A 429 is treated as reachable (server is live); 404/410 stays flagged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the browser re-check of a 401/403 reveals a real 404/410, record it as a 404 error instead of a 403 warning, so a genuinely dead link can't hide behind a bot-block and pass CI. Links that still can't be verified stay non-failing warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Re-check 401/403 links with a browser-impersonating client (curl_cffi) before warning. Bot-blocked-but-valid links are cleared; a browser-confirmed 404 is escalated to a failing error so dead links can't hide behind a 403.
Related Issue
#1575
Motivation and Context
Many third-party sites (FINRA, Coinbase, Bitfinex, etc.) return 401/403 to plain HTTP clients on valid links, flooding the link checker with false warnings.
Types of changes
Checklist:
bug-<issue#>-<description>orfeature-<issue#>-<description>