Tier unsupported sites by scraping difficulty#357
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
There was a problem hiding this comment.
Stale comment
Risk assessment: Very Low.
The actual diff only adds a short informational
<Info>callout with an external link inbrowsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt, infrastructure, auth, permissions, or data changes, and I didn’t find aCODEOWNERSfile in the repo.The PR is already approved on the current head commit, so I’m leaving this assessment as a comment only and not re-approving.
Sent by Cursor Automation: Assign PR reviewers
There was a problem hiding this comment.
Stale comment
Risk assessment: Very Low.
The actual diff only adds a short
<Info>callout with an external Scraperly link inbrowsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt/model-instruction, infrastructure, auth, permissions, or data changes, and I didn’t find aCODEOWNERSfile in the repo.The PR already has an approval recorded, so I’m leaving this assessment as a comment only and not re-approving.
Sent by Cursor Automation: Assign PR reviewers
There was a problem hiding this comment.
Risk assessment: Very Low.
The current diff only updates copy in browsers/faq.mdx to categorize unsupported websites by scraping difficulty. There are no codepath, config, navigation, infrastructure, auth, permissions, data, or model-instruction changes, and I didn’t find a CODEOWNERS file in the repo.
The PR is already approved, so I’m leaving this assessment as a comment only and not re-approving.
Sent by Cursor Automation: Assign PR reviewers
There was a problem hiding this comment.
Risk assessment: Very Low.
The actual diff changes only user-facing documentation in browsers/faq.mdx: it reorganizes six unsupported websites into two scraping-difficulty tiers and expands the explanatory copy. It doesn't modify executable code, prompts/model instructions, site configuration, navigation, infrastructure, authentication, permissions, or data, so the blast radius is limited to the rendered FAQ content. No CODEOWNERS file is present in the repository.
The PR already has an approval recorded, so I haven't approved it again.
Sent by Cursor Automation: Assign PR reviewers


Summary
Restructures the Unsupported Websites section of the browsers FAQ into a tiered list (Very Hard / Hard) of sites that are difficult or infeasible to automate today, with a short description of what to expect at each tier.
Inspired by per-site difficulty resources elsewhere in the scraping community, but using our own observations rather than linking out.
Drops the earlier
<Info>callout inbot-detection/overview.mdx.Preview
https://tbd-6fc993ce-hypeship-scraperly-link.mintlify.app/browsers/faq#unsupported-websites
Test plan
#unsupported-websitesanchor still resolvesNote
Low Risk
Documentation-only FAQ copy and structure; no product or runtime behavior changes in this diff.
Overview
The Unsupported Websites FAQ section is rewritten from a flat bullet list into Very Hard and Hard tiers, each with guidance on what automation friction to expect (login walls/CAPTCHAs vs. stealth/proxy mitigations).
Very Hard groups LinkedIn, Facebook, and Instagram; Hard covers X, Amazon, and Reddit. A new intro clarifies the list is incomplete and based on internal experience, including when stealth, residential proxies, and profiles still aren’t enough.
Reviewed by Cursor Bugbot for commit 23689d7. Bugbot is set up for automated code reviews on this repo. Configure here.