Skip to content

Tier unsupported sites by scraping difficulty#357

Closed
guergabo wants to merge 6 commits into
mainfrom
hypeship/scraperly-link
Closed

Tier unsupported sites by scraping difficulty#357
guergabo wants to merge 6 commits into
mainfrom
hypeship/scraperly-link

Conversation

@guergabo

@guergabo guergabo commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Restructures the Unsupported Websites section of the browsers FAQ into a tiered list (Very Hard / Hard) of sites that are difficult or infeasible to automate today, with a short description of what to expect at each tier.

Inspired by per-site difficulty resources elsewhere in the scraping community, but using our own observations rather than linking out.

Drops the earlier <Info> callout in bot-detection/overview.mdx.

Preview

https://tbd-6fc993ce-hypeship-scraperly-link.mintlify.app/browsers/faq#unsupported-websites

Test plan

  • Mintlify preview renders the tiered list cleanly
  • #unsupported-websites anchor still resolves

Note

Low Risk
Documentation-only FAQ copy and structure; no product or runtime behavior changes in this diff.

Overview
The Unsupported Websites FAQ section is rewritten from a flat bullet list into Very Hard and Hard tiers, each with guidance on what automation friction to expect (login walls/CAPTCHAs vs. stealth/proxy mitigations).

Very Hard groups LinkedIn, Facebook, and Instagram; Hard covers X, Amazon, and Reddit. A new intro clarifies the list is incomplete and based on internal experience, including when stealth, residential proxies, and profiles still aren’t enough.

Reviewed by Cursor Bugbot for commit 23689d7. Bugbot is set up for automated code reviews on this repo. Configure here.

@mintlify

mintlify Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
Kernel 🟢 Ready View Preview May 11, 2026, 5:22 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

cursor[bot]
cursor Bot approved these changes May 11, 2026
cursor[bot]
cursor Bot approved these changes May 11, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low.

The actual diff only adds a short informational <Info> callout with an external link in browsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt, infrastructure, auth, permissions, or data changes, and I didn’t find a CODEOWNERS file in the repo.

The PR is already approved on the current head commit, so I’m leaving this assessment as a comment only and not re-approving.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@guergabo guergabo changed the title Link to Scraperly in bot anti-detection overview Tier unsupported sites by scraping difficulty May 11, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low.

The actual diff only adds a short <Info> callout with an external Scraperly link in browsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt/model-instruction, infrastructure, auth, permissions, or data changes, and I didn’t find a CODEOWNERS file in the repo.

The PR already has an approval recorded, so I’m leaving this assessment as a comment only and not re-approving.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk assessment: Very Low.

The current diff only updates copy in browsers/faq.mdx to categorize unsupported websites by scraping difficulty. There are no codepath, config, navigation, infrastructure, auth, permissions, data, or model-instruction changes, and I didn’t find a CODEOWNERS file in the repo.

The PR is already approved, so I’m leaving this assessment as a comment only and not re-approving.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@masnwilliams
masnwilliams marked this pull request as ready for review July 22, 2026 06:16

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk assessment: Very Low.

The actual diff changes only user-facing documentation in browsers/faq.mdx: it reorganizes six unsupported websites into two scraping-difficulty tiers and expands the explanatory copy. It doesn't modify executable code, prompts/model instructions, site configuration, navigation, infrastructure, authentication, permissions, or data, so the blast radius is limited to the rendered FAQ content. No CODEOWNERS file is present in the repository.

The PR already has an approval recorded, so I haven't approved it again.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants