[Bots] Web Bot Auth docs

[Oxyjun]

Summary

We're now introducing "Web Bot Auth" (WBA), which is a more secure authentication method for verifying bots. Previously, verifying bots was only possible through two flavours of IP validation, public IP list, and reverse DNS.

Web Bot Auth may become the new IETF standard in the near future, and paves the way for better bot detection across the Internet.

This PR restructures and adds information for WBA. Specifically:

1. Creates a new chapter Verified bots requirements, which explains what's involved for a bot to be verified.
2. Cuts down the existing Verified bots policy chapter to only talk about the policy
3. Creates a new chapter Verification methods, which :
   • Guides users through verifying their bot via WBA
   • Guides users through verifying their bot via IP validation (text taken from existing Policy chapter)
   • Talks about generic user agents (text taken from existing Policy chapter)
   • Points users to other useful resources under "Additional resources" header.

Things that we need to improve:

1. Link to blog
2. Possible restructure such that WBA is its own page (and how this affects rest of IA)
3. Consider absorbing Categories chapter into Verification methods (or somewhere else) Decided against after discussion.
4. Consider defining the three main required headers for WBA into its own subheadings (such that they can be hyperlinked). This relates to point (2).
5. Work out where the FAQs should go. Possibly a new Reference > FAQs > "WBA FAQs"?

Screenshots (optional)

Documentation checklist

• The documentation style guide has been adhered to.
• If a larger change - such as adding a new page- an issue has been opened in relation to any incorrect or out of date information that this PR fixes.
• Files which have changed name or location have been allocated redirects.

[hyperlint-ai]

Howdy and thanks for contributing to our repo. The Cloudflare team reviews new, external PRs within two (2) weeks. If it's been two weeks or longer without any movement, please tag the PR Assignees in a comment.

We review internal PRs within 1 week. If it's something urgent or has been sitting without a comment, start a thread in the Developer Docs space internally.

---

PR Change Summary

Introduced Web Bot Auth (WBA) documentation, enhancing bot verification methods.

• Created a new chapter on Verified Bots Requirements detailing verification criteria.
• Restructured the Verified Bots Policy chapter to focus solely on policy aspects.
• Added a new chapter on Verification Methods, outlining WBA and IP validation processes.

Modified Files

• src/content/docs/bots/concepts/bot/verified-bots/categories.mdx
• src/content/docs/bots/concepts/bot/verified-bots/policy.mdx

Added Files

• src/content/docs/bots/concepts/bot/verified-bots/requirements.mdx
• src/content/docs/bots/concepts/bot/verified-bots/verification.mdx

---

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

[github-actions]

This pull request requires reviews from CODEOWNERS as it changes files that match the following patterns:

Pattern	Owners
/public/__redirects	@GregBrimble, @KianNH, @pedrosousa, @WalshyDev, @cloudflare/pcx-technical-writing
/src/content/docs/bots/	@patriciasantaana, @cloudflare/pcx-technical-writing
*	@cloudflare/pcx-technical-writing

[github-actions]

Preview URL: https://dbaad358.preview.developers.cloudflare.com
Preview Branch URL: https://jun-bots-wba.preview.developers.cloudflare.com

Files with changes (up to 15)

Original Link	Updated Link
https://developers.cloudflare.com/bots/concepts/bot/verified-bots/web-bot-auth/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/verified-bots/web-bot-auth/
https://developers.cloudflare.com/bots/frequently-asked-questions/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/frequently-asked-questions/
https://developers.cloudflare.com/bots/concepts/bot/verified-bots/ip-validation/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/verified-bots/ip-validation/
https://developers.cloudflare.com/bots/concepts/bot/verified-bots/policy/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/verified-bots/policy/
https://developers.cloudflare.com/bots/concepts/bot/verified-bots/overview/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/verified-bots/overview/
https://developers.cloudflare.com/bots/concepts/bot/verified-bots/categories/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/verified-bots/categories/
https://developers.cloudflare.com/bots/concepts/bot/	https://jun-bots-wba.preview.developers.cloudflare.com/bots/concepts/bot/

[github-actions]

This PR changes current filenames or deletes current files. Make sure you have redirects set up to cover the following paths:

• /bots/troubleshooting/frequently-asked-questions/

[AkshatM]

One or two final comments, but marking as appriove to unblock you

[JohnDaWalka]

ThxSent from my Apple WatchOn Jun 23, 2025, at 18:15, Akshat Mahajan ***@***.***> wrote: @AkshatM approved this pull request. One or two final comments, but marking as appriove to unblock you —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>