No action needed: robots.txt already resolves GEO audit finding

[Copilot]

The GEO audit flagged a missing robots.txt on the docs site (scoring 0/18) with no explicit Allow rules for AI crawlers like GPTBot, ClaudeBot, and PerplexityBot.

Investigation

• Confirmed docs/public/robots.txt already exists, introduced in a prior same-day commit (Configuring agentic workflows with Aider documentation #50293).
• The file already covers the exact requirements from the audit:
  • User-agent: * with Allow: /
  • Dedicated Allow: / blocks for GPTBot, ClaudeBot, Claude-SearchBot, claude-web, anthropic-ai, PerplexityBot, Perplexity-User, Google-Extended, Google-CloudVertexBot, OAI-SearchBot, ChatGPT-User, and other major AI/search bots
  • A Sitemap: https://github.github.com/gh-aw/sitemap.xml directive
• Verified against docs/astro.config.mjs (site: 'https://github.github.com', base: '/gh-aw/') that Astro will serve this file correctly at https://github.github.com/gh-aw/robots.txt once deployed.

Changes

• None. The working tree is clean; no code changes were required since the fix already landed.

Next GEO audit run after deploy should reflect the updated robots_txt score.

• Fixes [geo-optimizer] Add robots.txt with explicit AI-bot Allow rules to docs site #50335