Skip to content

Fix docs site domain: github.github.io, not github.github.com - #50630

Closed
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/geo-optimizer-create-llms-and-robots
Closed

Fix docs site domain: github.github.io, not github.github.com#50630
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/geo-optimizer-create-llms-and-robots

Conversation

Copilot AI commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

The GEO audit flagged llms.txt and robots.txt sub-scores at 0 across the docs site. Root cause: the docs site is a GitHub Pages project site, served at https://github.github.io/gh-aw/, but the repo hardcoded the non-resolving domain https://github.github.com/gh-aw/ throughout config, static assets, and tests.

Config

  • docs/astro.config.mjs: site corrected to https://github.github.io

Static assets

  • docs/public/robots.txt, docs/public/sitemap.xml, docs/public/.well-known/ai.txt: sitemap and AI-discovery URLs corrected

Components

  • docs/src/components/CustomHead.astro, CustomFooter.astro, docs/src/config/recruitmentBanner.ts: hardcoded URLs corrected

Workflows

  • .github/workflows/daily-geo-optimizer.md: audit now targets github.github.io/gh-aw so future GEO audits score the real site
  • .github/workflows/dev.md: reference URLs corrected
  • Recompiled .lock.yml for both

Tests & docs

  • docs/tests/robots-txt.spec.ts, docs/tests/schema-jsonld.spec.ts: expected URLs updated to match corrected domain
  • docs/README.md: doc text corrected

Verified via npm run build in docs/ that generated robots.txt, llms.txt, and sitemap-index.xml all resolve to github.github.io.

@pelikhan

pelikhan commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot geooptimizer to deal with the fact that this is a github pages site and the llms.txt will be at https://github.github.io/gh-aw/llms.txt NOT at the domain level.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Create /llms.txt and robots.txt allow rules for docs site Fix docs site domain: github.github.io, not github.github.com Aug 5, 2026
Copilot AI requested a review from pelikhan August 5, 2026 17:15

Copilot AI commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

@copilot geooptimizer to deal with the fact that this is a github pages site and the llms.txt will be at https://github.github.io/gh-aw/llms.txt NOT at the domain level.

Already addressed in d74f4a2astro.config.mjs, robots.txt, sitemap.xml, .well-known/ai.txt, and all llms.txt references now use https://github.github.io/gh-aw/ (verified with npm run build, generated llms.txt resolves under that base).

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[geo-optimizer] Create /llms.txt and AI-crawler robots.txt allow rules for docs site

2 participants