[SEO] Sitemap and robots.txt

[2witstudios]

Summary

Generate sitemap.xml and configure robots.txt for search engine crawling.

Technical Details

• Dynamic sitemap generation using next-sitemap or custom
• Include all public pages
• Priority and changefreq configuration
• robots.txt with sitemap reference

Files

• apps/marketing/next-sitemap.config.js
• apps/marketing/public/robots.txt
• apps/marketing/src/app/sitemap.ts

Acceptance Criteria

• sitemap.xml generated at build time
• All public pages included
• robots.txt allows crawling
• Sitemap accessible at /sitemap.xml
• No private/auth pages in sitemap

[2witstudios]

Completed in commit 2af9dbc on branch landing.

Acceptance Criteria Met:

• sitemap.xml generated at build time
• All public pages included (12 URLs)
• robots.txt allows crawling
• Sitemap accessible at /sitemap.xml
• No private/auth pages in sitemap

Implementation:

• Created src/app/sitemap.ts using Next.js 15 App Router conventions
• Sitemap includes all planned marketing routes with proper priorities:
  • / (priority 1.0)
  • /pricing, /downloads (priority 0.8-0.9)
  • /tour, /integrations, /docs, /blog (priority 0.7)
  • /faq, /changelog (priority 0.6)
• Updated robots.txt to disallow /screenshots/, /api/, /_next/