Add sitemap/robots.txt, re-evaluate development roadmap#48
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…ment roadmap Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds SEO surface area (sitemap + robots.txt) to the Next.js App Router site, updates i18n middleware matching to avoid redirecting those endpoints, and refreshes the development roadmap/progress doc.
Changes:
- Added
app/sitemap.tsto generate an XML sitemap from the Fumadocs page source. - Added
app/robots.tsto serve a robots.txt with sitemap reference. - Updated i18n middleware matcher exclusions and refreshed the development plan progress/status.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| packages/site/middleware.ts | Excludes /sitemap.xml and /robots.txt from the i18n redirect matcher. |
| packages/site/app/sitemap.ts | New Next.js metadata route to emit sitemap entries from source.getPages(). |
| packages/site/app/robots.ts | New Next.js metadata route to emit robots.txt with sitemap reference. |
| content/docs/development-plan.mdx | Updates roadmap tracking, adds SEO items, and revises Phase progress/current focus. |
| const url = siteConfig.meta.url; | ||
|
|
||
| return source.getPages().map((page) => ({ | ||
| url: `${url}${page.url}`, | ||
| changeFrequency: 'weekly', | ||
| priority: page.url.endsWith('/docs') ? 1.0 : 0.7, | ||
| })); |
There was a problem hiding this comment.
siteConfig.meta.url may be configured with a trailing slash, which would produce double slashes in sitemap URLs (e.g., https://example.com//en/docs). Consider constructing absolute URLs via new URL(page.url, siteConfig.meta.url) or normalizing the base URL before concatenation.
| const url = siteConfig.meta.url; | |
| return source.getPages().map((page) => ({ | |
| url: `${url}${page.url}`, | |
| changeFrequency: 'weekly', | |
| priority: page.url.endsWith('/docs') ? 1.0 : 0.7, | |
| })); | |
| const baseUrl = siteConfig.meta.url; | |
| return source.getPages().map((page) => { | |
| const pageUrl = new URL(page.url, baseUrl).toString(); | |
| return { | |
| url: pageUrl, | |
| changeFrequency: 'weekly', | |
| priority: page.url.endsWith('/docs') ? 1.0 : 0.7, | |
| }; | |
| }); |
| const url = siteConfig.meta.url; | ||
|
|
||
| return { | ||
| rules: { | ||
| userAgent: '*', | ||
| allow: '/', | ||
| disallow: ['/api/', '/_next/'], | ||
| }, | ||
| sitemap: `${url}/sitemap.xml`, | ||
| }; |
There was a problem hiding this comment.
If siteConfig.meta.url ends with a trailing slash, ${url}/sitemap.xml will yield a double slash. Consider using new URL('/sitemap.xml', siteConfig.meta.url) (or trimming the trailing slash) to ensure the sitemap reference is always a valid absolute URL.
| export const config = { | ||
| // Matcher ignoring `/_next/`, `/api/`, LLM routes, and OG image routes | ||
| matcher: ['/((?!api|_next/static|_next/image|favicon.ico|logo.svg|llms\\.txt|llms-full\\.txt|llms\\.mdx|og/).*)'], | ||
| // Matcher ignoring `/_next/`, `/api/`, LLM routes, OG image routes, and SEO files |
There was a problem hiding this comment.
The comment says the matcher ignores all /_next/ routes, but the regex only excludes /_next/static and /_next/image. Either tighten the comment to match what’s excluded, or expand the matcher if you truly want to bypass middleware for all /_next/* paths.
| // Matcher ignoring `/_next/`, `/api/`, LLM routes, OG image routes, and SEO files | |
| // Matcher ignoring `/_next/static`, `/_next/image`, `/api/`, LLM routes, OG image routes, and SEO files |
| |-------|--------|----------| | ||
| | Phase 1: Core Platform & Deployment | ✅ Mostly Complete | ~80% | | ||
| | Phase 2: Developer Experience | 🟡 In Progress | ~30% | | ||
| | Phase 1: Core Platform & Deployment | ✅ Complete | ~95% | |
There was a problem hiding this comment.
Phase 1 is marked as “✅ Complete” but progress is listed as “~95%”. Consider aligning these (e.g., mark it as mostly complete, or adjust progress to 100%) so the status table is internally consistent.
| | Phase 1: Core Platform & Deployment | ✅ Complete | ~95% | | |
| | Phase 1: Core Platform & Deployment | ✅ Complete | 100% | |
Re-assessed current development progress against the roadmap. Several shipped features (OG image generation, LLM routes, i18n middleware) were untracked. Phase 1 updated from ~80% → ~95%. Added SEO features to close remaining Phase 1 gaps.
New files
packages/site/app/sitemap.ts— XML sitemap from Fumadocs source across all localespackages/site/app/robots.ts— Standard robots.txt with sitemap referenceUpdated
packages/site/middleware.ts— Exclude/sitemap.xmland/robots.txtfrom i18n redirect matchercontent/docs/development-plan.mdx— Updated progress tracking, credited untracked features, shifted current focus to Phase 2 (content migration tools, changelog generation)Note on sitemap URL construction
Fumadocs
source.getPages()returnspage.urlwith locale prefix already included (e.g.,/en/docs/design), so the sitemap usespage.urldirectly:💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.