fix(seo): robots.txt, noindex, JSON-LD, sitemap drift check, keywords, README badges#358
Conversation
The Workers asset binding's SPA fallback silently served index.html for /robots.txt since the file never existed, giving crawlers no crawl directives and hiding the sitemap location. Closes #307 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
RootLayout (used only by DashboardLayout, which every dashboard/* route and oauth/consent render) had no robots meta tag, so Clerk-gated app shells could be indexed and shown as blank/broken search results. Closes #308 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
Add SoftwareApplication and Organization schema.org JSON-LD to the marketing homepage, making the site eligible for rich results. Closes #310 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
Add <lastmod> to the hand-maintained sitemap.xml, and add scripts/check-sitemap.mjs (wired into ci.yml) to fail the build if a new public marketing page is added under src/pages without a matching sitemap entry. Closes #314 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
Add npm keywords to @agentstate/sdk and PyPI keywords/classifiers to the Python package so both surface in registry search (npmjs.com, pypi.org) for terms like "agent memory" and "langgraph checkpoint". Bump the TS SDK patch version so publish-sdk.yml actually ships it. Closes #311 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
Surface the existing ci.yml status plus the published @agentstate/sdk and agentstate PyPI versions as badges alongside the existing License/PRs-welcome row. Closes #313 Co-Authored-By: Duyet Le <me@duyet.net> Co-Authored-By: duyetbot <bot@duyet.net>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
Warning Review limit reached
Next review available in: 49 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Summary
robots.txtwithDisallow: /dashboard,/oauth, and aSitemap:pointer — previously the SPA fallback silently servedindex.htmlfor/robots.txt.noindexprop (defaulttrue) toRootLayout.astro, which is exclusively used byDashboardLayout.astro(everydashboard/*route +oauth/consent), so<meta name="robots" content="noindex, nofollow">now covers all auth-gated app shells.SoftwareApplication+OrganizationJSON-LD to the marketing homepage (index.astro).<lastmod>tositemap.xmland a newscripts/check-sitemap.mjs(wired intoci.yml) that fails CI if a public marketing page undersrc/pages/has no matching sitemap entry.keywordstopackages/sdk/package.json(+ patch bump to 0.1.4 sopublish-sdk.ymlships it) andkeywords/classifierstopackages/python-sdk/pyproject.toml(bumped to 0.1.2 — note there's no automated PyPI publish workflow in this repo, so that bump only takes effect on the next manual publish).README.mdalongside the existing License/PRs-welcome badges.Closes #307, #308, #310, #311, #313, #314
Skipped (need real image assets, out of scope for this PR):
Test plan
bunx biome check packages/api/src/— pre-existing failure invalidation.tsconfirmed unrelated (no diff vsmainfor that file)bunx tsc --noEmit -p packages/api/tsconfig.json— passesbun scripts/check-sitemap.mjs— passes, confirms all 3 marketing routes covered