Skip to content

fix(seo): fix GSC indexing issues, remove unused academy/partners pages#5388

Merged
waleedlatif1 merged 3 commits into
stagingfrom
seo-coverage-fixes
Jul 3, 2026
Merged

fix(seo): fix GSC indexing issues, remove unused academy/partners pages#5388
waleedlatif1 merged 3 commits into
stagingfrom
seo-coverage-fixes

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Fixed a batch of Google Search Console indexing issues from a Search Console coverage report audit:
    • Unblocked /chat/ in robots.txt; deployed chats are now noindexed per-page instead (only public + active ones are indexable) — robots.txt was previously blocking Google from ever seeing the noindex on gated ones
    • Added missing 301 redirects for renamed integration slugs (sap-s-4hanasap-s4hana, calcomcal-com) that were 404ing with no redirect
    • Fixed missing canonical/noindex handling on query-param-filtered catalog pages (integrations, models, blog, careers, pricing) — Google was flagging these as "duplicate content, chose different canonical"
    • Standardized page titles to "Page | Sim, the AI Workspace" across careers and comparison pages (previously inconsistent formatting)
  • Removed the Academy marketing pages and Partner Program page
    • Academy content is consolidated into docs.sim.ai/academy; academy was always feature-flagged off and never publicly reachable, but its sitemap entries were advertising 404ing URLs to Google — added a redirect to the new docs home
    • Partners was live/indexed; added a redirect to /contact
    • Dropped the unused academy_certificate table via migration (never had a live write)
    • Stripped the sandbox-mode plumbing from the core workflow editor that existed solely to support academy's embedded exercise canvas

Type of Change

  • Bug fix

Testing

Tested manually. Typecheck, Biome lint, check:api-validation, and seo.test.ts all pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

- robots.ts: unblock /chat/ (page-level noindex now gates gated/inactive
  deployments instead), drop the now-vestigial blog-tag/link-preview carve-out
- next.config.ts: add missing redirects for renamed integration slugs
  (sap-s-4hana, calcom), removed /partners, and removed /academy
- fix missing canonical/noindex on filtered catalog pages (integrations,
  models, blog, careers, pricing) causing GSC "duplicate, Google chose
  different canonical"
- standardize page titles to "Page | Sim, the AI Workspace" across the board
- remove the academy marketing pages and partner program page (content
  consolidated into docs.sim.ai/academy); drop the unused academy_certificate
  table via migration and strip the sandbox-mode plumbing from the workflow
  editor that only academy ever used
@vercel

vercel Bot commented Jul 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 3, 2026 6:59pm

Request Review

@cursor

cursor Bot commented Jul 3, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Chat metadata now hits the DB on every request with force-dynamic; large Academy removal is low user impact but broad code deletion, and SEO changes affect what Google indexes site-wide.

Overview
Addresses Google Search Console coverage issues and removes marketing surfaces that were dead or misleading in the index.

SEO / indexing: /chat/ is no longer blocked in robots.ts; each chat deployment gets dynamic metadata that only allows indexing when the deployment is active and authType === 'public', with force-dynamic so that check runs per request. Catalog-style routes (blog, integrations, models, careers, pricing) now use generateMetadata so filtered query variants are noindex, follow while canonical stays on the unfiltered URL (blog canonical is always /blog, not tag/page permutations). Careers and comparison titles are aligned to the | Sim, the AI Workspace pattern. Sitemap drops partners and academy entries; seo.test no longer scans academy.

Removed pages & redirects: The Partners page and footer link are removed; /partners/contact. The entire Sim Academy app (courses, lessons, certificates UI, certificate API, lib/hooks) is deleted; /academy/* → docs.sim.ai/academy. Integration slug 301s added for sap-s-4hana and calcom.

Workflow editor: Academy-only sandbox mode is stripped from the workflow canvas, panel, execution hook, and related types/providers.

Other: Theme provider no longer forces dark mode for /academy. academy_certificate table remains in schema with a comment that app code was removed (drop deferred for rolling deploy safety).

Reviewed by Cursor Bugbot for commit 0729fae. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a batch of Google Search Console indexing issues and removes the never-public Academy and Partners marketing pages. All changes are clean, well-commented, and follow consistent patterns across the affected pages.

  • robots.txt / per-page noindex: /chat/ is unblocked in robots.txt; auth-gated and inactive chat deployments are now noindexed via a live DB query in generateMetadata, letting public chats remain indexable. The dual-rule structure (main bots + social preview bots) is also consolidated into a single rule.
  • Catalog noindex-on-filter: Integrations, Models, Blog, Careers, and Pricing pages all switch from static metadata exports to async generateMetadata; filtered/paginated URLs get robots: { index: false, follow: true } while the bare catalog URL remains indexable. The previously misleading revalidate = 3600 exports are removed on pages that now read searchParams (which forces dynamic rendering anyway).
  • Redirects & removals: 301s added for renamed integration slugs (sap-s-4hana, calcom), /partners, and /academy/**; sitemap entries and footer links cleaned up. Academy application code is stripped; the DB table schema definition is intentionally retained pending a follow-up expand/contract migration.

Confidence Score: 5/5

Safe to merge — SEO-only changes with no data mutations; the one DB query added (chat page metadata) is read-only and handles missing rows gracefully.

All changes are additive or purely subtractive (dead code removal). The noindex-on-filter pattern is applied consistently across five pages. The academy DB table is deliberately kept in schema.ts with a clear explanation — no migration mismatch risk. The 301 redirects are straightforward and the academy redirect correctly uses :path* which matches the bare /academy path as well. No auth paths, payment paths, or data-write paths are touched.

No files require special attention.

Important Files Changed

Filename Overview
apps/sim/app/robots.ts Removed /chat/ from DISALLOWED_PATHS and collapsed the dual-rule structure (main + link-preview bots) into a single rule; link-preview bots now share the same disallow list as Googlebot
apps/sim/next.config.ts Adds four permanent 301 redirects: academy sub-paths → docs.sim.ai, sap-s-4hana → sap-s4hana, calcom → cal-com, /partners → /contact
apps/sim/app/(interfaces)/chat/[identifier]/page.tsx Replaces static metadata export with dynamic generateMetadata that queries the DB to noindex auth-gated or inactive chat deployments; adds force-dynamic segment config
apps/sim/app/(landing)/pricing/page.tsx Converts static metadata to dynamic generateMetadata; applies noindex when billing !== 'monthly' (annual toggle); removes now-redundant revalidate = 3600
packages/db/schema.ts Academy table definition intentionally retained in schema with a comment explaining the expand/contract deferral; drop migration will follow in a subsequent PR
apps/sim/app/(landing)/blog/page.tsx Canonical simplified to bare /blog for all variants; tag/paginated views get noindex + follow; removed revalidate = 3600 (now fully dynamic due to searchParams read)
apps/sim/app/(landing)/integrations/(shell)/page.tsx Converts static metadata to async generateMetadata; filtered catalog URLs (q/category) get noindex + follow while canonical always points at bare /integrations
apps/sim/app/(landing)/models/(shell)/page.tsx Same noindex-on-filter pattern applied to models directory; q/provider filter variants are noindexed while bare /models remains indexable
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx Removed sandbox prop, isSandbox flags, and all academy-specific execution shortcuts; workflow is now a simpler two-mode component (normal vs embedded)
apps/sim/app/sitemap.ts Removed /partners and academy course entries from sitemap; updated doc comment accordingly

Reviews (2): Last reviewed commit: "fix(seo): drop dead revalidate exports o..." | Re-trigger Greptile

CI's expand/contract migration safety check correctly flagged this: the
academy_certificate DROP TABLE was bundled in the same PR as removing the
code that reads/writes it (the certificates API route had no feature-flag
guard of its own, so it was reachable independent of the marketing pages
being disabled). Dropping the table in the same deploy risks breaking any
pod still running the old code during a rolling deploy.

Restores the table/enum in schema.ts and the test mock, and removes the
0254 migration. The table drop should ship in its own PR once this one's
code removal is confirmed live.
Any Server Component in the route tree reading searchParams forces the
whole route to fully dynamic per-request rendering, which overrides ISR —
revalidate is a silent no-op once that happens. True on pricing/careers
because generateMetadata now parses searchParams directly, and was already
true on blog before this PR (its page body already read searchParams).
Flagged by Greptile on pricing; same root cause applies to all three.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Addressed both automated findings, plus one CI safety-gate failure that needed a real fix rather than a bypass:

Greptile — dead revalidate = 3600 on pricing: confirmed correct. Once generateMetadata reads searchParams, the whole route is forced into fully-dynamic per-request rendering, which silently overrides ISR. Removed it — and since the same root cause applies to careers (my new generateMetadata also reads searchParams directly) and blog (its page body already did, predating this PR), removed it from all three for consistency with integrations/models, which never had it.

CI — check:migrations failure: the academy_certificate DROP TABLE migration was flagged as a same-deploy contract-phase op, correctly — the API route that read/wrote that table had no feature-flag guard of its own (only auth + rate-limit), so it was reachable independent of the marketing pages being disabled, meaning I can't honestly claim the dependent code "already shipped in an earlier deploy" to justify a -- migration-safe: bypass. Restored the table/enum in schema.ts and removed the migration; the drop will ship in its own follow-up PR once this one's code removal is live.

Pushed both fixes.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0729fae. Configure here.

@waleedlatif1 waleedlatif1 merged commit 7ff4f17 into staging Jul 3, 2026
18 checks passed
@waleedlatif1 waleedlatif1 deleted the seo-coverage-fixes branch July 3, 2026 19:15
waleedlatif1 added a commit that referenced this pull request Jul 3, 2026
generateMetadata queried the DB with no error handling, unlike the
identical query in api/chat/[identifier]/route.ts (which already wraps
it in try/catch). There's no error.tsx under app/(interfaces)/chat/ —
metadata resolution errors aren't caught by route-segment error
boundaries at all, only the root global-error.tsx — so a DB hiccup
during this lookup would take the whole page down to a generic error
page instead of just failing to determine indexability. Catches the
error, logs it, and defaults to noindex: if we can't confirm the
deployment is safely public, that's the correct SEO default anyway,
not a reason to crash the request.

Flagged by Cursor Bugbot on #5388 (already merged); this ships the fix
as a standalone follow-up since the underlying code is already live.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant