fix(proxy): hardcode INDEXABLE_HOSTS, decouple from NEXT_PUBLIC_SITE_URL

[JohnRDOrazio]

Summary

After PR #47 fixed the prod-was-being-noindex'd bug, the staging
deploy still didn't get noindex'd. Verified after staging redeploy
(run 25193770888):

$ curl -sI https://staging.catholicdigitalcommons.org/
HTTP/2 200
(no x-robots-tag — should have one)

Root cause

PR #42 derived the indexable-host set from NEXT_PUBLIC_SITE_URL.
PR #43 + the build env work set NEXT_PUBLIC_SITE_URL_STAGING so
the staging build's share buttons and canonical URLs would point at
staging. But that meant proxy.ts on staging computed:

PRODUCTION_HOSTS = {staging.catholicdigitalcommons.org, www.staging…}

…and treated the incoming staging host as indexable. The opposite
of intent.

Two distinct concepts were collapsed into one variable:

Concept	Source	Per-environment?
"what URL THIS deployment thinks it is"	NEXT_PUBLIC_SITE_URL	yes
"what URLs search engines should index"	(this PR: INDEXABLE_HOSTS)	no

Fix

Hardcode INDEXABLE_HOSTS in proxy.ts. No env override:

const INDEXABLE_HOSTS = new Set<string>([
  'catholicdigitalcommons.org',
  'www.catholicdigitalcommons.org',
])

If the canonical hostname ever changes, that's a code change.
Eliminates the env-derived bug class entirely.

NEXT_PUBLIC_SITE_URL_PROD / NEXT_PUBLIC_SITE_URL_STAGING still
diverge per environment — they drive content-side absolute URLs
(share buttons, sitemap canonicals) which DO need to be different.

Urgency

Staging is currently NOT noindex'd. Lower urgency than PR #47 (which
was fixing the inverse — production being noindex'd) but still
worth deploying soon to stop staging URLs being indexable.

Test plan

• npm run build clean locally
• Merge → re-run Build and Deploy (production, no behavior
  change expected; prod still NOT noindex'd) and Build and
  Deploy to Staging (staging now noindex'd)
• Verify:
  • curl -I https://catholicdigitalcommons.org/ → no
    X-Robots-Tag
  • curl -I https://staging.catholicdigitalcommons.org/ →
    X-Robots-Tag: noindex, nofollow

🤖 Generated with Claude Code

[coderabbitai]

Warning

Rate limit exceeded

@JohnRDOrazio has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 48 minutes and 48 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 56b9e08a-2f64-4387-97f0-60c44328b3c5

📥 Commits

Reviewing files that changed from the base of the PR and between 1b9a9f7 and e3aa782.

📒 Files selected for processing (1)

• proxy.ts

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/proxy-decouple-indexable-from-site-url

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 48 minutes and 48 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics -7 complexity · 0 duplication

Metric	Results
Complexity	-7
Duplication	0

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}