feat(pit): make /n/ crawlable - #175
Merged
Merged
Conversation
`/n/<name>` is the point of the pit being on the clearnet — a name nobody holds is a page somebody should be able to find. Nothing advertised them: robots.txt 404'd, there was no sitemap, and the pages carried no canonical or description. - GET /robots.txt names /n/ as allowed and points at the sitemap, while keeping /api, /app, /settings and /sessions out of an index. - GET /sitemap.xml is generated from the registry, so it cannot lag behind what has actually been claimed. Whole names only: /n/<ending> is not a name and 400s, so listing endings would advertise dead URLs. - A name's page carries a canonical URL, a description and OpenGraph tags. An aliased name canonicalises to what it resolves to, so one page reached by two names does not compete with itself as a duplicate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
ralyodio
added a commit
that referenced
this pull request
Aug 1, 2026
#176) #175 built the canonical URLs from config.origin, which is PUBLIC_ORIGIN — app.moshcode.sh. But the pit answers on pit.moshcode.sh too, and both hosts serve byte-identical pages, so every canonical, every sitemap entry and the Sitemap: line all named the duplicate as the original. That points crawlers at the wrong host and leaves the two competing. Add config.pitOrigin (PIT_ORIGIN, defaulting to origin so nothing changes where it is unset) and build the namespace's public URLs from it. Kept separate from origin on purpose: rpID is derived from origin, so moving that would invalidate every passkey already registered. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merged
ralyodio
added a commit
that referenced
this pull request
Aug 1, 2026
install.sh resolves releases/latest, so the ten commits merged since v0.14.0 have been sitting on main unreachable — including the reason `curl <name>` did not work. The headline is parking. A parked name always resolved somewhere, but the address it pointed at was a host that routes by Host header and answers "Application not found" for a name it has never heard of, so `curl scrambled.eggs` resolved and then died one layer up (#180). It could not be fixed there: the platform rejects a Moshpit ending as a custom domain and no public CA will certify a TLD outside the DNS root. The bridge is already running locally for the name to resolve at all, so it now serves the answer too — parked names point at loopback and a catch-all responder 302s them to the Pit. Underneath that was a quieter one. fetchTlds read the first page of the ending list and stopped; the registry answers 200 rows and reports the real total, but 200 rows look exactly like a complete list of 200. `.eggs` sat past that line, so `dns install` wrote a config that did not route it and the name failed to resolve — a DNS-shaped symptom three layers from the cause. It was hiding 94% of the namespace: 3707 endings, 200 visible (#181). Also here: dns resolve now reports a parked name's page in the Pit instead of an IP that answers for nobody, with --open to go there (#179) the pit's /n/ pages are crawlable — robots.txt, a generated sitemap, and canonical tags that name the pit host rather than the app host it shares a service with (#175, #176) the claim box takes a whole name, claiming the ending first when it is free and minting the name under it (#173) the endings list pages instead of stopping at 200 (#174), the paste field reads names as well as endings (#172), related endings keep the name you are reading (#177), and integrations ship JSON support matrices (#178) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/n/<name>is the point of the pit being on the clearnet — a name nobody holds is a page somebody should be able to find. Nothing advertised them:Change
GET /robots.txt— names/n/as allowed and points at the sitemap, while keeping/api,/app,/settingsand/sessionsout of an index.GET /sitemap.xml— generated from the registry rather than stored, so it can't lag behind what's actually been claimed. Whole names only:/n/<ending>is not a name and 400s, so listing endings would advertise dead URLs.Uses
config.origin(PUBLIC_ORIGIN) for absolute URLs, as a sitemap requires.The proxied half of a name (
/n/<name>/<path>) is somebody else's site reached through us, so only the name's own page is offered for indexing.Verification
npm testinapps/pwa— 340 pass, 0 fail (4 new, route-level against a throwaway libSQL db):/n/, points at the sitemap, disallows the private half/n/eggs, which would 400🤖 Generated with Claude Code