Skip to content

fix(router): route appsync-realtime-api and cloudfront.net, and DRY the wildcard-domain list - #350

Merged
Neaox merged 2 commits into
mainfrom
claude/appsync-realtime-and-base-parity
Jul 28, 2026
Merged

fix(router): route appsync-realtime-api and cloudfront.net, and DRY the wildcard-domain list#350
Neaox merged 2 commits into
mainfrom
claude/appsync-realtime-and-base-parity

Conversation

@Neaox

@Neaox Neaox commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Follow-up to #345. Three more gaps in host-based addressing, found by putting every addressing form against every recognised base in one grid rather than testing them one at a time.

Form localhost .overcast.sh .localstack.cloud .floci.io amazonaws.com
S3 bare {bucket}.{base} s3 s3 s3 was ✗ (correct — not an AWS form)
S3 labelled {bucket}.s3.{region}.{base} s3 s3 s3 s3 s3
{id}.execute-api.{region}.{base} svc svc svc svc svc
{id}.lambda-url.{region}.{base} svc svc svc svc svc
{id}.appsync-api.{region}.{base} svc svc svc svc svc
{id}.appsync-realtime-api.{region}.{base} was ✗ s3 was ✗ s3 was ✗ s3 was ✗ was ✗
{distributionId}.cloudfront.net was ✗

What was wrong

appsync-realtime-api was claimed as an S3 bucket. Not merely unrouted — the label was unregistered, so the bare virtual-hosted form matched and produced a bucket named {apiId}.appsync-realtime-api.{region}. A subscription to the hostname AWS actually serves landed on the S3 handler.

That matters more than "advertise a safe value for dns.REALTIME" allowed for, which is what #345 did: Amplify and the AppSync SDK derive the realtime host by substituting appsync-apiappsync-realtime-api into the GraphQL URL and never read dns.REALTIME. So the hostname has to route regardless of what the API returns. It now reaches the same endpoint appsync-api serves, with the query string preserved because AppSync carries connection auth there (?header=…&payload=…).

{distributionId}.cloudfront.net was unrouted. It needs no new grammar — the segments are {id}.cloudfront.{base}, so registering cloudfront is enough; CloudFront is global and ParseHostRoute already treats the region as optional. Distribution.DomainName is now minted on the caller's hostname rather than the literal cloudfront.net, which Overcast cannot serve without a DNS override. The staging-distribution lookup takes the ID from the first label instead of trimming a fixed suffix, so it holds for both forms.

localhost.floci.io was missing from the S3 bare-form bases while internal/containerendpoint already advertised it, so a bucket was unreachable on a domain Overcast tells users resolves to it. The two lists were maintained separately and drifted. Both now derive from config.WildcardDNSDomains, and TestVirtualHostBases_coverEveryWildcardDomain fails if they diverge again.

Guardrail change

cloudfront is the first single-word label. The rule that labels must not be plausible bucket-name segments previously required a hyphen; it now accepts a single word only with a written rationale in nonHyphenatedLabelRationale, which keeps the exception visible in review. Generic words (logs, data, cache) still cannot be added.

Documentation

docs/networking.md gains an inventory of every known AWS resource subdomain and whether Overcast routes it, including the ones deliberately absent. AWS publishes no such list — the SDK endpoint rulesets and Smithy's endpointPrefix cover control-plane endpoints only and carry none of execute-api, lambda-url or appsync-api — so it has to be maintained here.

Engine data-plane endpoints (rds, cache, kafka, es, the Cognito hosted UI's auth) are deliberately not routed: Overcast emulates those control planes, so there is nothing behind the hostname, and every one of those labels is a bare common word that would make names like my.cache unaddressable for no benefit.

Separately, OVERCAST_HOSTNAME guidance was inconsistent and in one place pointed the wrong way — cdk.md's worked example set it to localhost.localstack.cloud, so the canonical CDK-on-Windows fix recommended another project's domain. localhost.overcast.sh is now the recommendation across networking.md, cdk.md, the env-var table and the S3 service doc. localhost.localstack.cloud and localhost.floci.io stay recognised and documented so carried-over setups keep working. Where the wildcard domains do not work — offline, and behind DNS rebinding protection — stays surfaced at each site, with the fallbacks named.

Verification

gofmt, go build -tags slim ./..., go vet -tags slim ./..., full ./internal/... and ./tests/integration/..., make lint-go (golangci-lint v2.8.0), and docs-index --check all clean. The 43-cell matrix in TestHostClassifier_everyFormOnEveryBase passes; it was red on the two real gaps before the fix.

Neaox added 2 commits July 28, 2026 21:38
…he base list

Three gaps in host-based addressing, found by putting every form against every
base in one grid (TestHostClassifier_everyFormOnEveryBase).

appsync-realtime-api was not a registered label, so the bare S3 form claimed
it: a subscription to {apiId}.appsync-realtime-api.{region}.{host} -- the
hostname AWS actually serves, and the one Amplify derives by substituting into
the GraphQL URL rather than reading dns.REALTIME -- landed on the S3 handler as
a bucket named "{apiId}.appsync-realtime-api.{region}". It now routes to the
same endpoint appsync-api serves, with the query string preserved because
AppSync carries connection auth there.

{distributionId}.cloudfront.net was unrouted. It needs no new grammar: the
segments are {id}.cloudfront.{base}, so registering "cloudfront" is enough --
CloudFront is global, and ParseHostRoute already treats the region as optional.
Distribution.DomainName is now minted on the caller's hostname rather than the
literal cloudfront.net, which Overcast cannot serve without a DNS override;
the staging-distribution lookup takes the ID from the first label instead of
trimming a fixed suffix, so it holds for both forms.

localhost.floci.io was missing from the S3 virtual-hosted bases while
internal/containerendpoint already advertised it, so a bucket was unreachable
on a domain Overcast tells users resolves to it. Both now derive from
config.WildcardDNSDomains, and TestVirtualHostBases_coverEveryWildcardDomain
fails if they diverge again.

"cloudfront" is the first single-word label, so the guardrail that labels must
not be plausible bucket-name segments now takes a written rationale rather than
requiring a hyphen -- generic words still cannot be added.

docs/networking.md gains an inventory of every known AWS resource subdomain and
whether Overcast routes it. AWS publishes no such list: the SDK endpoint
rulesets and Smithy's endpointPrefix cover control-plane endpoints only, and
carry none of execute-api, lambda-url or appsync-api.
Guidance was inconsistent and in one place pointed the wrong way: cdk.md's
worked example set OVERCAST_HOSTNAME=localhost.localstack.cloud, so the
canonical CDK-on-Windows fix recommended another project's domain. networking.md
listed plain localhost first as the default and the wildcard domains as an
alternative, which is backwards for anyone on Windows, where *.localhost does
not resolve at all.

localhost.overcast.sh is now the recommendation in networking.md, cdk.md, the
env-var table and the S3 service doc. localhost.localstack.cloud and
localhost.floci.io stay recognised and are documented as such, so a setup
carried over from either tool keeps working -- they are just no longer what a
new setup is steered towards.

Where it does not work stays surfaced rather than buried: all three wildcard
domains need a public DNS lookup, so none works offline or behind DNS rebinding
protection, and the fallbacks (plain localhost on Linux/macOS, a hosts-file
entry anywhere) are called out at each site.

The S3 doc also gains the bare-form base list and the reserved-label exception,
which it predated.
@Neaox
Neaox merged commit b6c9188 into main Jul 28, 2026
41 checks passed
@Neaox
Neaox deleted the claude/appsync-realtime-and-base-parity branch July 28, 2026 09:55
Neaox added a commit that referenced this pull request Jul 28, 2026
…351)

* docs: audit AWS::URLSuffix use sites and realign the addressing tables

H4's gate was to classify every AWS::URLSuffix use site in synthesised CDK
templates before touching the resolver. Done, and the result contradicts the
hazard the plan recorded: there are no IAM service-principal use sites.

Synthesised compat/suites/cdk (aws-cdk-lib 2.220.0, CDK CLI 2.1133.0) -- S3,
SQS, SNS, Lambda, IAM roles and managed policies, API Gateway REST,
EventBridge, Step Functions, nested stack -- and walked the template for every
{"Ref": "AWS::URLSuffix"} node. Exactly two, both URL hosts: the API Gateway
endpoint output and a nested stack's TemplateURL. Principals are emitted as
literal strings ("lambda.amazonaws.com", "states.amazonaws.com"), because CDK
v2 resolves them through its region-info database rather than joining over the
suffix.

The TemplateURL site is in fact already broken -- it resolves to
https://s3.us-east-1.amazonaws.com/..., which Overcast cannot fetch -- so
nested stacks need the substitution rather than merely tolerating it. And
EnforceIAM defaults to false, so a mis-substituted principal would be inert
even in the residual hand-written-template case.

Also realigns docs/networking.md, which #350 left half-updated: the
appsync-realtime-api and cloudfront labels were added to the new inventory
table but not to the "what works today" table, the precedence rule, or the
reserved-label note.

* fix(cloudformation): return stack output URLs on a reachable origin

CDK composes an API Gateway invoke URL in the template itself:

  ["https://", {"Ref":"Api"}, ".execute-api.us-east-1.",
   {"Ref":"AWS::URLSuffix"}, "/", {"Ref":"Stage"}, "/"]

The scheme is a literal and there is no port, so DescribeStacks handed back
https://abc.execute-api.us-east-1.amazonaws.com/prod/ -- a URL that resolves to
real AWS. AWS::URLSuffix cannot express a scheme or a port, so substituting it
can never produce a dialable URL, and a value containing ":4566" would make the
pseudo-parameter lie about what it is. It still resolves to amazonaws.com, as
it does on AWS.

The correction belongs at output emission, where Overcast has already assembled
the finished string and still holds the request context. Handler.reachableURL
parses with middleware.ParseHostRoute -- the same grammar that decides inbound
routing -- and re-mints through serviceutil.HostRoutedURLFromBase, the helper
every service handing back such a URL already uses. The grammar is stated once
and applied in both directions, so a rewritten output is by construction a URL
this router accepts.

Only a registered host-route label is claimed. ECR registry URIs, S3 URLs, ARNs
and plain strings pass through untouched, because Overcast does not serve those
hostnames and must not pretend to. Both the Query and typed (Smithy) paths share
one implementation.

Two corrections to the plan, both from evidence:

  - The IAM service-principal hazard does not exist. Synthesising for
    us-east-1, cn-north-1 and region-agnostic shows every URLSuffix use site is
    a stack Output, and principals are emitted as literals in all three -- even
    with an explicit region on the principal.
  - The nested-stack TemplateURL was called a latent bug. It is not: both
    fetchers dispatch u.Path internally and discard the host, so Overcast never
    dials it.

* docs: align service and SDK docs with host-based addressing

H6. Every doc that states which hostnames or URL forms Overcast supports,
reviewed against what it now does.

Two were actively wrong rather than merely incomplete:

  - migration-from-localstack.md said virtual-hosted style "requires DNS
    resolution of *.localhost which doesn't work without extra configuration",
    and told readers to force path-style. Both forms work, the bare one needs
    no s3. prefix (unlike LocalStack), and OVERCAST_HOSTNAME makes it resolve
    everywhere.
  - sdk-cli.md linked to #s3-asset-upload-fails-on-windows-or-macos; the
    heading is #s3-asset-upload-fails-on-windows, so the link was dead.

apigateway.md and appsync.md gained the endpoint-URL sections they never had:
the host-routed forms, what apiEndpoint and the uris/dns maps report and why
they differ, and why dns.REALTIME reports the appsync-api host. Both inserted
before the generated capability block, which is left untouched.

The reserved-label exception is now stated wherever bucket naming is
documented, not only in networking.md -- a user meets it at CreateBucket, not
while reading about DNS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant