Skip to content

Build a card for a site that will not show us its page (GRYT-913) - #134

Merged
sivert-io merged 2 commits into
mainfrom
claude/GRYT-913-link-resolvers
Sep 4, 2026
Merged

Build a card for a site that will not show us its page (GRYT-913)#134
sivert-io merged 2 commits into
mainfrom
claude/GRYT-913-link-resolvers

Conversation

@sivert-io

Copy link
Copy Markdown
Member

A MakerWorld link draws an empty card. Not because the page publishes no OpenGraph — the page is never served to us at all:

HTTP/2 403
cf-mitigated: challenge
server: cloudflare

Cloudflare's managed challenge, to the GrytBot agent and to a current Chrome agent alike. What the parser gets is 5.8 KB of "Just a moment…". Headless Chrome would be fighting that, and a method that passes today and stops when Cloudflare retunes is worse than a bare card.

Their own front end reads the model from an API that is not behind the challenge, and answers 200 to the ordinary GrytBot agent with no pretending to be a browser:

GET https://makerworld.com/api/v1/design-service/design/1642496

The shape

linkResolvers.ts is a small registry of site-specific ways in, tried before the ordinary fetch and only for a host that has one. Opt-in, not a fallback for anything that came back thin — a registry that guessed would spend a request on every dead link to learn it had nothing.

A resolver returns metadata or null, never half. Null — wrong shape, id not found, request refused — falls through to the OpenGraph path, so the worst case is exactly today's behaviour. Most of the test file is about that.

Three things about the MakerWorld one

  • The cover cannot be derived from the model id. The path carries the uploader's id (USc2288dd345d5c5) and a per-upload date and hash (2025-07-31_f2d4e3ba416b2). Neither is a function of 1642496, which is what the request is for.
  • The image URL asks for what a card needs. That CDN is Alibaba OSS, so the processing parameters are ours. Measured: raw is 1.68 MB, application/octet-stream; resize,w_640/format,webp is 75.5 KB, image/webp. The content type matters as much as the size — the raw URL does not declare itself an image.
  • It claims model URLs only. A search page or a profile goes down the ordinary path rather than costing a request to find out.

What to look at

  • jsonFetcher in the route. This is the part I'd most want a second pair of eyes on. A resolver gets its fetcher passed in rather than importing one, so it cannot route around the guards — same hop-by-hop private-host check, same abort signal, 512 KB cap, and nothing parsed that does not call itself JSON. If there's a hole, it's there.
  • status: 200 on a resolved card. The client reads that to tell "publishes no metadata" from "gone". The HTML page's own 403 isn't the answer to what happened, but it is a judgement call.

Verification

yarn test (736), yarn build and npx eslint . all pass; the 5 eslint warnings are pre-existing and in untouched files.

Checked against the live site, not only the fixture:

real model title, description, author, date, and an image answering 200 image/webp 75,550 bytes
id 99999999999 null → falls back
/en/search?q=box not claimed at all

Mutation-tested, 6 mutations, all caught. One of them found a genuine gap — weakening the tag strip to <p> alone left every case passing, because </p> becomes a space first, so a summary of pure paragraphs is identical either way. A link or an image in one would have reached the card as literal markup. Second commit adds that case.

Not in this PR

A branded MakerWorld card. siteName, author and publishedAt mean the standard card already draws well; a logo and brand colour would go in links/providers.ts in @gryt/core, which is a client-side table and a separate release chain.

🤖 Generated with Claude Code

sivert-io and others added 2 commits September 4, 2026 20:58
A MakerWorld link draws an empty card, and not because the page publishes no
OpenGraph. The page is never served to us:

    HTTP/2 403
    cf-mitigated: challenge
    server: cloudflare

Cloudflare's managed challenge, to the GrytBot agent and to a current Chrome
agent alike. What the parser gets is 5.8 KB of "Just a moment…".

Their own front end reads the model from an API that is not behind that, and
answers 200 to the ordinary GrytBot agent with no pretending to be a browser:

    GET https://makerworld.com/api/v1/design-service/design/1642496

So `linkResolvers.ts` is a small registry of site-specific ways in, tried
before the ordinary fetch and only for a host that has one. Opt-in rather than
a fallback for anything that came back thin: a registry that guessed would
spend a request on every dead link to learn it had nothing.

**A resolver returns metadata or null, never half.** Null — wrong shape, id not
found, request refused — falls through to the OpenGraph path, so the worst case
is exactly the behaviour there was before. Most of the test file is about that.

Three things worth knowing about the MakerWorld one:

- **The cover cannot be derived from the model id.** The path carries the
  uploader's id and a per-upload date and hash, so it has to be looked up. That
  is what the request is for.
- **The image URL asks for what a card needs.** That CDN is Alibaba OSS, so the
  processing parameters are ours: the raw cover is 1.68 MB and arrives as
  `application/octet-stream`, and `resize,w_640/format,webp` is 75.5 KB of
  `image/webp`. The content type matters as much as the size — the raw URL does
  not declare itself an image.
- **It claims model URLs only.** A search page or a profile is left to the
  ordinary path rather than costing a request to find out.

The resolver reaches the network through a fetcher passed in by the route, not
by importing one. It keeps `linkResolvers.ts` testable with no network, and it
means a resolver cannot route around the guards: the same hop-by-hop private
host check, the same abort signal, a 512 KB cap, and nothing parsed that does
not call itself JSON.

Checked against the live site rather than only the fixture. A real model comes
back with a title, a description, an author, a date and an image that answers
200 image/webp at 75,550 bytes; id 99999999999 comes back null; a search URL is
not claimed at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mutation testing found it: weakening the tag strip to `<p>` alone left every
case passing. `</p>` is turned into a space before the strip runs, so a summary
made only of paragraphs comes out identical either way — and a link or an image
in one would have reached the card as literal markup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sivert-io
sivert-io merged commit ad80e46 into main Sep 4, 2026
2 checks passed
@sivert-io
sivert-io deleted the claude/GRYT-913-link-resolvers branch September 4, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant