Build a card for a site that will not show us its page (GRYT-913) - #134
Merged
Conversation
A MakerWorld link draws an empty card, and not because the page publishes no
OpenGraph. The page is never served to us:
HTTP/2 403
cf-mitigated: challenge
server: cloudflare
Cloudflare's managed challenge, to the GrytBot agent and to a current Chrome
agent alike. What the parser gets is 5.8 KB of "Just a moment…".
Their own front end reads the model from an API that is not behind that, and
answers 200 to the ordinary GrytBot agent with no pretending to be a browser:
GET https://makerworld.com/api/v1/design-service/design/1642496
So `linkResolvers.ts` is a small registry of site-specific ways in, tried
before the ordinary fetch and only for a host that has one. Opt-in rather than
a fallback for anything that came back thin: a registry that guessed would
spend a request on every dead link to learn it had nothing.
**A resolver returns metadata or null, never half.** Null — wrong shape, id not
found, request refused — falls through to the OpenGraph path, so the worst case
is exactly the behaviour there was before. Most of the test file is about that.
Three things worth knowing about the MakerWorld one:
- **The cover cannot be derived from the model id.** The path carries the
uploader's id and a per-upload date and hash, so it has to be looked up. That
is what the request is for.
- **The image URL asks for what a card needs.** That CDN is Alibaba OSS, so the
processing parameters are ours: the raw cover is 1.68 MB and arrives as
`application/octet-stream`, and `resize,w_640/format,webp` is 75.5 KB of
`image/webp`. The content type matters as much as the size — the raw URL does
not declare itself an image.
- **It claims model URLs only.** A search page or a profile is left to the
ordinary path rather than costing a request to find out.
The resolver reaches the network through a fetcher passed in by the route, not
by importing one. It keeps `linkResolvers.ts` testable with no network, and it
means a resolver cannot route around the guards: the same hop-by-hop private
host check, the same abort signal, a 512 KB cap, and nothing parsed that does
not call itself JSON.
Checked against the live site rather than only the fixture. A real model comes
back with a title, a description, an author, a date and an image that answers
200 image/webp at 75,550 bytes; id 99999999999 comes back null; a search URL is
not claimed at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mutation testing found it: weakening the tag strip to `<p>` alone left every case passing. `</p>` is turned into a space before the strip runs, so a summary made only of paragraphs comes out identical either way — and a link or an image in one would have reached the card as literal markup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A MakerWorld link draws an empty card. Not because the page publishes no OpenGraph — the page is never served to us at all:
Cloudflare's managed challenge, to the GrytBot agent and to a current Chrome agent alike. What the parser gets is 5.8 KB of "Just a moment…". Headless Chrome would be fighting that, and a method that passes today and stops when Cloudflare retunes is worse than a bare card.
Their own front end reads the model from an API that is not behind the challenge, and answers 200 to the ordinary GrytBot agent with no pretending to be a browser:
The shape
linkResolvers.tsis a small registry of site-specific ways in, tried before the ordinary fetch and only for a host that has one. Opt-in, not a fallback for anything that came back thin — a registry that guessed would spend a request on every dead link to learn it had nothing.A resolver returns metadata or null, never half. Null — wrong shape, id not found, request refused — falls through to the OpenGraph path, so the worst case is exactly today's behaviour. Most of the test file is about that.
Three things about the MakerWorld one
USc2288dd345d5c5) and a per-upload date and hash (2025-07-31_f2d4e3ba416b2). Neither is a function of1642496, which is what the request is for.application/octet-stream;resize,w_640/format,webpis 75.5 KB,image/webp. The content type matters as much as the size — the raw URL does not declare itself an image.What to look at
jsonFetcherin the route. This is the part I'd most want a second pair of eyes on. A resolver gets its fetcher passed in rather than importing one, so it cannot route around the guards — same hop-by-hop private-host check, same abort signal, 512 KB cap, and nothing parsed that does not call itself JSON. If there's a hole, it's there.status: 200on a resolved card. The client reads that to tell "publishes no metadata" from "gone". The HTML page's own 403 isn't the answer to what happened, but it is a judgement call.Verification
yarn test(736),yarn buildandnpx eslint .all pass; the 5 eslint warnings are pre-existing and in untouched files.Checked against the live site, not only the fixture:
200 image/webp 75,550 bytes99999999999null→ falls back/en/search?q=boxMutation-tested, 6 mutations, all caught. One of them found a genuine gap — weakening the tag strip to
<p>alone left every case passing, because</p>becomes a space first, so a summary of pure paragraphs is identical either way. A link or an image in one would have reached the card as literal markup. Second commit adds that case.Not in this PR
A branded MakerWorld card.
siteName,authorandpublishedAtmean the standard card already draws well; a logo and brand colour would go inlinks/providers.tsin@gryt/core, which is a client-side table and a separate release chain.🤖 Generated with Claude Code