Separate App Store absence from transport failures

[kasnder]

Two fixes to how the metadata refresher classifies App Store errors. Both are independent of each other; neither changes the refresh schedule or its request volume.

Absence is not a 404

Apple answers an unknown bundle ID with HTTP 200 and an empty result set — verified against the live endpoint, for both the bundleId and numeric id forms:

Request	Status	Body
bundleId=com.definitely.not.a.real.app.zzz999	200	{"resultCount":0,"results":[]}
bundleId=com.apple.Pages	200	resultCount:1

lib/appStore.js was synthesising an App not found (404) error for that case, and refresh-app-store-metadata.js was regex-matching the number back out of the message it had just written. A genuine HTTP 404 from the transport produced a different message but parsed to the same 404, so it landed in the same bucket — and was exempted from the consecutive-failure cap, as though six apps had left the store simultaneously rather than the endpoint being unreachable.

Errors raised by requestJson now carry statusCode and retryAfter from the response, and inferred absence is flagged with absent: true. Absence still bypasses the transport cap; a real 404 now counts against it. The App not found (404) message text is unchanged because routes/index.js:590 matches on it.

A rate limit no longer penalises the app it interrupted

On a 403 or 429 the run recorded a failure against whichever app happened to be next in the queue before aborting. That incremented refresh_failures, which drives the exponential backoff in the selection query — costing an innocent app up to 30 days of skipped refreshes for a rate limit aimed at the client. Repeated stops walked down the queue poisoning one app each time.

The stop now aborts without the write. The run-level failed counter still reflects it, and Retry-After is surfaced in stoppedReason when Apple sends one.

Tests

node --test — 123 passing. Updated the existing 429 test, which asserted the old write, and added coverage for the Retry-After reason, a real 404 consuming the cap, and absence classification by both flag and legacy message shape. The new lib/appStore.js test hits the live endpoint, consistent with the others in that file.

Not included

Also discussed but deliberately left out: a canary app per run and an empty-result-rate cap, both proposed to distinguish a soft ban from a delisting. That distinction turned out not to need them — a ban arrives as 403/429 or a dropped connection, which the existing stop conditions already separate at the transport layer. The not-found semantics around regional availability are untouched; country still defaults to gb, so an app pulled from the UK store while alive elsewhere is still recorded as app_not_found.

---

Generated by Claude Code