Skip to content

Releases: hdae/fetch-cache

v0.8.0

Choose a tag to compare

@hdae hdae released this 07 Sep 05:10

This release adds range reads on cached entries: openCachedUrl (and openHfFile on the HuggingFace layer) opens an entry that is already in the cache and reads just the byte range you ask for — one row out of a multi-hundred-MB table instead of the whole thing — without ever touching the network. The API is additive — no function, type, default or error-message prefix changes and no migration is needed.

✨ Features

  • openCachedUrl(url, options?) — open a cached entry and read ranges from it. Returns { read, strategy } for a cached entry and undefined when there is none; it never goes to the network, so warming the entry stays the caller's job (fetchBytes / prefetchUrl). read(offset, length, { signal? }) resolves to a new Uint8Array of exactly length bytes; a range past the end of the entry, or a body that ends early, throws instead of returning a short array, and a negative or non-integer offset / length throws before anything is read. fetchBytes is unchanged and still reads a cache hit in full.
  • Two read strategies, chosen per runtime. A browser Blob is a lazy handle to the stored entry, so "blob" (response.blob() once when opening, then blob.slice(...) per read) is constant-time there; Deno's blob() loads the whole entry into memory, so under Deno the default is "stream", which re-reads the body from the cache on every read and skips forward to the offset — a cost proportional to the offset. The default is "stream" when globalThis.Deno exists and "blob" otherwise; strategy: "blob" | "stream" forces one (any other value throws rather than silently falling back), and strategy on the returned entry tells you which one is in use. options.signal on read aborts a "stream" read at the next chunk boundary and rejects with signal.reason; the "blob" strategy checks the signal once before slicing.
  • Verification is the recorded hash, as a string. With sha256, the entry opens only when its recorded x-fetch-cache-sha256 matches — the same string comparison a fetchBytes cache hit makes; the bytes themselves are never hashed, since a range read cannot compute the hash of the whole entry. A mismatch self-heals: the entry is evicted and undefined comes back. An entry with no recorded hash also returns undefined but is not evicted — read it once through fetchBytes with the same sha256 and the record is backfilled, after which it opens. Without sha256 the entry opens as is; a malformed sha256 (anything but 64 lowercase hex digits) throws before the cache is touched, as it does on fetchBytes. There is deliberately no validate / decode / recheck / into: a range read only ever sees the stored raw form, and full-entry verification stays with fetchBytes.
  • Failures while opening degrade, failures while reading throw. A cache I/O failure when opening (caches.open, cache.match, or blob() under the "blob" strategy) is treated like a miss — undefined — and reported through onCacheError (a blob() failure is reported as op: "match", a failed self-heal delete as op: "delete"), whose default handler says the entry is treated as absent rather than that the call falls back to the network. A failure during read throws, because there is nothing to fall back to. On a runtime without caches at all, both openCachedUrl and openHfFile return undefined rather than throwing (the openHfFile check for a missing sha256 still runs first). An open handle is not a snapshot: a "stream" handle re-reads the cache on every call, so it throws once the entry has been evicted (evictUrl / evict / a self-heal; after clearCache it depends on the runtime: under Deno 2.9 the Cache object the handle holds stops matching once the namespace is deleted, so it throws there too, while browsers are unmeasured) — and, when opened with sha256, it re-checks the recorded hash on every read and throws if another write has replaced the entry under it (a self-heal refetch with a different sha256, or any write that lands on the same key after the entry was evicted). A "blob" handle keeps the Blob it opened and can still be read after the entry is gone.
  • openHfFile(ref, spec, options?) on the HF layer does the same for a file whose sha256 is declared: that is the content key, which does not contain the revision, so nothing is resolved and nothing is fetched — fetchHfFile / prefetchHfFile warm the same key. A spec without sha256 throws (its key is the revision-pinned resolve URL, and resolving that would need the network). The spec's format checks still run (a negative or non-integer expectedBytes throws with the same message as on fetchHfFile), even though expectedBytes / validate / decode / into play no part in a range read. strategy, onCacheError and caches are passed through to the cache layer.
  • HfResolveOptions — the options type of resolveHfRevision (fetch / init / retry / onRetry), now exported from ./hf so a wrapper can take the same options without copying the declaration. No change in behavior.
  • New exported types: CachedEntry and OpenCachedOptions from ., CachedEntry, HfOpenOptions and HfResolveOptions from ./hf.

📝 Docs

  • README gains a Reading a range out of a cached entry section (and a "Range reads on cached entries" feature bullet). The design — why a separate open/read API instead of a fetchBytes option, why two strategies, why verification is the recorded hash only, and the measured numbers behind the runtime default — is recorded in ADR 0012; the by-design constraints (no verification of the bytes read, no network and no single-flight, offset-proportional cost under Deno, handles are not snapshots, aborts land on chunk boundaries) are in limitations; that "stream" needs a body it can stream is noted in ADR 0012 and on OpenCachedOptions.strategy.

Full Changelog: v0.7.0...v0.8.0

v0.7.0

Choose a tag to compare

@hdae hdae released this 05 Sep 21:12

This release makes rate limits survivable: a 429 or 503 is now retried automatically, honoring the server's Retry-After, and on the HuggingFace layer expectedBytes also acts as an upper bound so a response that overruns its declared length is cut off instead of downloaded in full. The API is additive — no function, type or error-message prefix changes and no migration is needed — but two defaults change: a 429 / 503 now waits and retries instead of throwing at once (retry: false restores the old behavior), and prefetchHfFile now reads expectedBytes from the spec (see below for the two cases that turn from success into a throw).

✨ Features

  • Retries for 429 / 503, on by default (retry on fetchBytes / prefetchUrl / fetchHfFile / fetchHfFiles / prefetchHfFile / resolveHfRevision; in the HF layer retry / onRetry travel to both the revision resolution and the file download, like init). Defaults are { statuses: [429, 503], maxRetries: 5, baseDelayMs: 1000 }. A Retry-After header wins when present — delta-seconds, or an HTTP-date (a date in the past means "now"); only a value starting with a letter is handed to Date.parse, so anything else — a malformed number such as 1.5 or +120 included — counts as no header. The header is followed as given unless you set maxDelayMs. Without the header the wait doubles from baseDelayMs (1, 2, 4, 8, 16 seconds); the exponent counts every retry, so a retry that followed a Retry-After advances it too. The cap is per wait, so a server that keeps answering Retry-After: 3600 is waited out once per retry (five hours at the default maxRetries); a server that keeps answering 503 without the header takes up to 31 seconds before the call rejects. Only GET and HEAD are retried: any other method (reachable through cache: false) throws on the first 429 / 503 as before. Waiting respects the AbortSignal passed via init (an abort rejects with signal.reason), the failed response's body is released before each wait, and onRetry is called before each wait with { url, status, attempt, delayMs, retryAfter } — a throwing listener is warned about and ignored, like onProgress. Connection errors, transfers cut off mid-body and every other status still throw on the first response. A call that joins an in-flight single-flight fetch waits out the leader's retries; its own retry / onRetry go unused, like the other joiner options.
  • When the retries run out, the error keeps its prefix. It is still fetch-cache: HTTP 429 Too Many Requests (url), with (再試行 N 回の後) appended whenever at least one retry ran — also when the final status is not 429 / 503 (a 429 that turns into a 404 reports the 404 with the suffix). Matching on the prefix keeps working; matching on the whole string will not once a retry has happened. A wait setTimeout cannot hold (longer than 2**31 - 1 ms, about 24.8 days) is not honored and not retried: that round's response comes back as it is, uncounted and without an onRetry call, while retries already made still count — pass maxDelayMs to cap the wait and retry anyway.
  • Malformed retry policies throw before any request is made: maxRetries must be a safe integer ≥ 0, baseDelayMs / maxDelayMs finite numbers ≥ 0, and every entry of statuses an integer in 400–599 (NaN would otherwise slip past the wait bound, and a maxRetries of NaN would never stop). The message starts with fetch-cache: retry.<field> は.
  • New exported types RetryPolicy and RetryContext, from both . and ./hf (the same declarations).
  • HF expectedBytes is an upper bound as well. A response that grows past the declared length is cut off at the chunk that crosses it and throws fetch-cache: 受信が申告 N バイトを超えた(M バイト以上) (url) right there, instead of being downloaded in full only to fail the exact-length check at the end (a short response still fails at the end, as before, with the familiar バイト数不一致 message). With into, an overrun is reported as this declaration violation, not as a too-small buffer. expectedBytes: 0 is a valid declaration for an empty file. The overrunning read is never reported to onProgress. On a runtime whose response.body is null the check runs after the whole body has arrived, so only the error changes, not the bytes transferred (and no progress event is emitted for it). The generic layer's expectedBytes stays an allocation hint and never a check.
  • prefetchHfFile now reads expectedBytes from the spec — as an upper bound only. A response overrunning it never becomes a cache entry (a short one is still not detected while warming; the exact-length check belongs to the read). Two cases that succeeded in 0.6.0 therefore now throw: a warming spec whose expectedBytes is smaller than the actual file while the reading spec declares no length (with the same declaration on both sides the read already failed and self-healed), and a malformed expectedBytes (negative, non-integer, or beyond the safe-integer range), which is now rejected before any request on every HF entry point with fetch-cache: expectedBytes は 0 以上の整数で指定してください: <value> (<path>) (fetchHfFile already failed on it, at the end of the download).
  • When a bad prefetch entry cannot be removed by the safety-net delete, the AggregateError names which check failed (SHA-256 mismatch or received-bytes overrun); the SHA-256 wording is unchanged.

📝 Docs

  • README gains a Rate limiting (429) and retries section, and the HuggingFace section describes the upper bound and what prefetchHfFile reads. The designs are recorded in ADR 0010 (including why no jitter is added to the waits and why there is no default cap) and ADR 0011; the contracts (what joiners inherit, cumulative waits, the missing-entry window while a self-heal refetch waits) are in limitations.

Full Changelog: v0.6.0...v0.7.0

v0.6.0

Choose a tag to compare

@hdae hdae released this 02 Sep 17:47

This release adds into: read a download — or a cache hit — straight into a buffer you allocated and get a prefix view of it back, so a sequential loop over dozens of large shards reuses one buffer instead of allocating one per call. It is additive: no API is removed or changed and no migration is needed. One caveat: the JavaScript floor moves up to Baseline 2024 (see below).

✨ Features

  • into — caller-owned buffers (FetchBytesOptions.into, and per file as HfFileSpec.into for fetchHfFile / fetchHfFiles — one buffer per spec, since fetchHfFiles runs in parallel). A network download is written into your buffer instead of a freshly allocated one, and a cache hit is read out of Cache Storage through its body stream into it instead of being materialized with arrayBuffer(). The return value is into.subarray(0, n); the buffer stays yours, so the view (and the raw bytes handed to validate / decode) is only valid until the next write into the same buffer. With decode, the buffer holds the stored form and the decoded form is returned separately. With into, expectedBytes is no longer an allocation hint: what remains is the pre-check that rejects a declared length the buffer cannot hold. prefetchUrl / prefetchHfFile have no use for into (they never hold the bytes).
  • Too-small buffers fail loud instead of degrading. A network download is cancelled at the chunk that overflows and nothing is cached; a cache hit throws but keeps the entry (it is neither corruption nor a cache I/O failure, so it neither self-heals nor falls back to the network); an expectedBytes larger than the buffer throws before any request is made — in the HuggingFace layer that includes the revision-resolution request. The error is a plain Error with name === "IntoCapacityError"; after any throw the buffer's contents are undefined.
  • Concurrent use of one buffer is rejected at the entrance. Passing the same buffer to two in-flight calls (for example the same into on several HfFileSpecs of a parallel fetchHfFiles) throws at the cache layer's entry point, before that call's own download starts, because interleaved writes could otherwise store bytes that do not match the recorded hash. Reuse a buffer sequentially; a SharedArrayBuffer-backed view is rejected at the same point.
  • Single-flight keeps caller-owned buffers private. A caller that joins an in-flight download never receives the leader's into buffer: when the leader used into and someone has joined, the leader copies the stored bytes once before sharing them — so that flight peaks at twice the size, and a failure to allocate that copy throws even though the download, verification and cache write all succeeded (the retry is a cache hit). A joiner with its own into gets the bytes copied into it. Nothing is copied for sharing when nobody joins. Note that a capacity failure on the leader's buffer fails the whole flight, joiners without into included.
  • sha256 verification of materialized bytes now hands partial views straight to WebCrypto (only a SharedArrayBuffer-backed view is copied first), so an into prefix view is hashed without a JS-level copy. WebCrypto still takes its own copy internally, as the spec requires.

⚡ Runtime floor

  • Baseline 2024 JavaScript. The single-flight path now uses Promise.withResolvers (Safari 17.4, Chrome 119, Firefox 121, Deno 1.38, Node.js 22 and later); 0.5.0 did not use it anywhere in the library, so on a runtime below that line calls that worked in 0.5.0 will now throw. The TypeScript floor is unchanged (5.7 or later, where Uint8Array takes a type argument) but is now visible in the public API: into is typed Uint8Array<ArrayBuffer>.

📝 Docs

  • README gains a Reusing one buffer across many reads (into) section stating what into does and does not save (a joined flight's copy, body-less runtimes, Cache implementations that do not stream, WebCrypto's own copy); the design is recorded in ADR 0009 and the contract in limitations. Runtime support now records the JavaScript and TypeScript floors above.

Full Changelog: v0.5.0...v0.6.0

v0.5.0

Choose a tag to compare

@hdae hdae released this 28 Aug 18:22

This release redesigns the cache-control API around a first-class sha256 option: bytes are verified when stored, the hash is recorded on the entry, and later cache hits are decided by a plain string comparison — so multi-GB models no longer need a full re-hash on every startup. This is a breaking release — see Migrating from 0.4.0 in the README for the upgrade path.

💥 Breaking Changes

  • cacheName is gone. The namespace is a single fixed "fetch-cache". The HF layer's old "fetch-cache-hf" namespace is no longer referenced — reclaim the space once with caches.delete("fetch-cache-hf"). The admin APIs (clearCache / listCachedUrls) no longer take a positional cacheName; they take an options object ({ caches }) instead. Careful in plain JS: clearCache("...") does not error — the string is ignored and the default namespace is cleared.
  • cacheName-based partitioning has no direct successor. evict prefixes only address the HF content keys; to partition per app / per tenant, pass a custom caches implementation (e.g. a thin wrapper remapping open(name)) to every call, or keep the URLs distinct.
  • verifiedMarker / trustCachedSha256 are gone, absorbed by sha256 + recheck — and the default is reversed: 0.4.0 validated every hit unless you opted in; 0.5.0 trusts a matching recorded hash unless you pass recheck: true. The old x-fetch-cache-verified header is not read (treated as "no record"): such entries are re-hashed once on their first verified read and the record is backfilled, after which hits are string comparisons again.
  • HF default cache keys changed. Files that declare a sha256 are cached under a content key ["hf", kind, repo, path, sha256] — a README-only revision bump is still a cache hit, and switching revisions keeps both contents side by side. Entries cached by 0.4.0 simply miss and are re-downloaded (it is a cache; no data is lost).
  • An explicit expectedBytes now fails loud when the receive buffer cannot be allocated, throwing before a single byte is read instead of downloading everything and failing at the end for the same reason (ADR 0007). A server-declared content-length still degrades as before.

✨ Features

  • First-class sha256 with a recorded hash — fetchBytes now takes sha256 directly (0.4.0 had it only in the HF layer): verify on download, record the hash on the entry, and decide later hits by string comparison. A mismatching record means "the content changed" and self-heals (evict + refetch); recheck: true re-hashes the bytes when you don't want to trust local storage. Entries without a record are hashed once on their first verified read and backfilled.
  • prefetchUrl / prefetchHfFile now bake the recorded hash into entries they verify in flight (replacing the 0.4.0 marker), and compare an existing entry's record against the expectation — a stale or unverified entry is re-warmed with verification, and a failed re-warm leaves the existing entry intact.
  • Prefix management — new evict(prefix) / listKeys(prefix) operate on the HF content keys (evict(["hf", "model", "owner/name"]) frees a repo's sha256-declared files). Files fetched without a sha256 are URL-keyed; sweep them with listCachedUrls + evictUrl (recipe in the README).
  • caches injection on all five admin APIs (CacheAdminOptions) — for test isolation, fault injection, and partitioning.
  • ./hf now re-exports the cache-layer types used in its public signatures (FetchProgress, CacheErrorContext, ValidateBytes, DecodeBytes).

🐛 Fixes

  • The single-flight join key now matches the Cache API's URL normalization — concurrent calls for differently-spelled URLs of the same resource (uppercase host, default port, fragment) join one download instead of fetching twice.
  • The HF entry points validate the file spec (sha256 format) before the revision-resolution request — a malformed declaration no longer reaches the network (0.4.0 leaked one resolution request for mutable refs, and fetchHfFiles could start sibling downloads).
  • A trailing slash in hubUrl is absorbed (no more double-slash URLs causing 404s or duplicate cache keys).
  • A failing insurance delete in prefetchUrl is no longer swallowed; it is reported together with the integrity error as an AggregateError.
  • A caller joining an in-flight download during a progress notification no longer receives the same progress event twice.

📝 Docs

Full Changelog: v0.4.0...v0.5.0

v0.4.0

Choose a tag to compare

@hdae hdae released this 09 Aug 05:55

Full Changelog: v0.3.1...v0.4.0

v0.3.1

Choose a tag to compare

@hdae hdae released this 17 Jul 22:37

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Choose a tag to compare

@hdae hdae released this 10 Jul 13:44

Full Changelog: v0.2.0...v0.3.0

v0.2.0

Choose a tag to compare

@hdae hdae released this 10 Jul 11:21

Full Changelog: v0.1.0...v0.2.0