v0.9.0

Batch loading for views, iterators, and S3-FIFO

ViewOptions.BatchLoader and View.GetManyOrLoad. Views load many keys in one upstream call, by the caller's keys, with the cache's semantics: single flight and invalidation per key, GetOrLoad through the batch loader when there is no Loader, due keys refreshed in one call. ViewStats.Batches counts the calls.

Cache.All and View.All iterate over the cached values, for dumping a cache or warming another from it. The loop may use the cache; negative and expired entries are skipped, and nothing is counted or reordered.

S3FIFO, a new eviction policy: new entries wait in a small queue until they are read again, so keys read once leave before they push out the ones in steady use. Reads take only a read lock, as with ClearOnFull, and Set still always stores. In the comparison benchmarks, with 4096 entries against 100,000 keys on zipf s=1.01, it reaches 71.3% hits against 65.8% for LRU and 71.5% for otter, at 20 ns per read on eight cores.

A cache of any used through views needs no Cost under MaxBytes when every view brings one: NewView requires it then, and values written to the cache directly are charged as negative entries are.

Fixed: since 0.8.0, a failed load had its zero value passed to Cost, so a Cost that reads through a nil pointer turned the load's error into a panic.

Breaking: requires Go 1.25; New and NewView panic on a TTL, NegativeTTL or RefreshAfter shorter than ten ClockGranularity intervals; ViewStats gained Batches, so code that builds it positionally needs updating. See CHANGELOG.md.

Install:

go get github.com/andared/sanecache@v0.9.0

v0.8.0

Hot keys that stay warm, costs that come with the value, and increments for exporters

Options.RefreshAfter and ViewOptions.RefreshAfter. A read that finds a value older than this returns it at once and reloads it in the background, so a key in steady use is replaced before it expires instead of making the next caller wait for the upstream. Each value gets one refresh, shared with the key's other loads, and invalidation keeps a stale result out. The TTL still holds: a failed refresh is not retried, and the value lives out its TTL. GetOrLoad, GetManyOrLoad and GetOrLoadFunc refresh with their loaders, Get and Lookup with the ones in Options; GetManyOrLoad refreshes its due keys in one BatchLoader call. Off by default. Counted in the new Refreshes and RefreshErrors, apart from Loads.

SetWithCost and GetOrLoadFuncWithCost on Cache and View take the value's cost along with it, for values whose size is known only where they are produced, such as the response they were decoded from.

Stats.Sub and ViewStats.Sub return the counters gained since a previous snapshot, which is what a statsd exporter sends each interval.

The doc comments and the README are shorter, and the GetOrLoadFunc and GetManyOrLoad examples on pkg.go.dev no longer carry each other's descriptions.

Stats and ViewStats gained fields: code that builds them positionally needs updating. See CHANGELOG.md.

Install:

go get github.com/andared/sanecache@v0.8.0

v0.7.0

Loads that need more than the key, and load errors that are errors

Cache.GetOrLoadFunc and View.GetOrLoadFunc take the loader with the call. A loader in Options sees only the key, so an upstream call that needs a query string, a header or a request the caller has already built had to be encoded into the key and parsed back out. Everything else is GetOrLoad's: single flight, storage with the cache's or view's cost and TTLs, ErrNotFound as a negative entry, the error, cancellation and panic policies. Loads are shared with GetOrLoad and GetManyOrLoad of the same key, so callers of one key wait for whichever load started first, and the function must return what any other caller's would for that key. Neither Options.Loader nor ViewOptions.Loader is needed for it.

LoadErrors no longer counts ErrNotFound. "Does not exist" is an answer, and counting it as a failure made a service asking for ids that are gone look broken on every dashboard built on LoadErrors. Those loads are counted in the new Stats.LoadNotFound and ViewStats.LoadNotFound, keys missing from a BatchLoader answer included. LoadErrors + LoadNotFound is what LoadErrors used to report.

See CHANGELOG.md for migration notes.

Install:

go get github.com/andared/sanecache@v0.7.0

v0.6.0

Loading many keys in one upstream call

Options.BatchLoader and Cache.GetManyOrLoad. Keys a batch does not find in the cache go to the upstream in one call instead of one call per key. Single flight, invalidation and cancellation work key by key: batches and GetOrLoad calls that overlap share loads rather than repeating them, and a caller who gives up on part of a batch does not cancel it for someone waiting on another part. A requested key missing from the loader's answer is cached as "does not exist"; an error fails the whole call, and nothing from that call is cached.

One loader is enough. GetOrLoad falls back to BatchLoader with a batch of one when Loader is not set, and GetManyOrLoad falls back to concurrent Loader calls when BatchLoader is not set.

Counters. Stats.Batches counts BatchLoader calls. Stats.Loads and Stats.LoadErrors now count keys rather than loader calls, which keeps Coalesced comparable with Loads; for Loader alone nothing changes. The text of ErrNoLoader names Options.BatchLoader too; compare with errors.Is, not with the string.

Views have no batch loading yet; ViewOptions is unchanged.

See CHANGELOG.md for migration notes.

Install:

go get github.com/andared/sanecache@v0.6.0

v0.5.0

Caching that can be switched off, and view stats from the cache

Disabled views. NewView accepts a nil cache and opens a view that stores nothing. It is meant for configurations where a TTL of zero means "do not cache", so code keeps calling GetOrLoad either way. The loader runs on every call, concurrent callers still share one call, and nothing is kept, ErrNotFound included. Lookups are misses, Delete reports false, and writes return the new ErrDisabled. NewView(nil, …) used to panic.

Cache.ViewStats() returns counters for every view opened on the cache, keyed by view name, so a metrics exporter needs no registry of its own. View counters now belong to the name rather than the instance: views opened with the same name on the same cache share one set, as they already share keys.

Honest cost for negative entries. Under a byte budget, a "does not exist" entry is charged 128 bytes plus the length of a string key, instead of a flat 64. BenchmarkNegativeEntryMemory measures 115–133 bytes before the key for a Cache[string, any], so a flood of lookups for missing ids used two to three times the memory the budget reported. Budgets now fill with negative entries sooner, and a shard budget below roughly 130 bytes rejects them with ErrTooLarge.

Documentation. Options.Loader no longer describes a stale-publication race that 0.4.0 removed. The text of ErrNoLoader now names both loader sources; compare with errors.Is, not with the string.

See CHANGELOG.md for migration notes.

Install:

go get github.com/andared/sanecache@v0.5.0

v0.4.0

Stronger invalidation during loading

Delete, Clear, and successful writes now prevent outstanding loaders from putting obsolete data back into the cache. New callers do not join invalidated loads; callers already waiting still receive their loader result.

The guarantee follows the storage key across typed views and parent-cache operations. Successful loader publications also supersede competing outstanding loads. Rejected writes leave loads unchanged. Tracking exists only for active loads.

Public API signatures are unchanged. This release changes late-publication behaviour: a cancelled loader that ignores cancellation can still warm the cache only if no invalidation or successful write has superseded it. Clear remains shard-by-shard, and Close only stops maintenance goroutines.

The README and API documentation describe ordering and migration implications. Regression tests cover positive and negative results, absent-key deletion, clearing, replacements, rejected writes, competing loads, and unrelated keys.

Install:

go get github.com/andared/sanecache@v0.4.0

v0.3.0

Typed views now support loaders: concurrent misses for the same key through one View share a single upstream call while all value types continue to use the parent cache's byte budget.

• Add ViewOptions.Loader and View.GetOrLoad, with typed results, per-view cost and TTLs, negative caching, cancellation and panic handling matching Cache.GetOrLoad.
• Add ViewStats.Loads, LoadErrors and Coalesced, also included in parent-cache statistics.
• Add runnable basic-cache and HTTP-loader examples, an executable view-loader example, and updated getting-started documentation.

Reuse one View instance to share in-flight work. Loaders receive unprefixed keys; separate view instances and the parent cache have independent loaders. A view without its own loader returns ErrNoLoader.

Compatibility: ViewOptions and ViewStats gained fields. Replace positional struct literals with named fields. Go 1.24 remains the minimum; the library has no dependencies.

Install:

go get github.com/andared/sanecache@v0.3.0

Full changelog: v0.2.0...v0.3.0

v0.2.0

The three things v0.1 said were next, and one the benchmarks turned up on the way.

import "github.com/andared/sanecache"

Go 1.24 or newer. Still no dependencies, and still a hard rule rather than a current state of affairs.

Single-flight loading

Options.Loader and GetOrLoad: a cold key is fetched once however many callers arrive while it is running. Written here rather than taken from golang.org/x/sync/singleflight, partly because of the dependency rule and partly because two of its behaviours were worth doing differently.

• "Does not exist" is one error either way. A loader returns ErrNotFound, that becomes a negative entry, and later callers get the same ErrNotFound. How you handle "no such object" does not depend on whether the cache happened to remember it.
• Giving up does not cancel the load for everyone else. The loader's context carries the first caller's values but is cancelled only once every waiting caller has gone. x/sync/singleflight hands the load the first caller's context, so one request timing out takes down the load the other ninety-nine are waiting for.
• Errors other than ErrNotFound are passed through and not cached, so the next call tries again. A value is published to the cache before the waiters are released, so a caller can read back what it was just handed. A panicking loader reaches the callers, with the stack of where it happened, instead of taking the process down.

Typed views over one byte budget

NewView, ViewOptions, View, ViewStats. A cache declared as Cache[string, any] can hold several value types under one budget, each view fixing a type, namespacing its keys with its name, and bringing its own Cost and TTLs if it wants them. Eviction stays global, which is the point: splitting a fixed amount of memory between types up front is the guess a byte budget exists to avoid.

A coarse clock

Options.ClockGranularity. Over half of an uncontended hit was time.Now() — 27 ns of 47. A background goroutine can hold the time instead, and a hit costs 20 ns, with expiry accurate to within one interval either way. Off by default: a TTL that quietly means something else is the kind of surprise this library is about.

Comparison benchmarks, and what they found

benchmarks/ is a separate module so the root keeps its promise, comparing against otter v2, ristretto v2, golang-lru/v2/expirable and ttlcache v3. make bench-compare runs it.

Measuring serially made a defect visible. Per operation this cache was already cheaper than otter — 61 ns against 77 — but got five times less out of eight cores, and sharding did nothing about it: reads sat at a flat ~51 ns whether they had 16 shards or 256. The counters were one shared cache line that every core wrote to on every lookup. The lock was never what they were queuing for.

Counters now live on the shards, on their own cache lines, and the padding mattered as much as the split.

Get, 16 shards	v0.1	v0.2
LRU	64 ns	40 ns
ClearOnFull	55 ns	24 ns

Stats() is unchanged: it already walked every shard.

The README now carries the comparison tables, including the ones this library loses. otter is worth 5.6 points of hit rate on a working set that does not fit, and that is a better reason to pick it than any throughput number on the page.

Changed

Stats gained TypeMisses, Loads, LoadErrors and Coalesced, which moves Entries and Bytes within the struct. Code that reads fields by name is unaffected; code that builds a Stats positionally is not.

Stability

The major version is 0: the API may change in any release. Breaking changes are listed in CHANGELOG.md.

Next

A loader on views, so a typed view over a shared budget gets the same stampede protection the cache has.

If a different cache fits your problem better, the README says which one and why.

v0.1.0

The first release of a small in-memory cache for Go that aims to be predictable before it is fast.

There is no shortage of Go caches, and several of them are excellent. This one exists because the failures that actually cost time in production were never about throughput: a write that reports success and is dropped a moment later, a hit rate that quietly collapses because every value is too big for the budget, a limit expressed in entries when the thing you are protecting is a memory limit, and a batch of keys that all expire in the same millisecond and stampede the upstream together.

import "github.com/andared/sanecache"

Go 1.24 or newer. No dependencies, and that is a hard rule rather than a current state of affairs.

What's in it

• Cache[K, V] with TTL, optional jitter, and both lazy and background expiry.
• Byte budgets via MaxBytes and a Cost function, alongside MaxEntries. A byte budget without a cost function is a construction-time panic rather than a budget that silently counts entries.
• LRU and ClearOnFull eviction policies. The second trades precise eviction for a read path that needs only a read lock.
• First-class negative caching: SetNegative, SetNegativeTTL, and StatusNegative from Lookup, so "the upstream says this does not exist" does not have to be smuggled inside your value type.
• Optional sharding via Shards, off by default — the README has the numbers on when it earns its place, and for most services it does not.
• Built-in counters via Stats, and an OnEvict callback carrying a reason.
• ErrTooLarge from Set, for values that can never fit and would otherwise fail silently forever.

Stability

The major version is 0: the API may change in any release. Breaking changes are listed in CHANGELOG.md.

Next

A loader with single-flight so a cold key is fetched once rather than once per concurrent caller; typed views over a shared byte budget; and a coarse clock, since half the cost of a lookup today is reading the wall clock to check expiry.

If a different cache fits your problem better, the README says which one and why.