A request-counted concurrent cache for Swift.
Create a concurrency-safe key/value cache in Swift. It accepts an asynchronous, throwing refresh closure and a positive parameter N. Every successfully refreshed value has N request slots, including the request that triggered the refresh. Concurrent requests count individually.
With N == 3, seven sequential requests for one key observe refresh generations:
[1, 1, 1, 2, 2, 2, 3]
The design must preserve that accounting under concurrent admission, refresh failure, caller cancellation, actor reentrancy, and task-priority changes.
let cache = TallyCache<String, Payload>(requestsPerValue: 3) { key in
try await fetchPayload(for: key)
}
let value = try await cache.value(for: "account")Key must be Hashable and Sendable, Value must be Sendable, and the refresh closure is @Sendable. requestsPerValue is fixed for the lifetime of a cache and must be greater than zero.
- A successful refresh generation creates exactly
Ndelivery slots for its value. The request that caused the refresh occupies one of those slots. - A value is never successfully returned more than
Ntimes. After its final slot is consumed, the next request starts a new generation. - Concurrent callers are individual requests; coalescing their refresh work does not coalesce their slot accounting.
- Only a successful delivery consumes a slot. A caller removed by cancellation does not consume one, and a failed refresh creates no cached value.
- There is no time-to-live, size limit, persistence, or manual invalidation policy. An unused value remains cached only with its remaining request slots.
The actor owns independent state for every key:
- at most one cached value and its remaining slot count;
- at most one active refresh generation and its admitted waiters; and
- a FIFO queue of waiters reserved for later capacity or later generations.
A cached value is never stored alongside an active generation or queued demand. The cache-hit decision and miss enrollment also occur in one uninterrupted actor job: no request can be enqueued behind an available cached value, and the cancellation handler cannot reach the actor before its waiter has been registered. Preserving that no-suspension boundary is essential; splitting it would allow a cancellation to be lost and a continuation to remain stranded.
The request path is:
request
|-- cached slot available --> decrement the slot count and return the value
|-- active generation has capacity --> join that generation
|-- active generation is full or closing --> enter the FIFO queue
`-- no value and no generation --> admit up to N waiters and start a refresh
All admission, accounting, cancellation, and completion transitions are isolated by the actor. The potentially slow refresh closure runs outside the actor, so unrelated actor operations remain responsive.
Refreshes for the same key are strictly serialized: no second refresh closure for that key starts before the first has returned or thrown. Different keys have independent state and may refresh concurrently.
When a refresh succeeds, its value first serves the waiters admitted to that generation. If cancellation freed some of the generation's N slots, the oldest queued waiters are promoted into those slots before any unused capacity is cached. Remaining slots are then banked with the value for later callers.
If the generation used all N slots and more waiters remain queued, the next generation starts. While active, the refresh is retained only as Task<Void, Never>; the task handle is discarded at completion and can never retain the returned value as its result.
A thrown refresh error is not cached. The same failure is delivered only to the waiters admitted to that generation; callers already queued for later capacity are not failed by work they were not admitted to. After the failed generation ends, queued demand starts the next generation.
Failures therefore consume no request slots and do not poison the key. A later request may retry by starting another refresh.
Every suspended caller has an identity and a checked continuation owned by the actor. This makes cancellation and completion a single serialized decision and ensures each continuation is resumed exactly once.
- A caller already canceled on entry immediately throws
CancellationError. - Canceling a queued waiter removes it from the FIFO queue and resumes it with
CancellationError. - Canceling an admitted waiter removes it from the generation without consuming a slot. While the generation remains open, the oldest queued waiter is promoted into the freed slot.
- When the last admitted waiter cancels, the generation stops accepting replacements and its refresh task is canceled.
- Requests arriving after that full cancellation remain queued until the dying refresh returns or throws. This preserves the no-overlap guarantee for a key.
Task cancellation is cooperative. A refresh closure must observe cancellation and throw if it cannot produce a valid value. If it ignores cancellation but returns normally, that return is treated as a valid success: the value may serve queued requests and any remaining slots may be cached. Discarding such a value merely because its original waiters canceled would waste valid work and force a redundant refresh; a closure that produced an unusable or partial result is responsible for throwing instead.
Cancellation can race with delivery. The actor's ordering defines the outcome: if cancellation removes the waiter first, the caller gets CancellationError; if successful delivery wins first, the caller receives the value and occupies one slot.
Strict serialization has a deliberate cost: a refresh that ignores cancellation and never finishes blocks later refreshes for that key. That is backpressure from the refresh dependency, not something the cache can safely solve without adding a timeout, overlapping generations, or stale-result fencing.
A generation starts at the highest priority of the waiters initially admitted to it. A later higher-priority waiter escalates the active refresh, whether it joins that generation directly or queues behind it; a queued waiter depends transitively on the active refresh finishing before its own generation can begin.
On runtimes with Swift's task-priority escalation APIs, escalation of an already suspended caller is also forwarded to the refresh task. Waiter identity is checked before forwarding, so a delayed callback cannot raise the priority of an unrelated later generation. On Apple operating systems before version 26, dynamic escalation is unavailable and the refresh retains its spawn priority. Priority remains a scheduling hint rather than a guarantee of execution order.
Refresh work runs in a detached task. It therefore does not inherit the triggering caller's actor isolation or task-local values. Every input that can change the refreshed value—such as tenant, locale, credentials, or configuration—must be represented in Key or captured explicitly in a Sendable dependency. This prevents whichever caller happened to miss first from silently defining a shared value for other callers.
Priority is forwarded explicitly because it affects scheduling rather than value identity. Cancellation is forwarded through the refresh task and remains cooperative.
The refresh closure must not directly or indirectly call value(for:) for a key already present in the same refresh dependency path. For example, a refresh of A that waits for B while B waits for A forms a value cycle and cannot complete. The cache intentionally does not attempt dependency-cycle detection.
Generation identities ensure that a late completion can affect only the generation that created it. The detached refresh task captures the cache weakly, avoiding a cache-to-task-to-cache ownership cycle. Empty per-key state is removed after its cached slots, generation, and queue are all gone.
Waiter-array mutations temporarily remove a key's state from the dictionary, mutate it with unique storage, and reinsert every nonempty state. This avoids unnecessary copy-on-write cloning while preserving the actor's atomic state transitions.
- FIFO waiter arrays use linear
removeFirstand identity lookup. This keeps the package dependency-free and is appropriate for ordinary queue depths. A deque and keyed waiter index should be considered only if measurements show long per-key queues make these operations material. - There is intentionally no invalidation API. Adding one requires a product decision about cached slots, active refreshes, queued callers, and whether a late result may repopulate an invalidated key; those semantics should not be guessed.
- Same-key refresh serialization is stronger than maximum throughput after cancellation. It avoids overlapping refreshes and ambiguous competing results at the cost of waiting for an uncooperative refresh to unwind.
- This is a request-count policy, not a time-, memory-, or consistency-based cache. Time-to-live, least-recently-used eviction, stale-while-revalidate, persistence, and cross-key deduplication are separate policies.
The package requires a Swift 6.2 or newer toolchain. Its manifest explicitly declares these Apple deployment floors:
- macOS 15
- iOS 15
- tvOS 15
- watchOS 9
SwiftPM's platforms array configures versioned deployment targets; it is not an allowlist. Linux and Windows have no deployment version to declare there and are not excluded by the Apple entries. Platform values such as .linux and .windows are used only when a dependency or build setting needs a platform condition.
The library target imports no Apple frameworks and relies only on Swift concurrency and standard-library types, so the implementation is intended to build on Linux and Windows with Swift 6.2 or newer. The tests import only XCTest. Linux and Windows builds have not yet been run, however, so they remain expected rather than verified support until both hosts are added to continuous integration.
The source has been type-checked with the installed Swift 6.4 toolchain for macOS 10.15, iOS 13, tvOS 13, watchOS 6, and visionOS 1. These checks establish source compatibility below the package's declared Apple deployment floors; the full suite has been run only on macOS.
Task priority is always a scheduling hint. A generation requests the highest relevant spawn priority on every platform, but dynamic escalation depends on runtime support. It is enabled on Apple operating systems version 26 or later; earlier Apple systems retain the spawn priority, and current non-Apple runtimes may provide weaker or no dynamic escalation. This does not affect request accounting, serialization, cancellation, or error behavior.
swift build --build-system native
swift test --build-system nativeThe native build-system option avoids the automatic test-bundle signing performed by the newer Swift build backend on macOS.