Cache and Pinning

File cache, mappings, pins, and write-back

This page describes SpiritBoot's reconstructed kernel at main 27149596. It documents implementation behavior; it does not assert complete equivalence to a commercial Xbox kernel. Public exports are indexed in Export-Index and individually described in Runtime-Calls. Cc* and Nxc* are primarily internal interfaces reached through filesystem operations.

Source and integration

The NXK_MM_CACHE build selects ntoskrnl/xb/mm/cache.c instead of the original section/VACB cache. The initial implementation is in the clean-room kernel patch. Later behavior is defined by the ordered source manifest, particularly patches 0008, 0020, 0027, and 0028. Reading only Roswell's original cache implementation or an old patch is insufficient.

The introductory source comment still mentions 64 KiB buffers. The actual constants select 16 KiB blocks, four visible blocks initially, at least four visible blocks, and at most 32 visible blocks. Two additional slots form a write-back reserve and are excluded from FscGetCacheSize(). Title-visible default capacity is therefore 64 KiB; maximum visible capacity is 512 KiB. These are cache limits, not disk cluster sizes or a promise that every title gets the size it requests.

Objects and ownership

Object	Identity and purpose	Lifetime
NXC_MAP	Shared map for a section-object pointer; records allocation size, EOF, valid-data length, backing file object, callbacks, private-map list	Shared by file objects; the implementation drops it when its last private cache map closes
NXC_BLOCK	(map, aligned file offset) plus 16 KiB data, validity, dirty/busy flags, references and LRU links	Fixed descriptor slot; buffer may become inactive or be reassigned only under the cache's rules
NXC_BCB	Public BCB prefix, range, block/map pointer and reference/pin counts	Each successful mapping/pin acquires a handle; final unpin releases its block reference
Idle BCB list	At most eight fully released descriptors	Retains no counted block/map reference; stale pointer fields are overwritten before reuse; overflow is freed outside the mutex
Paging MDL	Physical frames for an in-flight block transfer	Built after relocation is disabled for the buffer; released after completion, then movability is restored

A cached hit does not mean a returned buffer can survive an unpin. A BCB must remain referenced while the caller uses the mapped bytes. CcRepinBcb() adds a reference; the matching release must eventually occur. The idle-list optimization changes allocation churn, not the lifetime contract of a live handle. On final CcUnpinData(), NxcPutBlock() runs before idle-list insertion; stale pointer values in the retained descriptor are not owning references. Reuse zeroes the entire descriptor before establishing its new block/map and counts.

Acquisition path

CcMapData() calls NxcPinCommon() with map semantics. CcPinRead() first requires the shared map's PinAccess, then calls the same helper with pin semantics. MAP_WAIT and PIN_WAIT select whether a miss or busy block may wait. The other flags must not be assumed to implement the complete Windows cache-manager contract merely because their names are familiar.

The helper derives the aligned block offset and byte offset inside that block. A negative file offset or a range longer than the remaining block raises STATUS_INVALID_PARAMETER; it does not truncate the request or stitch together multiple blocks. Subtraction-based checking avoids unsigned addition overflow. A caller using a 32/64 KiB disk cluster must split pins at the 16 KiB cache boundary.

Under the cache mutex, a BCB is reused or allocated. Allocation failure sets the BCB output to null and raises STATUS_INSUFFICIENT_RESOURCES. NxcGetBlock() finds or claims a block. A no-wait miss/busy condition returns false with null BCB/buffer outputs. A read failure is raised after releasing the mutex; the provisional BCB is freed during exception unwind. Success initializes the entire BCB and returns a pointer into the referenced block.

CcPinMappedData() uses a temporary new BCB. On success it replaces and releases the old mapping. On failure it preserves the existing mapping. This matters for non-waiting misses: overwriting the caller's handle before success would leak the old reference.

CcPreparePinWrite() still obtains data through the pin-read path; Zero == TRUE zeroes the requested span after acquisition. It is not a general no-read full-overwrite fast path.

Hit, miss, and eviction behavior

NxcLookupBlock() scans the fixed descriptors for matching map and offset. On a valid hit the implementation references the block and moves its LRU link to the tail, including when it was already most recent. A busy block returns null in no-wait mode; wait mode drops the mutex for a 1 ms delay, reacquires it, and retries. A failed earlier read can leave an invalid block that a later waiting acquisition retries.

The claim path uses clean-first eviction and excludes referenced/busy blocks. Dirty eviction requires write-back. The mutex is dropped for I/O, so a rescan is needed afterwards; another request may have installed the same map/offset while it was unlocked. Reserve slots address recursive filesystem metadata reads during write-back. The current eligibility counter NxcWbDepth is global; this page does not claim that it proves per-thread recursive ownership under unrelated concurrent requests.

Paging I/O and completion

NxcPagingIo() allocates an MDL, marks the buffer's physical pages nonmovable, builds the nonpaged-pool MDL, initializes a notification event and IOSB, and submits IoPageRead() or IoSynchronousPageWrite().

• STATUS_PENDING requires waiting on the completion event and then reading the final IOSB status.
• A synchronous return uses that return status; the transfer count comes from IOSB information.
• An MDL allocation failure is an explicit resource failure.
• System mappings, MDL storage and the temporary nonmovable state are cleaned up after completion.
• The event reports operation completion, not title-level acceptance of the resulting file.

NxcReadBlock() chooses up to one cache block, rounding a final partial span to page size. It normalizes STATUS_END_OF_FILE to success and zeroes bytes after the reported transfer count before marking the block valid. A tail read can therefore legitimately have fewer bytes than a full block. That policy must not be used to hide arbitrary device failures.

Dirty data and flushing

CcSetDirtyPinnedData() marks the block dirty. Unpinning releases ownership; it does not by itself prove disk durability. CcUnpinRepinnedBcb(WriteThrough=TRUE) flushes the BCB's mapped range, records status/information, then unpins.

NxcFlushBlock() snapshots the map's EOF, writes a page-rounded span when its block lies before EOF, releases the cache mutex during paging write, and updates state after reacquisition. On an NT-success status it clears Dirty, increments counters, and advances valid-data length up to EOF. It returns the observed byte count.

Known source-level concerns, not qualified fixes: the success path does not independently require a full-length transfer before clearing dirty state; NxcFlushMap() restarts after a failed dirty write while the block remains dirty, so a permanent error can repeatedly retry. These observations require focused failure/short-completion qualification before claiming bounded flush behavior. Historical eviction/shrink fixes do not automatically prove every explicit-flush/close path.

CcUninitializeCacheMap() removes a private map. Last close flushes, discards unreferenced blocks, removes the shared map, frees it, and dereferences its backing file object. The discard path skips referenced blocks. A BCB surviving final map destruction therefore needs an external lifetime guarantee or a separately demonstrated deferred-teardown contract; neither is inferred from the call's TRUE return. The optional completion event is signaled at the end of this implementation.

There is no lazy-writer thread. CcCanIWrite() is unconditional true; conventional lazy-write/read-ahead/MDL entry points include reduced or unsupported implementations. Use the exact function entry and source before relying on a desktop Windows expectation.

Validation and performance interpretation

Maintained fixtures cover pin exception unwind, range rejection, bounded BCB reuse, dirty eviction failure, and cache shrinking. See the corresponding ordered patches and firmware workflow. Native file operations additionally need reopen/byte verification and error-path tests; a game reaching its menu is not a durable-write oracle.

BCB reuse removes allocator calls on repeated map/unpin pairs. It does not establish an FPS gain or remove storage I/O on a cache miss. Cache-hit overhead, polling latency, recursive reserve use, repeated FAT mapping, and write-back failure are separate mechanisms and should remain separately attributable changes.

Related: FATX-and-Save-Contracts, Clean-Room-and-Validation.