Cache Format

Cache format

Everything on this page was verified against a real Firestorm cache (D:\FS Cache\texturecache) and cross-checked against the viewer sources (indra/newview/lltexturecache.{h,cpp}). The complete evidence - marker dumps, PSOT arithmetic, per record samples - is in docs/format-notes.md; the constants live in src/utils/Constants.h.

Files

File	Layout	Reference cache
texture.entries	44 byte header + N x 28 byte records	5 507 644 B = 44 + 196 700 x 28
texture.cache	N x 600 B: the first 600 bytes of every codestream, same order as the records	118 020 000 B
<0..f>/<uuid>.texture	the rest of the codestream of one texture	54 825 files, ~7.3 GiB
FastCache.cache	N x 1040 B: (w, h, c, discardLevel) as S32 + a <=16x16 RGBA preview	204 568 000 B

The shard directory is the first hex digit of the UUID (87d1503c-... -> 8/). texture.entries and texture.cache are preallocated to the maximum number of records, so both contain capacity slots, most of them free.

texture.entries (little endian)

Header, 44 bytes:

F32  mVersion            1.71                  (text "1.71" in the reference cache)
U32  mAddressSize        64
char mEncoderVersion[32] "KDU v8.4.1"          (NUL padded)
S32  mEntries            196700                (capacity, not the live count)

Record, 28 bytes:

LLUUID mID         16 bytes, canonical byte order - matches <uuid>.texture names 1:1
S32    mImageSize  "total size of image if known", -1 for a brand new record
S32    mBodySize   size of the <uuid>.texture body; <= 0 means "free slot"
U32    mTime       seconds since 1970-01-01

Verified invariants:

• the number of records with mBodySize > 0 equals the number of .texture files on disk exactly (54 825);
• a free slot can still have its old body file on disk;
• mTime is not monotonic (62 189 of 196 700 records break the order), because slots are reused - never assume an ordering.

Assembling a codestream

codestream = texture.cache[index * 600 .. +600] ++ <uuid>.texture

This mirrors LLTextureCacheWorker::doWrite(): the 600 byte block holds mWriteData[0..600], the body file holds the rest and mBodySize = mDataSize - 600.

Two populations of records

Splitting the 54 825 live records by mImageSize - mBodySize yields exactly two groups:

Group	Count	mImageSize - mBodySize	COM marker in the header block	Decodes
A - complete	22 790	600	Kakadu-v4.2.1, Kdu-Layer-Info: ...	yes
B - truncated	32 033	601	\x00\x01a=<uuid>&c=<rgba>&z=<timestamp>	only best effort

Group A is a complete JPEG 2000 codestream: PSOT matches 600 + mBodySize - sotOffset - 2, the stream ends with EOC (FF D9), and bpp = 8 * streamSize / (w * h * components) is realistic (1.89 for a 64x64x4 texture).

Group B is a codestream that was cached only in part. The record and the body file are self consistent (mImageSize == mBodySize + 601, file size == mBodySize), the PSOT is consistent with the announced length as well; what is missing is simply not on disk. Patching PSOT down to the cached length exposes the real error: segment too long (78) with max (35) for codeblock ... - the last packet is cut in the middle. With OpenJPEG in lenient mode (opj_decoder_set_strict_mode(OPJ_FALSE)) the prefix still decodes; the missing bytes show up as missing detail, so the picture is blurry instead of broken.

The tool therefore classifies every record (TextureEntry::isComplete() / isPartial(), AssembledTexture::complete) and writes <uuid>.partial.png for group B - best effort output that is not presented as the original texture.

Component samples to PNG pixels

OpenJPEG hands out one plane per component; the tool always writes 8 bit RGBA:

Components	Mapping
1	luminance -> R = G = B, opaque
2	luminance + alpha
3	RGB, opaque
4	RGBA
5	RGBA from components 0..3, the fifth plane dropped + warning

A 1-in-100 sample of the reference cache (527 records, read from each header block's SIZ marker): 264 with 3 components, 226 with 4, 37 with 5 - so roughly one texture in seven carries a surplus plane.

That fifth plane is not alpha: in record 123386 (2048x2048x5) the planes have means 123.7 / 87.4 / 75.0 (photographic), 253.3 (99.2 % at 255) and 0.0, and the viewer's own FastCache.cache preview of the same record has alpha 253.3 as well, i.e. component 3. Components 3 and 4 of records 123378 and 123412 are 255 and both previews show alpha 255. So the surplus plane is constant per texture and its meaning is unknown - hence the warning and the drop.

Subsampled components (dx/dy != 1) are rejected rather than resampled: the viewer never writes them, and silently handling them would hide a format change. 16 bit samples are normalised to 8 bit.

Open questions

• What produced group B? Its COM payload is sim/bake metadata (a=<uuid>&c=<rgba>&z=<timestamp>) and those a= UUIDs are not in texture.entries at all - the viewer simply cached fewer bytes than its own record announces.
• What is the fifth component? Constant per texture (0 or 255 in the records inspected), not alpha, unexplained by the cache.
• FastCache.cache is not needed for decoding; it is documented only because it shares the record index space.