perf(build): cache resolved Diff per BuildId within File.ReadAt

[levb]

A single ReadAt call can iterate the mapping loop many times after post-dedup memfile fragmentation (4 KiB-granular mappings across a 2 MiB hugepage fault). Each iteration hit DiffStore.Get → ttlcache.Get, taking the shared ttlcache mutex per mapping. ttlcache executes a full Write lock there, see https://github.com/jellydator/ttlcache/blob/db85e4f64251c73b33ba055e3fe07d70870992ce/cache.go#L259

Stack-allocated fixed-cap slices (16 entries) + linear scan cover the typical unique build IDs in a chain. Allocation-free regardless of read size; eviction race handled by dropping the stale entry on ErrCacheClosed and re-resolving.

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
2676	2	2674	7

View the full list of 2 ❄️ flaky test(s)

github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 58.38% (Passed 658 times, Failed 923 times)

Stack Traces | 62.4s run time

=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:27: Build completed successfully
--- FAIL: TestSandboxMemoryIntegrity (62.38s)

github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 58.51% (Passed 648 times, Failed 914 times)

Stack Traces | 99.3s run time

=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{start:{pid:1254}}
Executing command bash in sandbox iw2i8bea7uoucl1p59baq (user: root)
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\nUsed memory before tmpfs mount: 190 MB\nFree memory before tmpfs mount: 794 MB\nMemory to use in integrity test (80% of free, min 64MB): 635 MB\n"}}
Executing command bash in sandbox iw2i8bea7uoucl1p59baq (user: root)
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"635+0 records in\n635+0 records out\n665845760 bytes (666 MB, 635 MiB) copied, 3.12423 s, 213 MB/s\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"\tCommand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=635\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 3.10\n\tPercent of CPU this job got: 99%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:03.13\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2628\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 2\n\tMinor (reclaiming a frame) page faults: 344\n\tVoluntary context switches: 3\n\tInvoluntary context switches: 149\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 829 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:70: Command [bash] completed successfully in sandbox iuibfvtk3k20ylq65fq2e
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1270}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{data:{stdout:"c93d6134239530970cc0476bac7f2a3aed29b8edfc0a20f9f2d75c1eaa688964\n"}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:80: Command [bash] completed successfully in sandbox iuibfvtk3k20ylq65fq2e
Executing command bash in sandbox iq8657khpt2c0oopkddya (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1273}}
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
Executing command bash in sandbox iuibfvtk3k20ylq65fq2e (user: root)
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:81
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox iuibfvtk3k20ylq65fq2e: unavailable: HTTP status 502 Bad Gateway
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:78
        	            				.../tests/orchestrator/sandbox_memory_integrity_test.go:110
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (99.26s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[gemini-code-assist]

Code Review

If a newly resolved build is closed or evicted immediately after being retrieved from the store, it is appended to the cache with hitIdx remaining -1. When ReadAt subsequently fails with a CacheClosedError, the stale entry is not removed from the cache because hitIdx is negative, causing a redundant loop iteration and a second failed read attempt on the next loop cycle. Setting hitIdx to the index of the newly appended entry ensures that any immediate failure correctly evicts the stale build from the cache without redundant retries.

[bchalios]

LGTM. Have you seen this having a measurable performance impact?

[levb]

LGTM. Have you seen this having a measurable performance impact?

@bchalios yes, on my cluster while running integration tests and watching grafana. Resume times improved. A big *** is that all this is very noisy. pprof also showed considerable time spent in that ttlcache function.