Proposal: Template memory hotset profiling & prewarm for faster starts

[youhangwang]

Templates can ship a small "startup hot set" profile; at restore time the VMM pre-reads exactly those regions into the page cache. First-touch faults hit memory instead of disk, cutting cold-start latency. Default off (CUBE_TEMPLATE_PROFILE_ENABLE), pure optimization, degrades to current behavior on any failure.

Problem

fast-restore MAP_PRIVATE-maps the template memory file into the VMM, so every guest first-touch fault is a real disk read. On a cold node (reboot, memory reclaim) NVMe I/O dominates startup latency. One structural fact makes this very fixable: all sandboxes from a template map the same shared inode, so warming its hot set once benefits every sandbox on the node.

Proposal

• Produce (build time, Cubelet): after the snapshot lands, boot two throwaway sandboxes from the new template through the normal restore path, collect touched pages from /proc//{maps,pagemap}, intersect the two runs to drop jitter, and write the merged extents as hot-pages.json next to the snapshot memory file.
• Consume (restore time, VMM): if a valid profile exists, issue posix_fadvise(WILLNEED) per extent while the guest is starting up — async, tens of ms, overlapped with device restore.

Safety

No valid profile → restore path is byte-for-byte unchanged. Stale profiles are deleted unconditionally on rebuild. Profiling and prewarm failures only log; they never block a template release or sandbox create.