perf: WHP warm-start optimizations#87
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR reduces Windows WHP warm-start (snapshot restore) latency by adjusting the host-side restore path to preserve read-only initrd file mappings across restores, and by switching the embedded Hyperlight host dependency to a fork/branch that contains WHP warm-start optimizations (including whp-no-surrogate).
Changes:
- Update
Sandbox::restore()to userestore_preserving_file_mappings()when an initrd file mapping exists, avoiding unmap/remap on restore. - Remove stored initrd mapping base address from
Sandbox/finish_evolve()since restores no longer re-register the file mapping. - Switch
hyperlight-hostdependency todanbugs/hyperlightperf/whp-warm-startand enablewhp-no-surrogate(with corresponding lockfile updates).
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| host/src/lib.rs | Uses preserving restore variant when initrd mapping exists; removes now-unneeded initrd remap bookkeeping. |
| host/Cargo.toml | Points hyperlight-host at the WHP warm-start optimization branch and enables whp-no-surrogate. |
| host/Cargo.lock | Updates locked git revision and transitive deps from the new hyperlight-host source. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
30
to
31
| # hyperlight-dev/hyperlight#1373 — Ludvig's Snapshot::to_file/from_file | ||
| # Uses PAGE_READONLY mapping (no host commit charge per VM). |
| # hyperlight-dev/hyperlight#1373 — Ludvig's Snapshot::to_file/from_file | ||
| # Uses PAGE_READONLY mapping (no host commit charge per VM). | ||
| hyperlight-host = { git = "https://github.com/hyperlight-dev/hyperlight", branch = "disk_snapshot_copy", features = ["executable_heap", "hw-interrupts"] } | ||
| hyperlight-host = { git = "https://github.com/danbugs/hyperlight", branch = "perf/whp-warm-start", features = ["executable_heap", "hw-interrupts", "whp-no-surrogate"] } |
Call restore_preserving_file_mappings when a file mapping (initrd) exists, skipping the unmap/remap cycle on every snapshot restore. Remove the now-unused file_mapping_base field. Signed-off-by: danbugs <danilochiarlone@gmail.com>
Pin to rev 3ed18580 for reproducible builds. Enables whp-no-surrogate for VirtualAlloc allocation, direct WHvMapGpaRange, and demand-zero scratch reset. Signed-off-by: danbugs <danilochiarlone@gmail.com>
danbugs
force-pushed
the
perf/whp-warm-start
branch
from
62e3785 to
3537f04
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Linux Benchmarks
Details
| Benchmark suite | Current: 1e95cc0 | Previous: b15806c | Ratio |
|---|---|---|---|
hello_world (median) |
30 ms |
20 ms |
1.50 |
pandas (median) |
110 ms |
110 ms |
1 |
density (per VM) |
11 MB |
11 MB |
1 |
snapshot (disk) |
656 MiB |
656 MiB |
1 |
This comment was automatically generated by workflow using github-action-benchmark.
Contributor
There was a problem hiding this comment.
Windows Benchmarks
Details
| Benchmark suite | Current: 1e95cc0 | Previous: b15806c | Ratio |
|---|---|---|---|
hello_world (median) |
218 ms |
347 ms |
0.63 |
pandas (median) |
213 ms |
1044 ms |
0.20 |
snapshot (disk) |
279 MiB |
663 MiB |
0.42 |
This comment was automatically generated by workflow using github-action-benchmark.
The MEM_DECOMMIT+MEM_COMMIT optimization in zero() replaced physical pages without notifying WHP, causing FC-aware dispatch (pyhl, pydriver-run) to read stale input data on Windows. Signed-off-by: danbugs <danilochiarlone@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimizes WHP snapshot-restore latency on Windows (~8x cold load, ~6x steady-state restore).
Hyperlight changes (danbugs/hyperlight@perf/whp-warm-start):
WHvSetVirtualProcessorRegisterscall for GPR + debug + special registers instead of three separate callsrestore_preserving_file_mappings— skips unmapping/remapping read-only file regions (e.g., initrd) during restore; safe because they arePAGE_READONLYat the hypervisor level and guest writes go through CoW via scratchwhp-no-surrogatefeature — for single-partition-per-process scenarios:VirtualAllocinstead ofCreateFileMappingA,WHvMapGpaRangeinstead ofWHvMapGpaRange2; scratch zeroing usesfill(0)in-place (MEM_DECOMMIT+MEM_COMMIT was removed as it desyncs the WHP GPA mapping)This repo:
restore_preserving_file_mappingswhen an initrd mapping existswhp-no-surrogateTest plan