v0.21.0

Added

• AppSandbox::snapshot_with_scratch_mb takes a snapshot with another scratch size, which the sandbox continues on and a restore of the snapshot gets: boot with what unpacking the rootfs needs, then run guest function calls with what they need, less or more. The snapshot records the size (snapshot_scratch_mb), as do later snapshots of the sandbox. A size too small for the kernel makes the sandbox's next entry, the restore, and a boot from the snapshot fail (restoring a snapshot that fits recovers the sandbox). A guest function call that runs out of scratch ends with its process killed (status 137 with a single-process driver, such as python's) or the guest aborted, depending on what runs out first. A restore recovers the sandbox.
• AppSandbox::dirty_bytes returns how many bytes the guest has written since the sandbox was last restored (a boot from a snapshot included) or snapshotted, not counting the kernel's resume entry; after a snapshot, the kernel's own re-initialisation on the next entry is counted (a small call then counts about 1.3 MiB, against about 0.6 MiB after a restore). The guest controls the value, and it can be ahead of what the guest wrote by up to about 1 MiB, the kernel's frame allocator taking its memory a chunk at a time, and by more around a large allocation, whose chunk is several times its size.
• SandboxBuilder::on_interrupt_handle hands a function the sandbox's interrupt handle before boot returns: on a boot from a snapshot, before the resume entry, so a guest that spins there can be killed, and boot then fails with Error::Hyperlight(ExecutionCanceledByHost) (a kill racing the end of the entry can lose, and boot succeeds). A kill made before the resume entry's guest call starts is dropped, and one made after it starts but while the vCPU is not running returns false yet cancels the call unless the entry is ending, so keep killing until kill returns true or boot returns, then stop the killer, and wait for it, before the sandbox's next guest call (see the method's documentation). A fresh guest boots before there is a handle, so there the function gets it after the boot. If boot fails before the sandbox exists, the function is never called, only dropped.

Changed

• Snapshots saved by v0.20.1 and earlier releases are refused (the kernel, the host's memory layout and the hyperlight-host dependency changed); save them again.
• The host places the transport arena at a fixed address below the top of scratch, so a snapshot keeps it when restored with a different scratch size.
• hyperlight-host and hyperlight-common move to 0.19. Hyperlight 0.19 gives the guest no heap region (hyperlight-dev/hyperlight#1968): the memory the guest allocates from is scratch. The kernel took its 1 MiB boot stack from that region, and now keeps it in its own image. The re-exported hyperlight_host is 0.19: its SandboxConfiguration::set_heap_size, SandboxConfiguration::DEFAULT_HEAP_SIZE, SandboxBuilder::heap_size and SandboxBuilder::get_heap_size (hyperlight's builder, not hluk's) are gone (size scratch instead), and HostFunctions, Snapshot (and what takes or returns one), Error::Hyperlight, and the interrupt handle AppSandbox::interrupt_handle returns and SandboxBuilder::on_interrupt_handle hands over are 0.19's types. For Hyperlight 0.19's other breaking changes (among them PtRootFinder gets a SnapshotMemoryReader and returns Result<Vec<u64>>, Sandbox::snapshot removes map_region and map_file_cow regions, snapshot ABI 9), see its CHANGELOG. A native Unikraft kernel (--kernel, SandboxBuilder::from_kernel) built against an earlier plat/hyperlight no longer boots: rebuild it against this kernel tree. plat/hyperlight now selects paging and refuses a build without a boot allocator, and the guest crashes if the canary word at the bottom of its 1 MiB boot stack (the init thread's, which also runs a native main() without LIBUKBOOT_MAINTHREAD) is found overwritten at the end of boot, at the start of an entry from the host, at the end of a call's entry, or at a shutdown.
• A NetworkFilter's allows is asked at each send on a connected UDP socket rather than at its connect(), as it already was at an unconnected sendto(); a TCP connect is asked about as before. A BlockList looks its names up at those sends too, rather than at the UDP connect().
• CI: on the Linux arm64 runner, a failure of the http-express demo or of the smolagents or mcp integration jobs no longer fails the run: under nested virtualization their guest's vCPU wedges (hyperlight-dev/hyperlight#2025), so the nightly is not red for that known cause. They still run and log; macOS, Windows and every other job stay strict.
• CI: a wedged HTTP demo fails once its five-minute wait for an answer runs out (60 tries, each a 3 s request and a 2 s pause), instead of holding its job until the timeout (the Linux arm64 nightly's demos job sat at its 90-minute ceiling): the demo harness captures the server's output to a file instead of a pipe it never drained, and stops the server with a bounded terminate, then kill.

Fixed

• Under a restricted network policy (AllowList, BlockList or Filter), a name lookup with AI_ADDRCONFIG, as Node makes for a connection to a hostname, failed with getaddrinfo EACCES (#247): musl probes the address families it can use with a UDP connect(), to loopback among others, which the policy refused. A UDP connect() sends nothing, so it is no longer checked, and keeps the host's own errors; each send on a connected UDP socket is checked instead, against its peer, after a reconnect or an AF_UNSPEC disconnect too, as an unconnected sendto() already was. TCP is checked at its connect, as before.
• The kernel allocated scratch pages the host reserves at the top of scratch. It now stops below them.
• A guest that ran out of scratch during a copy-on-write fault crashed the VM (an unexpected exit) instead of aborting.
• On x86, a page fault the kernel could not handle early in boot or in its fault handler ended the guest as if it had finished: a guest function call that hit one looked complete. The guest aborts.
• hluk build, and hluk run --build, refused a [build] command that would have run: a program named by a relative path (./build.sh) was looked for in the directory hluk ran in rather than the project's, where the command runs, so hluk build -f path/to/project from elsewhere said it was not on PATH; and a program the shell names by expanding a word (${CC:-gcc}, "$GO", ~/bin/tool, %CC%) was looked up as written. Such a program is now looked for in the project's directory, or left to the shell.
• In the agent and python-shell images, and images built FROM them, Python's ssl.create_default_context() loaded no CA, so HTTPS from Python (urllib.request, http.client) failed with CERTIFICATE_VERIFY_FAILED: the images carry Debian's bundle at /etc/ssl/certs/ca-certificates.crt, but the OpenSSL that Python links looks for /usr/lib/ssl/cert.pem, and the hashed links in /etc/ssl/certs point into /usr/share/ca-certificates, which the images leave out. The bundle is now at /usr/lib/ssl/cert.pem too. pip was not affected, as it brings its own bundle.
• In the powershell image, cmdlets were not found by name (The term 'Write-Output' is not recognized) and PowerShell resolved no path (/opt became /:/pt), so Get-Module -ListAvailable found nothing (#138): .NET lists the drives from /proc/mounts, which the guest did not have. The image carries one now.
• On Windows, hluk build, and hluk run --build, did not look for a [build] command's program where cmd looks: a name in the project's directory (build.bat, or build), which cmd runs before looking in PATH, and a path to which cmd adds an executable extension from PATHEXT (scripts\build for scripts\build.cmd) were refused. Both are found now.
• hluk build, and hluk run --build, looked a [build] command's program up in PATH as hluk would, not as the command's shell does: a relative entry (node_modules/.bin, ., or an empty one) was taken from the directory hluk ran in rather than the project's, where the command runs, so command = "tsc" with node_modules/.bin on PATH was refused from outside the project; and a command that sets PATH for its program (PATH=$PWD/tools:$PATH mytool) had the program looked up in hluk's PATH. A relative entry is now taken from the project's directory, and a command that sets PATH for a program named without a path is left to the shell.
• hluk build, and hluk run --build, refused a [build] command that starts with a shell builtin it did not know as one: until, unset, ulimit, umask, hash, getopts and other sh builtins on Unix, and cmd's builtins (call, copy, del, type, …, in any case) on Windows, were looked up in PATH. They are left to the shell. A program named by a path that is not a file is no longer said to be missing from PATH, which a path is not looked up in: the message names the project's directory, where the command runs.

Pull requests

• Restore snapshots with another scratch size by @danbugs in #162
• hluk build: find a build command's program where the command runs by @NopeCharon in #227
• Give the nested-virt arm64 runner room on the timeouts it still blows by @danbugs in #244
• build(deps): bump ureq from 3.4.2 to 3.5.0 in the cargo group across 1 directory by @dependabot[bot] in #238
• build(deps): bump the images group across 3 directories with 2 updates by @dependabot[bot] in #239
• build(deps): bump the actions group across 2 directories with 3 updates by @dependabot[bot] in #240
• agent, python-shell: put the CA bundle where OpenSSL looks for it by @NopeCharon in #245
• powershell: give the guest a /proc/mounts, so cmdlets are found by name by @danbugs in #246
• Fix restricted DNS address-family probing by @devigned in #248
• hluk build: look for a build command's program where cmd looks, on Windows by @NopeCharon in #237
• hluk build: look a build command's program up in the PATH its shell searches by @NopeCharon in #236
• hluk build: leave every shell builtin to the shell, and say where a path was looked for by @NopeCharon in #235
• Add AppSandbox::dirty_bytes by @danbugs in #259
• ci: stop a wedged HTTP demo from hanging the whole job by @danbugs in #260
• ci(arm64): soft-fail the known nested-virt failures on linux by @danbugs in #263
• SandboxBuilder::on_interrupt_handle: the interrupt handle before boot returns by @danbugs in #264
• Release v0.21.0 by @danbugs in #269

New Contributors

• @NopeCharon made their first contribution in #227
• @devigned made their first contribution in #248

Full Changelog: v0.20.1...v0.21.0