|
| 1 | +## Phase 3 Day 5 — `VZVirtualMachineDelegate`-driven exit codes + vsock host→guest dial primitives |
| 2 | + |
| 3 | +> Outcome log. Status: complete. The supervisor's |
| 4 | +> `wait_for_exit` on Mac now reports the *real* exit reason |
| 5 | +> (clean shutdown vs Apple-reported crash vs host-initiated |
| 6 | +> stop) instead of the Day-3 placeholder `0`-for-everything, |
| 7 | +> and the FFI primitive for `VZVirtioSocketDevice.connectToPort:` |
| 8 | +> is wired through `RunningVm::connect_vsock` / |
| 9 | +> `VzProvider::vsock_connect`. End-to-end host→guest provider |
| 10 | +> bridging on Mac is deferred to Day 6 because that integration |
| 11 | +> needs a Mac-eligible capsule manifest to validate against. |
| 12 | +
|
| 13 | +### Goal (recap) |
| 14 | + |
| 15 | +Day 4 wired the Carrier console socketpair so capsule code |
| 16 | +inside the VM can talk to host providers via `/dev/hvc1`. The |
| 17 | +two remaining "is this really like Linux?" gaps were: |
| 18 | + |
| 19 | +1. **Exit-code distinction.** `RunningVm::wait_for_exit_code` |
| 20 | + polled `state != Running` every 100 ms and returned `0` for |
| 21 | + every terminal state. The supervisor couldn't tell clean |
| 22 | + shutdown from crash; `elastos status <handle>` showed |
| 23 | + `Stopped` for both. |
| 24 | +2. **Host→guest vsock.** Crosvm exposes `AF_VSOCK` directly on |
| 25 | + Linux. On Mac, Apple's only supported channel is |
| 26 | + `VZVirtioSocketDevice.connectToPort:completionHandler:` — |
| 27 | + we had the device attached (Day 2) but no FFI surface to |
| 28 | + actually dial it. |
| 29 | + |
| 30 | +Day 5 closes (1) end-to-end and lands the primitive that |
| 31 | +closes (2) — full provider-bridge integration follows in Day 6. |
| 32 | + |
| 33 | +### What landed |
| 34 | + |
| 35 | +1. **`elastos-vz/src/ffi/delegate.rs`** (new). Custom NSObject |
| 36 | + subclass `ElastosVzDelegate` declared via |
| 37 | + `objc2::define_class!` and conforming to |
| 38 | + `VZVirtualMachineDelegate`. Three delegate methods: |
| 39 | + |
| 40 | + | Apple selector | Reason | Exit code | |
| 41 | + |---|---|---| |
| 42 | + | `guestDidStopVirtualMachine:` | Guest cleanly stopped (`poweroff -h`, `init 0`) | 0 | |
| 43 | + | `virtualMachine:didStopWithError:` | Apple tore the VM down because of an internal error | 1 | |
| 44 | + | `virtualMachine:networkDevice:attachmentWasDisconnectedWithError:` | Logged only; non-terminal | n/a | |
| 45 | + |
| 46 | + The delegate holds a shared |
| 47 | + `Arc<Mutex<Option<oneshot::Sender<DelegateExit>>>>` (first- |
| 48 | + to-take-it-wins). Subsequent delegate fires are logged but |
| 49 | + do not poison the channel. |
| 50 | + |
| 51 | +2. **`VzMachineHandle::new`** creates the delegate + oneshot |
| 52 | + pair before the VM is constructed, then dispatches |
| 53 | + `setDelegate:` onto the VM's associated queue |
| 54 | + immediately after `initWithConfiguration_queue` returns. |
| 55 | + Apple uses a *weak* reference for delegates, so the |
| 56 | + `Retained<ElastosVzDelegate>` is held inside |
| 57 | + `VzMachineHandle` for the handle's lifetime via a |
| 58 | + `SendableDelegate` newtype (same pattern as `SendableVm`). |
| 59 | + |
| 60 | +3. **`VzMachineHandle::stop`** now also signals the shared |
| 61 | + exit channel with `DelegateExit::HostInitiatedStop` after a |
| 62 | + successful `stopWithCompletionHandler:` completion — Apple |
| 63 | + does NOT fire the delegate on host-initiated stops, so |
| 64 | + without this any `wait_for_exit` racing against a stop |
| 65 | + would hang. |
| 66 | + |
| 67 | +4. **`VzMachineHandle::wait_for_exit`** is a new public |
| 68 | + `pub(crate) async fn` that awaits the receiver and maps |
| 69 | + `DelegateExit → i32`. Consumes the receiver on first call; |
| 70 | + subsequent calls return a typed |
| 71 | + `"receiver already consumed"` error. The supervisor's |
| 72 | + single-waiter contract (one `wait_for_exit` per capsule |
| 73 | + handle) makes this safe. |
| 74 | + |
| 75 | +5. **`RunningVm::wait_for_exit_code`** replaced its |
| 76 | + 100 ms-polling `loop` with a single `handle.wait_for_exit() |
| 77 | + .await`. The exit code surfaces through `Result<i32>` exactly |
| 78 | + like before — no supervisor changes were necessary. |
| 79 | + |
| 80 | +6. **`elastos-vz/src/ffi/vsock.rs`** gained `connect_vsock`, |
| 81 | + the FFI primitive for `VZVirtioSocketDevice.connectToPort:`: |
| 82 | + |
| 83 | + ```text |
| 84 | + VZVirtualMachine.socketDevices[0] |
| 85 | + │ |
| 86 | + ▼ (downcast to VZVirtioSocketDevice) |
| 87 | + │ |
| 88 | + ▼ connectToPort:completionHandler: |
| 89 | + │ |
| 90 | + ▼ on completion: |
| 91 | + │ |
| 92 | + ▼ VZVirtioSocketConnection.fileDescriptor |
| 93 | + │ |
| 94 | + ▼ dup(fd) ──► OwnedFd to caller |
| 95 | + │ |
| 96 | + ▼ Apple's connection drops at block end |
| 97 | + (closes its own fd; our dup keeps the |
| 98 | + socket endpoint alive) |
| 99 | + ``` |
| 100 | + |
| 101 | + The `dup` pattern mirrors Day 4's carrier socketpair |
| 102 | + handling — Apple's docs are explicit that |
| 103 | + `VZVirtioSocketConnection.fileDescriptor` is owned by the |
| 104 | + connection object, so we duplicate before letting the |
| 105 | + connection drop. |
| 106 | + |
| 107 | +7. **`RunningVm::connect_vsock(port)`** public method delegates |
| 108 | + to `VzMachineHandle::connect_vsock`. Returns |
| 109 | + `std::os::fd::OwnedFd` so callers (Day 6+) can wrap in |
| 110 | + `tokio::io::unix::AsyncFd` or a `std::fs::File` for |
| 111 | + blocking I/O — same shape `vm_provider.rs::try_vsock_connect` |
| 112 | + already uses for its Linux AF_VSOCK fd. |
| 113 | + |
| 114 | +8. **`VzProvider::vsock_connect(handle, port)`** mirrors the |
| 115 | + other provider lifecycle methods. Returns `CapsuleNotFound` |
| 116 | + for unknown handles or handles that have been moved out via |
| 117 | + `take_running_vm` — the supervisor can dispatch on a single |
| 118 | + error variant. |
| 119 | + |
| 120 | +9. **Tests** (Mac-gated unless noted): |
| 121 | + |
| 122 | + | Crate | Test | Asserts | |
| 123 | + |---|---|---| |
| 124 | + | `elastos-vz` (lib) | `delegate_exit_maps_to_expected_codes` (new) | `GuestCleanStop`/`HostInitiatedStop → 0`; `StoppedWithError → 1`. | |
| 125 | + | `elastos-vz` (lib) | `delegate_signal_exit_sends_first_terminal_observation_only` (new, `#[tokio::test]`) | First call resolves the receiver; second is a no-op at the channel level; the shared sender is consumed. | |
| 126 | + | `elastos-vz` (lib) | `vsock_connect_result_passes_through_apple_errors_with_op_prefix` (new) | Diagnostic error string includes op label + nil-pointer reason. | |
| 127 | + | `elastos-vz` (integ) | `vz_provider_vsock_connect_fails_closed_for_unknown_handle` (new) | `vsock_connect` on an unknown handle returns `CapsuleNotFound`. | |
| 128 | + |
| 129 | +10. **Docs:** this file; `PLAN.md` Phase 3 header advances to |
| 130 | + "Day 5 complete"; `MAC.md` capability matrix updates the |
| 131 | + exit-code row from "polling, all 0" to "delegate-driven, |
| 132 | + distinguishes clean stop from crash". |
| 133 | + |
| 134 | +### Apple-API notes that shaped Day 5 |
| 135 | + |
| 136 | +- **Delegate weak references.** Apple's `VZVirtualMachine.delegate` |
| 137 | + is a `__weak` property — typical Cocoa pattern to avoid |
| 138 | + retain cycles. If we let the |
| 139 | + `Retained<ElastosVzDelegate>` drop while the VM is still |
| 140 | + alive, the VM's weak ref becomes nil and delegate methods |
| 141 | + silently stop firing. `VzMachineHandle::delegate` holds the |
| 142 | + retained delegate to ensure parity with the VM lifetime. |
| 143 | +- **`setDelegate:` threading.** Apple requires VM property |
| 144 | + mutations to happen on the VM's associated dispatch queue, |
| 145 | + so the delegate is set via `queue.as_raw().exec_sync(...)` |
| 146 | + after init. |
| 147 | +- **Host-initiated stops don't fire the delegate.** Apple's |
| 148 | + semantics distinguish "the VM was told to stop" (completion |
| 149 | + handler on `stopWithCompletionHandler:`) from "the VM |
| 150 | + stopped on its own" (delegate). We bridge the gap by |
| 151 | + signalling `DelegateExit::HostInitiatedStop` on the shared |
| 152 | + channel inside our `stop()` implementation. |
| 153 | +- **`VZVirtioSocketConnection` fd ownership.** Apple's docs: |
| 154 | + "The file descriptor is owned by the |
| 155 | + `VZVirtioSocketConnection`. It is automatically closed when |
| 156 | + the object is destroyed." We never call `close` explicitly; |
| 157 | + the `Retained<VZVirtioSocketConnection>` drops when the |
| 158 | + completion block exits, taking its fd with it. Our `dup`'d |
| 159 | + fd keeps the kernel socket endpoint alive on its own. |
| 160 | + |
| 161 | +### What is still *not* working after Day 5 |
| 162 | + |
| 163 | +- **Full host→guest provider bridge on Mac is not yet wired.** |
| 164 | + Day 5 ships the FFI primitive (`RunningVm::connect_vsock`) |
| 165 | + but `elastos-server/src/vm_provider.rs::try_vsock_connect` |
| 166 | + still uses the Linux-only `socket(AF_VSOCK)` path. Wiring it |
| 167 | + through requires a `MacVsockDialer` trait threaded into |
| 168 | + `VmCapsuleProvider` plus Mac-side `register_provider_route` |
| 169 | + changes (the Mac arm currently returns `None` because |
| 170 | + `vm_config.network` is `None` — no TAP). Day 6. |
| 171 | +- **TAP networking** is still rejected with the typed |
| 172 | + entitlement-required error from Day 2 (no silent NAT |
| 173 | + downgrade). |
| 174 | +- **`stop_capsule` race.** If the supervisor calls |
| 175 | + `stop_capsule` *while* `wait_for_exit` is awaiting, the |
| 176 | + shared channel resolves with `HostInitiatedStop` from the |
| 177 | + `stop()` path — but the wait task may have already taken |
| 178 | + the receiver. The current code returns the |
| 179 | + `HostInitiatedStop` exit code from `wait_for_exit` and |
| 180 | + silently ignores the second take from `stop_capsule`'s |
| 181 | + status update. This is fine in practice (both paths agree |
| 182 | + the VM has stopped) but worth a follow-up if any caller |
| 183 | + observes mismatched ordering. |
| 184 | + |
| 185 | +### Linux-untouched evidence |
| 186 | + |
| 187 | +- `scripts/check-linux-untouched.sh bcf5a0a`: green. |
| 188 | +- All new code lives in `elastos-vz` (Mac-only crate) or its |
| 189 | + Mac-gated test files. No `elastos-server` changes for the |
| 190 | + delegate work; `elastos-server` still receives the same |
| 191 | + `Result<i32>` shape from `RunningVm::wait_for_exit_code` |
| 192 | + it always has. |
| 193 | +- `cargo clippy --workspace --all-targets -- -D warnings`: |
| 194 | + clean on Mac AND Linux. One new `type VsockResultSender` |
| 195 | + alias to suppress `clippy::type_complexity` on the shared |
| 196 | + oneshot type. |
| 197 | +- 511 tests green locally on Mac (Day 4 ended at 507; Day 5 |
| 198 | + added 4 — three in `elastos-vz` lib, one in the integration |
| 199 | + smoke suite). |
| 200 | + |
| 201 | +### Day 6 handoff |
| 202 | + |
| 203 | +1. **`vm_provider.rs` Mac integration.** Introduce a |
| 204 | + `VsockDialer` trait or `Arc<dyn Fn(u32) -> ...>` closure on |
| 205 | + `VmCapsuleProvider`. On Mac, `start_capsule_vm_macos` |
| 206 | + captures a `Weak<RwLock<HashMap<String, RunningCapsule>>>` |
| 207 | + into a closure that looks up the handle, downcasts to |
| 208 | + `CapsuleBackend::VzVm`, and calls `RunningVm::connect_vsock`. |
| 209 | + Linux's `socket(AF_VSOCK)` path stays unchanged. |
| 210 | +2. **First-party capsule provider end-to-end.** Validate the |
| 211 | + full chain (`localhost-provider` capsule on Mac → vsock |
| 212 | + dial from a sibling capsule via the supervisor's provider |
| 213 | + registry) once Day 6 wires the dialer. This is the smoke |
| 214 | + test that proves "Mac capsules can serve each other". |
| 215 | +3. **`VZVirtualMachineDelegate` smoke test.** A standalone |
| 216 | + integration test that boots a real (minimal) kernel and |
| 217 | + asserts the delegate exit code matches the kernel's |
| 218 | + shutdown reason. Today's coverage is unit-level (mocking |
| 219 | + `signal_exit` directly); a boot-driven test would land |
| 220 | + alongside the `vm-debug boot` Phase 2 Day 5 harness. |
0 commit comments