Add openvmm upstream test and stress test#4457
Conversation
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
51f6969 to
387ebdc
Compare
There was a problem hiding this comment.
Pull request overview
Adds an OpenVMM “upstream vmm_tests” test suite and supporting Tool implementation, along with unit tests and a small schema refactor to share cloud-init configuration.
Changes:
- Introduces
OpenVmmUpstreamTestSuite.verify_openvmm_upstream_vmm_teststo run upstream OpenVMMvmm_testson a Linux x64 host with configurable guest/filter options. - Adds
OpenVmmTestsTool to clone/prepare the OpenVMM repo, run the upstream flowey pipeline, and parse results from JUnit/log output. - Refactors
CloudInitSchemainto a sharedsut_orchestrator/util/schema.pylocation and updates OpenVMM/Libvirt schemas accordingly.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| selftests/test_openvmm_tests_tool.py | Unit tests for JUnit/log parsing and failure summarization behavior. |
| selftests/test_openvmm_tests_suite.py | Unit tests for host capability checks, filter composition, and result message formatting. |
| lisa/sut_orchestrator/util/schema.py | Adds shared CloudInitSchema for reuse across orchestrators. |
| lisa/sut_orchestrator/openvmm/schema.py | Switches to importing shared CloudInitSchema instead of defining it locally. |
| lisa/sut_orchestrator/openvmm/node.py | Adjusts OpenVmmController construction to be explicitly host-node based. |
| lisa/sut_orchestrator/libvirt/schema.py | Switches to importing shared CloudInitSchema instead of defining it locally. |
| lisa/microsoft/testsuites/openvmm/openvmm_tests_tool.py | New Tool that clones/prepares OpenVMM repo, runs vmm tests, archives logs/artifacts, and parses JUnit/log results. |
| lisa/microsoft/testsuites/openvmm/openvmm_tests.py | New test suite entrypoint wiring variables/host checks into the Tool run. |
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
|
@copilot apply changes based on the comments in this thread |
|
@copilot fix failing checks. |
Applied all changes from the review thread (commit 8eba786):
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
⏭️ AI Test Selection — SKIPPED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest Selected test cases |
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
Fixed in d7c0d12. The CI failure was caused by Added |
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
d7c0d12 to
b6c740d
Compare
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
Add a LISA testsuite for running OpenVMM upstream vmm_tests through cargo-nextest, including runbook-facing test selection, artifact handling, and result reporting. Introduce an OpenVMM vmm_tests tool wrapper that discovers tests, stages required artifacts, executes selected tests, and parses nextest output for LISA result handling. Add selftests for the suite metadata and tool behavior so selection parsing, artifact discovery, command construction, and result parsing are covered without requiring a live OpenVMM host.
Agent-Logs-Url: https://github.com/microsoft/lisa/sessions/0fe71e59-42bb-45ed-a5e0-85413c527499 Co-authored-by: vyadavmsft <1424753+vyadavmsft@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/lisa/sessions/0fe71e59-42bb-45ed-a5e0-85413c527499 Co-authored-by: vyadavmsft <1424753+vyadavmsft@users.noreply.github.com>
… pollution from imported test modules Agent-Logs-Url: https://github.com/microsoft/lisa/sessions/f7158c05-06e1-45d1-b764-81392a2785b9 Co-authored-by: vyadavmsft <1424753+vyadavmsft@users.noreply.github.com>
Advertise the OpenVMM TAP gateway as both router and DNS server through dnsmasq DHCP options so guests can resolve external package and source endpoints during runtime tool installation. Allow DHCP and DNS traffic to the host TAP or bridge interface, track only the iptables rules added for a guest, and remove those tracked rules during teardown. Log local OpenVMM artifact copy start and completion times so large disk or firmware uploads are visible in pipeline logs instead of looking like a hang.
When multiple OpenVMM guests are provisioned on the same parent host, each guest can try to copy local artifacts through the shared host shell at the same time. The stress-ng guest run showed both guests entering the MSVM.fd copy and never reaching the matching completion log, so guest launch and stress-ng never started. Add a host-scoped artifact copy lock to the OpenVMM host context and use it for local-to-host artifact staging and cloud-init ISO copy. This keeps multi-guest provisioning parallel while serializing the host shell copy operation that is shared by guests on the same parent host.
OpenVMM guests do not currently expose the SerialConsole feature, but the stress-ng suite unconditionally queried node.features[SerialConsole] when handling stress execution failures. In OpenVMM guest stress runs this masked the original failure with a secondary LisaException about SerialConsole not being supported on the openvmm platform. Route panic checks through a helper that first checks node.features.is_supported(SerialConsole). Platforms with serial console support keep the existing panic detection behavior, while OpenVMM and other platforms without the feature log a debug skip and preserve the original stress-ng failure for diagnosis.
Install and stage stress-ng on every target node before launching any stress job. This keeps multi-node stress runs from starting load on earlier nodes while later nodes are still resolving package dependencies or copying job files. The split prepare/launch flow also makes failures easier to diagnose: setup issues are reported as preparation failures, while launch issues remain tied to starting the stress process.
Avoid repeatedly copying the same local OpenVMM artifacts from the controller to the host when a guest template expands into multiple guests. The controller now uploads each local artifact once into a per-host working cache and then creates a per-guest host-local copy with cp --reflink=auto. This keeps writable disk images unique for g0..gN while avoiding the long serialized SSH transfers seen in multi-guest stress-ng provisioning. The existing artifact copy lock still protects the shared cache population. Add selftest coverage that two guest working paths resolving the same source artifact reuse one host upload while producing distinct per-guest destinations.
9949177 to
01ce6bf
Compare
01ce6bf to
0256d0a
Compare
|
rebased with main |
|
| Count | |
|---|---|
| ✅ Passed | 0 |
| ❌ Failed | 0 |
| ⏭️ Skipped | 3 |
| Total | 3 |
Test case details
| Test Case | Status | Time (s) | Message |
|---|---|---|---|
| verify_openvmm_stop_start_in_platform (lisa_0_4) | ⏭️ SKIPPED | 5.172 | before_case skipped: This suite only applies to OpenVMM guest nodes. Actual node type: RemoteNode. |
| verify_openvmm_guest_boot (lisa_0_2) | ⏭️ SKIPPED | 5.762 | before_case skipped: This suite only applies to OpenVMM guest nodes. Actual node type: RemoteNode. |
| verify_openvmm_restart_via_platform (lisa_0_3) | ⏭️ SKIPPED | 5.723 | before_case skipped: This suite only applies to OpenVMM guest nodes. Actual node type: RemoteNode. |
✅ AI Test Selection — PASSED5 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
Use the canonical OpenVMM tests tool import path so the upstream test suite and host tool cache reference the same OpenVmmTests class object. Fail upstream vmm_tests result parsing when neither the local JUnit report nor the copied log exists, preventing artifact copy-back failures from being treated as successful empty summaries. Track TAP host service INPUT rules only through tap_input_rules_added. This removes the stale DHCP-specific flag now that both DHCP and DNS rules are added and removed as a group. Clone configurable upstream OpenVMM repositories into repo-url-specific directories and refresh repo/artifact paths when repository settings change, so a previous checkout cannot mask a different repo_url. Agent-Logs-Url: https://github.com/microsoft/lisa/sessions/a8f6f648-499b-4476-8799-899857284ce0 Co-authored-by: vyadavmsft <1424753+vyadavmsft@users.noreply.github.com>
|
@copilot apply changes based on the comments in this thread |
…racebacks Agent-Logs-Url: https://github.com/microsoft/lisa/sessions/eae844c4-6362-49b2-8ce5-14c50efd0f1a Co-authored-by: vyadavmsft <1424753+vyadavmsft@users.noreply.github.com>
Applied in 6b22b4a. Changed |
✅ AI Test Selection — PASSED5 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
✅ AI Test Selection — PASSED5 test case(s) selected (view run) Marketplace image: microsoftcblmariner azure-linux-3 azure-linux-3 latest
Test case details
|
|
@vyadavmsft can you please share the test results for this PR? |
2026-05-07 20:50:59.638[140345414420288][INFO] lisa.RootRunner OpenVmmUpstreamTestSuite.verify_openvmm_upstream_vmm_tests: PASSED upstream vmm_tests:
|
|
@pupacha @anirudhrb please review, thanks. |
|
@LiliDeng pls check, |
Description
Adds an OpenVMM “upstream vmm_tests” test suite and supporting Tool implementation, along with unit tests and a small schema refactor to share cloud-init configuration.
Changes:
Introduces OpenVmmUpstreamTestSuite.verify_openvmm_upstream_vmm_tests to run upstream OpenVMM vmm_tests on a Linux x64 host with configurable guest/filter options.
Adds OpenVmmTests Tool to clone/prepare the OpenVMM repo, run the upstream flowey pipeline, and parse results from JUnit/log output.
Refactors CloudInitSchema into a shared sut_orchestrator/util/schema.py location and updates OpenVMM/Libvirt schemas accordingly.
Related Issue
Type of Change
Checklist
Test Validation
Key Test Cases:
Impacted LISA Features:
Tested Azure Marketplace Images:
Test Results