ch_tests_tool: run all perf tests in one go

[anirudhrb]

This pull request significantly refactors the Cloud Hypervisor performance metrics test logic to simplify test execution, improve result collection, and remove legacy per-test configuration. The main focus is on running all metrics subtests in a single invocation, collecting results from a generated JSON report, and eliminating no-longer-needed per-test environment variables and logic.

Related Issue

Type of Change

• Bug fix
• New feature
• Breaking change
• Refactoring
• Documentation update

Checklist

• Description is filled in above
• No credentials, secrets, or internal details are included
• Peer review requested (if not, add required peer reviewers after raising PR)
• Tests executed and results posted below

Test Validation

Key Test Cases:

Impacted LISA Features:

Tested Azure Marketplace Images:

Test Results

Image	VM Size	Result
		PASSED / FAILED / SKIPPED

[Copilot]

Pull request overview

This PR refactors the Cloud Hypervisor performance metrics test flow to run all metrics subtests in a single invocation, then derive per-subtest pass/fail and metrics from a generated JSON report, removing legacy per-test env/config handling.

Changes:

• Run metrics tests once with --continue-on-failure and collect per-subtest results from a JSON report.
• Consolidate one-time host setup into _ensure_host_setup() and remove per-test policies/env var logic.
• Remove MS-CLH variables for per-test block-size tuning that are no longer used.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
lisa/microsoft/testsuites/cloud_hypervisor/ch_tests_tool.py	Reworks metrics execution to one run + JSON report parsing; adjusts host setup and diagnostics behavior.
lisa/microsoft/testsuites/cloud_hypervisor/ch_tests.py	Removes now-unused variables for metrics block-size configuration passed into the tool.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

lisa/microsoft/testsuites/cloud_hypervisor/ch_tests_tool.py:539

• run_metrics_tests no longer treats the overall command exit status as authoritative. If the metrics runner exits non-zero (e.g., one or more subtests failed) but the JSON report still contains entries, the function can end up passing because failed_testcases stays empty and there is no result.exit_code/result.is_timeout assertion. Please fail the test when result.is_timeout is true or result.exit_code != 0, and/or cross-check the report contents against the expected testcase list so missing/failed subtests can’t be silently ignored.

        for testcase, entry in per_test_results.items():
            status, metrics, trace = self._classify_test_result(testcase, entry)

            if status == TestStatus.FAILED:
                failed_testcases.append(testcase)

            self._send_metrics_test_result(
                test_result, testcase, status, metrics, trace
            )

        if len(failed_testcases) > 0:
            diagnostic_info = self._extract_diagnostic_info(
                log_path, test_name, result
            )

            self._log.error(
                f"Test cases failed: {failed_testcases} "
                f"Diagnostics: {diagnostic_info}"
            )

        self._check_test_panic_from_logs(
            test_result=test_result,
            content=result.stdout,
            stage="metrics tests",
            test_name="run_metrics_tests",
        )

        self._save_kernel_logs(log_path)

        # Check for kernel panic after all tests complete
        if self.node.features.is_supported(SerialConsole):
            self.node.features[SerialConsole].check_panic(
                saved_path=log_path, force_run=True, test_result=test_result
            )

        assert_that(
            failed_testcases, f"Failed Testcases: {failed_testcases}"
        ).is_empty()

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.