Skip to content

CHORE: Flaky Tests - skip subprocess shutdown/teardown tests under QEMU and fix detection - #673

Merged
Gaurav Sharma (bewithgaurav) merged 3 commits into
mainfrom
bewithgaurav/skip-qemu-shutdown-teardown-tests
Aug 13, 2026
Merged

CHORE: Flaky Tests - skip subprocess shutdown/teardown tests under QEMU and fix detection#673
Gaurav Sharma (bewithgaurav) merged 3 commits into
mainfrom
bewithgaurav/skip-qemu-shutdown-teardown-tests

Conversation

@bewithgaurav

@bewithgaurav Gaurav Sharma (bewithgaurav) commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Work Item / Issue Reference

ADO Work Item: Fixed AB#47261


Summary

the subprocess shutdown/teardown tests in test_013_SqlHandle_free_shutdown and test_024_context_manager_transaction intermittently segfault on the qemu-emulated linux arm64 CI legs and force reruns. the crash is a latent use-after-free in odbc handle-teardown ordering (an unclosed DBC handle finalized by gc at interpreter shutdown after the process-lifetime ENV handle is gone, so SQLFreeHandle touches freed ENV memory); it is benign on native hardware and only turns fatal under qemu's page reuse plus altered gc timing, which is why it is flaky and clears on rerun.

test-only, two changes:

  • fix is_qemu_emulated() in conftest.py. the old check only matched CPU implementer 0x51, which never appears under multiarch/qemu-user-static (the emulated aarch64 process sees the x86_64 host's /proc/cpuinfo, which has no arm implementer field). it now flags an aarch64 process when cpuinfo advertises implementer 0x51 (qemu-system) or lacks the arm-only CPU implementer field entirely (qemu-user host passthrough); native arm64 and x86 stay unflagged. the existing test_013 skip was dead because of this and now takes effect.
  • add a class-level skipif(QEMU) to test_024's TestContextManagerCommit (all 24 cases run the subprocess-teardown path).

the conftest docstring documents the underlying teardown-order uaf as the known-failure case. that root-cause bug is tracked separately and is unaffected by this skip; native legs still run these tests, so a regression there is not masked.

…ection

the subprocess shutdown/teardown tests in test_013 and test_024 intermittently segfault on the qemu-emulated linux arm64 CI legs and force reruns. the crash is a latent use-after-free in odbc handle-teardown ordering (an unclosed DBC handle finalized by gc at interpreter shutdown after the process-lifetime ENV handle is gone, so SQLFreeHandle touches freed ENV memory). it is benign on native hardware and only turns fatal under qemu's page reuse plus altered gc timing, so it is a flaky heisenbug.

is_qemu_emulated() was effectively dead: it only matched 'CPU implementer 0x51', which never appears under multiarch/qemu-user-static because the emulated aarch64 process sees the x86_64 host's /proc/cpuinfo. fix it to flag an aarch64 process when cpuinfo advertises implementer 0x51 (qemu-system) or lacks the arm-only 'CPU implementer' field entirely (qemu-user host passthrough). native arm64 and x86 stay unflagged.

extend the existing skip to test_024's TestContextManagerCommit (all 24 cases run the subprocess-teardown path); test_013 already had the marker, and the detection fix makes it take effect. the underlying teardown-order uaf is tracked separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

82%


📈 Total Lines Covered: 7368 out of 8963
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 59.9%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 76.6%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@github-actions github-actions Bot added pr-size: small Minimal code update labels Aug 12, 2026
@bewithgaurav
Gaurav Sharma (bewithgaurav) marked this pull request as ready for review August 12, 2026 13:53
Copilot AI lite review requested due to automatic review settings August 12, 2026 13:53
@bewithgaurav Gaurav Sharma (bewithgaurav) changed the title CHORE: skip subprocess shutdown/teardown tests under QEMU and fix detection CHORE: Remove Flaky Tests - skip subprocess shutdown/teardown tests under QEMU and fix detection Aug 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Python test suite to avoid flaky subprocess shutdown/teardown crashes on QEMU-emulated Linux ARM64 CI, while keeping the same tests running on native hardware.

Changes:

  • Fixes QEMU user-mode emulation detection in tests/conftest.py so existing QEMU-only skips actually trigger on the affected CI legs.
  • Skips the entire subprocess-based context-manager transaction test class under QEMU to prevent intermittent SIGSEGVs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/test_024_context_manager_transaction.py Adds a class-level skipif(QEMU) to avoid flaky subprocess teardown crashes under QEMU emulation.
tests/conftest.py Updates is_qemu_emulated() logic to detect QEMU user-mode ARM64-on-x86_64 more reliably and documents the known-failure case.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/conftest.py
@bewithgaurav Gaurav Sharma (bewithgaurav) changed the title CHORE: Remove Flaky Tests - skip subprocess shutdown/teardown tests under QEMU and fix detection CHORE: Flaky Tests - skip subprocess shutdown/teardown tests under QEMU and fix detection Aug 12, 2026
@bewithgaurav
Gaurav Sharma (bewithgaurav) merged commit 40b2e32 into main Aug 13, 2026
38 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: small Minimal code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants