Skip to content

Fix pyarrow version which causes winml eval crash#750

Merged
zhenchaoni merged 1 commit into
release/v0.1.0from
private/zhenni/fix_pyarrow
May 26, 2026
Merged

Fix pyarrow version which causes winml eval crash#750
zhenchaoni merged 1 commit into
release/v0.1.0from
private/zhenni/fix_pyarrow

Conversation

@zhenchaoni
Copy link
Copy Markdown
Member

Fixes #749

PR Description

Fix: winml eval crash when --device is not specified

Problem

Running winml eval without an explicit --device flag crashes the process with an access violation (0xC0000005) during dataset loading.

Root Cause

The crash is triggered by a DLL incompatibility between pyarrow 24.x and the WinML/ONNX Runtime EP DLLs:

  1. When --device is omitted, winml eval resolves the device and preloads the EP DLLs (via _preload_bundled_onnxruntime_dll) before importing the datasets package.
  2. Importing datasets then loads pyarrow's C extension on top of the already-loaded EP DLLs.
  3. pyarrow 24.x ships a native binary that conflicts with the WinML EP DLLs in this load order, causing the access violation.

This was only reproducible with pyarrow 24.x; versions ≤ 23 load cleanly regardless of order.

Fix

Cap pyarrow to >=21,<24 in pyproject.toml. This stays within the range required by datasets (which needs pyarrow>=21), so no other dependencies are affected.

The existing comment above the pin already documents the crash details and the conditions for revisiting the cap (re-test 24.x patches / 25.x before relaxing).

Validation

  • uv run winml eval -m microsoft/resnet-50 --samples 100 now completes without --device specified.
  • No change to runtime behavior for users who already pinned pyarrow < 24.

@zhenchaoni zhenchaoni requested a review from a team as a code owner May 26, 2026 08:00
@zhenchaoni zhenchaoni merged commit 8588134 into release/v0.1.0 May 26, 2026
9 checks passed
@zhenchaoni zhenchaoni deleted the private/zhenni/fix_pyarrow branch May 26, 2026 08:34
timenick added a commit that referenced this pull request May 27, 2026
…757)

## Summary

- Folds the seven fixes merged after the v0.1.0 release notes into the
existing v0.1.0 changelog section: #745, #746 (inspect), #747, #753
(perf), #750 (eval), #744/#752, #754 (analyze/compile e2e).
- Trims the v0.1.0 section to focus on user-visible changes:
  - Removed telemetry mentions (#728 preview bullet and #693 fix line).
- Removed pure-internal CI/test items (CI display name rename, ADO
pipeline UI plumbing, mock fixes, branch back-merge).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants