Skip to content

Add winml eval support for audio-classification #1175

Description

@fuliucansheng

winml eval currently rejects the audio-classification task before dataset selection, even when winml inspect and winml build resolve/build a standard audio-classification model.

Repro from facebook/mms-lid-256 on current main:

uv run winml eval --schema --task audio-classification
uv run winml eval -m temp\facebook_mms-lid-256_tester\cpu\fp32\model.onnx --model-id facebook/mms-lid-256 --task audio-classification --ep cpu --device cpu

Observed:

Error: Task 'audio-classification' is not supported by `winml eval`.
Error: Evaluation failed: Task 'audio-classification' is not supported.

Expected: an audio-classification eval schema/adapter with dataset preprocessing for raw audio or feature-extractor-normalized waveform input and a task metric such as accuracy.

This blocks Goal-L3 validation for standard Wav2Vec2ForSequenceClassification audio classifiers such as facebook/mms-lid-256.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementmodel-scale-by-skillModel support PR created or maintained by the adding-model-support skill

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions