Skip to content

recipe(open-vakgyata): add CPU audio-classification recipes#1131

Merged
ssss141414 merged 1 commit into
mainfrom
ssss141414/add-onecxi-open-vakgyata-recipe
Jul 20, 2026
Merged

recipe(open-vakgyata): add CPU audio-classification recipes#1131
ssss141414 merged 1 commit into
mainfrom
ssss141414/add-onecxi-open-vakgyata-recipe

Conversation

@ssss141414

@ssss141414 ssss141414 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

This adds CPU fp32 and fp16 coverage recipes for onecxi/open-vakgyata, a Wav2Vec2 audio-classification checkpoint for identifying 10 Indian speech languages. Effort L0 / Outcome L0 ships recipe-only coverage for both required tuples. The committed Goal L2 was reached with L2 PASS for both precisions; supplementary task evaluation remains CLI-BLOCKED because winml eval has no registered audio-classification evaluator.

Model metadata

What the model does

open-vakgyata is a speech language-identification model: it accepts a normalized 16 kHz mono waveform and returns logits over 10 Indian speech-language labels (en-IN, hi-IN, or-IN, bn-IN, ta-IN, te-IN, kn-IN, ml-IN, mr-IN, and gu-IN). Confidence: verified.

  • Evidence: the pinned model card documents the model overview, supported languages, 16 kHz input, and inference; the pinned checkpoint config declares Wav2Vec2ForSequenceClassification, 10 labels, and id2label; the pinned preprocessor config declares 16 kHz normalization. The measured baseline interface was input_values float32[1,16000] -> logits float32[1,10].

Primary user stories

  • A user supplies a 16 kHz speech waveform to obtain the predicted language label for selecting language-aware downstream speech processing. Confidence: verified. Evidence: the pinned model card describes language identification, recommended audio format, and logits-to-id2label inference; the pinned checkpoint config defines the 10 locale-code labels.

Supported tasks

  • audio-classification, exposed by the checkpoint, Transformers, Optimum ONNX, and WinML. Confidence: verified. Evidence: the pinned checkpoint has pipeline_tag=audio-classification; Wav2Vec2Config resolves through AutoModelForAudioClassification to Wav2Vec2ForSequenceClassification; the Optimum probe includes audio-classification; and WinML at the pinned baseline resolves audio-classification, Wav2Vec2OnnxConfig, and WinMLModelForGenericTask and completes a recipe-free build.

Model architecture

Wav2Vec2ForSequenceClassification (58,746,794 parameters)
├── Wav2Vec2Model
│   ├── Feature encoder: Conv1d + LayerNorm + GELU x 7 (1 -> 512 channels)
│   ├── Feature projection: 512 -> 1280
│   └── Stable-LayerNorm encoder
│       ├── Grouped positional Conv1d (kernel 128, groups 16)
│       └── Encoder layer x 2
│           ├── Self-attention (16 heads, hidden size 1280)
│           ├── Feed-forward (1280 -> 5120 -> 1280, GELU)
│           └── Attention adapter (1280 -> 16 -> 1280, ReLU)
├── Projector: 1280 -> 1024
├── Temporal mean pooling
└── Classifier: 1024 -> 10 language logits

Validation and support evidence

Baseline

  • Pinned baseline: microsoft/winml-cli@e7509b1e908c74beff0a5655b8f8d7de69c5afae, winml, version 0.2.0.
  • Starting auto-config behavior: PASS; current main automatically resolved task=audio-classification, loader.model_class=AutoModelForAudioClassification, and concrete Wav2Vec2ForSequenceClassification. No task, loader, architecture, or metadata override was needed. The generated config was the starting recipe.
  • Recipe-free build: exit 0 in 34.6 s; input_values FLOAT[1,16000] -> logits FLOAT[1,10]; ONNX opset 17; 88 FLOAT and 0 FLOAT16 initializers.
  • Baseline CPU perf: exit 0 on CPUExecutionProvider, fp32, 100 iterations; mean 34.886 ms, p50 33.872 ms, p90 38.320 ms, throughput 28.66 samples/s; RAM model-load +165.34 MB, inference +106.93 MB, total +272.28 MB. OpenVINO plugin registration emitted a missing onnxruntime_providers_shared.dll error, but CPU benchmarking continued and completed successfully.
  • Baseline eval: exit 1, Evaluation failed: Task 'audio-classification' is not supported. The initial invocation without --model-id exited 2 with When using an ONNX file, --model-id is required for preprocessor and config resolution. This task-registry blocker occurs independently of dataset selection. The checkpoint supplies no evaluation dataset revision/config/split, audio-column mapping, label mapping, or task metric.
  • Baseline floor: L1 because recipe-free build and perf passed.
  • Optimum probe: VENDOR-ONLY. Vendor and post-WinML task lists were identical: audio-classification, audio-frame-classification, audio-xvector, automatic-speech-recognition, and feature-extraction; WinML added no task.

Goal

  • Effort: L0 — recipe-only.
  • Committed Goal ceiling: L2.
  • Outcome: L0 — recipes plus report.
  • Success definition: for both required CPU/cpu precisions, fresh recipe config/build and structural precision validation at L0; measured CPU perf at L1; and real named-input ONNX-vs-PyTorch numeric comparison at L2, with no PASS above FAIL.
  • Charter revision 2 was explicitly re-issued without changing the L2 ceiling; it clarified that the missing audio-classification eval registry and absent checkpoint dataset metadata are non-blocking L3/eval limitations rather than a blocking question.

Outcome

  • Shipped tier: L0.
  • Highest Goal verdict: L2 PASS.
  • Coverage: full; deferred tuples: none.
  • Shipped recipes:
    • examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp32_config.json
    • examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp16_config.json
  • Source/code changes: none.
  • Model finding wav2vec2-003 was captured separately: the standard stable-LayerNorm Wav2Vec2ForSequenceClassification checkpoint resolves without overrides, reaches CPU L2 parity at fp32 and fp16, and has Add=35, Mul=24, and Transpose=23 as the top-three operator counts in both analyzed artifacts.
  • No methodology friction observed.
  • Supplementary eval remains CLI-BLOCKED for both tuples because audio-classification has no registered WinML evaluator; the requested capability is an audio-classification schema, dataset adapter, preprocessing path, and task metric evaluator.

Per-EP/device/precision results — including perf and eval data

Goal ladder

Tier Overall verdict Evidence
L0 PASS Both CPU/cpu precision candidates were freshly configured, built, freshness/semantic-match checked, and structurally validated.
L1 PASS Both artifacts completed 100 measured CPU iterations with latency, throughput, and process/device memory captured.
L2 PASS The pinned PyTorch checkpoint and both ONNX artifacts were compared on identical Wav2Vec2FeatureExtractor-normalized named input from a valid 16 kHz mono 16-bit PCM WAV.

L0 structural results

EP / Device Precision Verdict Structural evidence
CPUExecutionProvider / cpu fp32 PASS Exit 0; onnx.checker full-check PASS; IR 8; ai.onnx opset 17; input_values FLOAT[1,16000] -> logits FLOAT[1,10]; initializers: FLOAT 88, INT64 6; external data 234,981,376 bytes and colocated; fresh recipe/config semantic match true; random-head warning false.
CPUExecutionProvider / cpu fp16 PASS Exit 0; onnx.checker full-check PASS; IR 8; ai.onnx opset 17; input_values FLOAT[1,16000] -> logits FLOAT[1,10]; initializers: FLOAT16 88, INT64 6; external data 117,490,688 bytes and colocated; fresh recipe/config semantic match true; random-head warning false.

Perf results

EP / Device Precision Verdict Iterations Mean p50 p90 Throughput RAM load / inference / total Δ VRAM local / shared total Δ
CPUExecutionProvider / cpu fp32 PASS 100 36.369 ms 35.356 ms 41.316 ms 27.5 samples/s 166.2 / 106.95 / 273.15 MB 0.0 / 0.0 MB
CPUExecutionProvider / cpu fp16 PASS 100 41.95 ms 41.124 ms 46.357 ms 23.84 samples/s 234.05 / 43.35 / 277.4 MB 0.0 / 0.0 MB

The provider snapshot contained DmlExecutionProvider and CPUExecutionProvider; each measured run selected CPUExecutionProvider.

L2 numeric results

The reference was the real Wav2Vec2ForSequenceClassification checkpoint pinned at f2754058e485dfc65cc62589b8d0e21c1d328399. Input was named input_values, shape [1,16000], dtype tensor(float), from a normalized valid 16 kHz mono 16-bit PCM WAV containing 16,000 samples.

EP / Device Precision Verdict Cosine similarity Max abs error Mean abs error Top-1 PT / ONNX Agreement
CPUExecutionProvider / cpu fp32 PASS 0.9999999996964098 0.0005121231079101562 0.00011473149061203003 4 / 4 true
CPUExecutionProvider / cpu fp16 PASS 0.999999301707027 0.015536665916442871 0.007226626574993134 4 / 4 true

Eval results

Schema probe verdict: CLI-FEATURE-GAP, exit 2. winml eval reports that audio-classification is unsupported and rejects the task before dataset selection.

EP / Device Precision Verdict Exit Dataset / revision / subset Metric Exact blocker
CPUExecutionProvider / cpu fp32 CLI-BLOCKED 1 — / — / — audio-classification has no registered winml eval evaluator; rejection occurs before dataset selection. The CLI also warns that --precision fp32 is ignored for pre-built ONNX inputs because precision is already baked into the model.
CPUExecutionProvider / cpu fp16 CLI-BLOCKED 1 — / — / — audio-classification has no registered winml eval evaluator; rejection occurs before dataset selection. The CLI also warns that --precision fp16 is ignored for pre-built ONNX inputs because precision is already baked into the model.

Delta

  • The fp32 recipe is semantically IDENTICAL to the frozen starting auto-config after recursively removing _note members.
  • The fp16 recipe changes exactly one JSON pointer relative to that starting auto-config: RFC 6902 replace at /quant, from null to the following existing WinML-generated precision configuration:
{"mode":"fp16","samples":10,"calibration_method":"minmax","weight_type":"uint8","activation_type":"uint8","per_channel":false,"symmetric":false,"weight_symmetric":null,"activation_symmetric":null,"save_calibration":false,"distribution":"uniform","seed":null,"calibration_load_path":null,"calibration_save_path":null,"op_types_to_quantize":null,"nodes_to_exclude":null,"task":"audio-classification","model_id":"onecxi/open-vakgyata","model_type":"wav2vec2","fp16_keep_io_types":true,"fp16_op_block_list":null}
  • Reason: explicit human precision intent for the required fp16 tuple, emitted by the existing generic WinML fp16 configuration path; it does not correct task, loader, architecture, or metadata resolution.
  • Both checked-in recipes are semantically identical to their respective generated config outputs. There are no source paths, code symbols, or class-wide behavior changes.
  • No-recipe acceptance: NOT-REQUIRED. Current main already resolves architecture, task, loader, input, and output without overrides, and no generalized source behavior was added or needed.
  • Reducibility is consistent with the charter. The actual fix class remains recipe; no safe missing metadata/config/architecture-derived class rule exists to implement.
  • examples/recipes/README.md remains untouched.

Analyze summary — component level and op level

Tester status: PASS for the complete emitted JSON and retained classifications; this is static rule compatibility analysis, not runtime execution. A fresh source checkout contains only the runtime-rule README, so public reproduction requires the versioned rules-v0.2.0.zip archive, SHA-256 check, 1,746-parquet extraction check, and WINMLCLI_RULES_DIR setup shown below. On this host the analyzer process exits 1 because optional provider registration fails, but still emits complete 11-row JSON whose metadata and classifications exactly match the retained internal rule-backed evidence. This is an honest partial process success, not proof of runtime execution; no component-mapping gaps remain.

Component-level summary

Artifact Architecture coverage Mapping Actionable EP findings
fp32 7-stage feature encoder; feature projection; stable-LayerNorm encoder with 2 layers; projector; temporal mean pool; classifier 163 mapped, 0 partial, 0 unmapped; confidence mapped QNN NPU and GPU: partial Div, Erf, Add, Mul; unsupported none
fp16 7-stage feature encoder; feature projection; stable-LayerNorm encoder with 2 layers; projector; temporal mean pool; classifier 165 mapped, 0 partial, 0 unmapped; confidence mapped QNN NPU and GPU: partial Div, Erf, Add, Mul; unsupported none

Op-level summary

Artifact Graph Dominant ops EP roll-up
fp32 163 operators / 15 unique types Add 35; Mul 24; Transpose 23; MatMul 22; LayerNormalization 15 Fully supported: TensorRT GPU and OpenVINO NPU/GPU/CPU. QNN NPU/GPU partial: Div, Erf, Add, Mul; unsupported none.
fp16 165 operators / 16 unique types Add 35; Mul 24; Transpose 23; MatMul 22; LayerNormalization 15 Fully supported: TensorRT GPU and OpenVINO NPU/GPU/CPU. QNN NPU/GPU partial: Div, Erf, Add, Mul; unsupported none.

Rule-less EPs — CUDA GPU, MIGraphX GPU, DirectML GPU, CPUExecutionProvider CPU, and Vitis AI NPU — classify all operator types as unknown for both artifacts.

Reproduce commands

The block below is the tester-supplied portable reproduction. It includes the exact version/HEAD and detached-baseline checks, recipe-free baseline build, fp32/fp16 winml config commands, versioned analyzer-rule prerequisite, complete L2 harness, perf runs, and the expected eval-blocker probes.

Show complete portable PowerShell reproduction
$OUT=(Join-Path (Get-Location) 'temp/onecxi-open-vakgyata-repro')
New-Item -ItemType Directory -Force -Path $OUT | Out-Null
winml --version
git fetch origin main
git rev-parse HEAD
$BASELINE_HEAD='e7509b1e908c74beff0a5655b8f8d7de69c5afae'
git rev-parse $BASELINE_HEAD
git worktree add --detach "$OUT/baseline-src" $BASELINE_HEAD
git -C "$OUT/baseline-src" rev-parse HEAD
Push-Location "$OUT/baseline-src"; winml build -m onecxi/open-vakgyata -o "$OUT/baseline" --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild --no-color; Pop-Location
git worktree remove "$OUT/baseline-src" --force
winml config -m onecxi/open-vakgyata --ep cpu --device cpu --precision fp32 --no-compile -o "$OUT/generated_config_fp32.json" --no-color
winml config -m onecxi/open-vakgyata --ep cpu --device cpu --precision fp16 --no-compile -o "$OUT/generated_config_fp16.json" --no-color
winml build -c examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp32_config.json -m onecxi/open-vakgyata -o "$OUT/fp32" --ep cpu --device cpu --precision fp32 --no-analyze --rebuild --no-color
winml build -c examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp16_config.json -m onecxi/open-vakgyata -o "$OUT/fp16" --ep cpu --device cpu --precision fp16 --no-analyze --rebuild --no-color
Invoke-WebRequest 'https://github.com/microsoft/winml-cli/releases/download/v0.2.0/rules-v0.2.0.zip' -OutFile "$OUT/rules-v0.2.0.zip"
if ((Get-FileHash "$OUT/rules-v0.2.0.zip" -Algorithm SHA256).Hash.ToLowerInvariant() -ne '6dcabbe7f5493fbc2bd23de195ab6e35b3578d510f18c2e220bc5538a1f232cc') { throw 'rules-v0.2.0.zip SHA-256 mismatch' }
Expand-Archive -Path "$OUT/rules-v0.2.0.zip" -DestinationPath "$OUT/rules-v0.2.0" -Force
if (@(Get-ChildItem "$OUT/rules-v0.2.0" -Recurse -Filter *.parquet).Count -ne 1746) { throw 'Expected 1746 runtime rule parquet files' }
$env:WINMLCLI_RULES_DIR=(Resolve-Path "$OUT/rules-v0.2.0").Path
winml analyze --model "$OUT/fp32/model.onnx" --ep all --device all --htp-metadata "$OUT/fp32/export_htp_metadata.json" --output "$OUT/analyze_fp32.json" --overwrite --format json --no-color
winml analyze --model "$OUT/fp16/model.onnx" --ep all --device all --htp-metadata "$OUT/fp16/export_htp_metadata.json" --output "$OUT/analyze_fp16.json" --overwrite --format json --no-color
Remove-Item Env:WINMLCLI_RULES_DIR
winml perf -m "$OUT/fp32/model.onnx" --ep cpu --device cpu --precision fp32 --iterations 100 --warmup 10 --memory --output "$OUT/perf_fp32.json" --overwrite --format json --no-color
winml perf -m "$OUT/fp16/model.onnx" --ep cpu --device cpu --precision fp16 --iterations 100 --warmup 10 --memory --output "$OUT/perf_fp16.json" --overwrite --format json --no-color
$L2 = @'
import json, math, sys
from pathlib import Path
import numpy as np
import onnxruntime as ort
import torch
from transformers import AutoFeatureExtractor, AutoModelForAudioClassification

MODEL_ID = "onecxi/open-vakgyata"
REVISION = "f2754058e485dfc65cc62589b8d0e21c1d328399"
SAMPLE_RATE = 16000

def cosine(a, b):
  av = a.astype(np.float64).ravel()
  bv = b.astype(np.float64).ravel()
  return float(np.dot(av, bv) / (np.linalg.norm(av) * np.linalg.norm(bv)))

t = np.arange(SAMPLE_RATE, dtype=np.float64) / SAMPLE_RATE
chirp_phase = 2.0 * math.pi * (180.0 * t + 0.5 * 620.0 * t * t)
waveform = (0.42 * np.sin(2.0 * math.pi * 220.0 * t)
            + 0.23 * np.sin(2.0 * math.pi * 440.0 * t + 0.3)
            + 0.17 * np.sin(chirp_phase))
pcm = np.round(np.clip(waveform, -1.0, 1.0) * 32767.0).astype("<i2")
decoded = pcm.astype(np.float32) / 32768.0
extractor = AutoFeatureExtractor.from_pretrained(MODEL_ID, revision=REVISION)
input_values = extractor(decoded, sampling_rate=SAMPLE_RATE, return_tensors="pt")["input_values"].cpu().numpy().astype(np.float32)
assert input_values.shape == (1, SAMPLE_RATE)
torch.manual_seed(0)
model = AutoModelForAudioClassification.from_pretrained(MODEL_ID, revision=REVISION).eval()
with torch.inference_mode():
    pt_logits = model(input_values=torch.from_numpy(input_values)).logits.cpu().numpy()
rows = []
for precision, model_path in (("fp32", sys.argv[1]), ("fp16", sys.argv[2])):
    session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
    specs = [{"name": i.name, "shape": list(i.shape), "dtype": i.type} for i in session.get_inputs()]
    assert specs == [{"name": "input_values", "shape": [1, SAMPLE_RATE], "dtype": "tensor(float)"}]
    onnx_logits = session.run(["logits"], {"input_values": input_values})[0]
    delta = onnx_logits.astype(np.float64) - pt_logits.astype(np.float64)
    rows.append({"ep": "cpu", "device": "cpu", "precision": precision,
                 "cosine_similarity": cosine(pt_logits, onnx_logits),
                 "max_abs_error": float(np.max(np.abs(delta))),
                 "mean_abs_error": float(np.mean(np.abs(delta))),
                 "top1_pytorch": int(np.argmax(pt_logits, axis=-1)[0]),
                 "top1_onnx": int(np.argmax(onnx_logits, axis=-1)[0]),
                 "top1_agreement": bool(np.argmax(pt_logits, axis=-1)[0] == np.argmax(onnx_logits, axis=-1)[0])})
payload = {"model_id": MODEL_ID, "revision": REVISION, "input": "deterministic 16 kHz mono PCM-equivalent mixed-tone/chirp", "results": rows}
Path(sys.argv[3]).write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
print(json.dumps(payload, indent=2))
'@
$L2 | uv run python - "$OUT/fp32/model.onnx" "$OUT/fp16/model.onnx" "$OUT/l2_numeric_comparison.json"
winml eval -m "$OUT/fp32/model.onnx" --model-id onecxi/open-vakgyata --task audio-classification --ep cpu --device cpu --precision fp32 --output "$OUT/eval_fp32.json" --overwrite --no-color
winml eval -m "$OUT/fp16/model.onnx" --model-id onecxi/open-vakgyata --task audio-classification --ep cpu --device cpu --precision fp16 --output "$OUT/eval_fp16.json" --overwrite --no-color

@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 18, 2026
@ssss141414

Copy link
Copy Markdown
Contributor Author

Independent reviewer verdict: REQUEST_CHANGES

Scope reviewed: only onecxi/open-vakgyata, PR #1131, head b67721d836041de1316707020eb361a53d46d3c2, against current origin/main e7509b1e908c74beff0a5655b8f8d7de69c5afae.

Required owner-routable changes

  1. Tester → Explainer: restore the contract-required baseline/config commands in the public body. The Baseline section reports the version, commit, recipe-free build, and config comparison, but neither it nor Reproduce commands cites the exact winml --version, git rev-parse HEAD, recipe-free winml build -m onecxi/open-vakgyata ... command, or the two winml config commands that generated the fp32/fp16 comparison inputs. Add the exact portable commands while retaining the pinned current-main result and field-by-field Delta.

  2. Tester → Explainer: make the headline L2 and analyze evidence durably/publicly reproducible. The highest claimed outcome is L2 PASS, but the public Reproduce commands contain no L2 harness or invocation; the only harness is internal scratch evidence and therefore cannot be run by a GitHub reader. Publish a durable self-contained harness or durable public artifact/URL and its exact invocation. Also fix the two public winml analyze commands: run exactly as written, they exit with Error: No runtime rule parquet files were found and produce no JSON, while the reported classifications were generated with an unmentioned external rules directory. Provide a publicly resolvable rules source/version plus portable setup/environment command so the published analyze commands reproduce the reported data. Do not expose a local workspace path.

  3. Learner: complete wav2vec2-003's mandatory artifact mining, in Lane A only. The finding records total operators and QNN partial types but omits the required top three op counts from analyze_result.json. Add Add=35, Mul=24, and Transpose=23 (for both artifacts), retain the actual analyzed EP set, refresh the learner handoff, and keep this knowledge append out of the Lane B model PR. Current Lane separation itself is correct.

  4. Explainer: use the exact methodology-evolution declaration. Outcome currently says “No new methodology finding was triggered”; the reviewer contract requires the explicit declaration “No methodology friction observed” when the no-friction path is selected.

Independently verified evidence

  • Conversation gate: enumerated 0 issue comments, 0 reviews, 0 line comments, and 0 GraphQL review threads; open threads: 0.
  • Shipment/scope: OPEN DRAFT; model-scale-by-skill present; merge base equals current origin/main; diff is exactly the two precision-suffixed recipe JSON files under examples/recipes/onecxi_open-vakgyata/cpu/cpu/; production recipe README and Python scope are unchanged.
  • Recipe semantics/JSON: both files parse; top-level schema is valid; fresh winml config outputs are semantically identical to their respective checked-in recipes. fp32 is identical to the starting auto-config; fp16 differs from that starting config only at /quant via the generated fp16 configuration.
  • Fresh detached-head L0 rerun: recipe-free baseline and both recipe builds printed Build complete. Both ONNX files pass full checker validation with IR 8/opset 17 and input_values FLOAT[1,16000] -> logits FLOAT[1,10]. fp32 has 88 FLOAT initializers and 234,981,376-byte colocated external data; fp16 has 88 FLOAT16 initializers and 117,490,688-byte colocated external data.
  • Fresh L1 rerun: 100-iteration CPU p50 was 41.646 ms fp32 and 49.198 ms fp16, within 20% of the reported 35.356/41.124 ms; memory and model precision fields agree.
  • Fresh L2 rerun: reproduced the exact reported cosines, max/mean errors, and top-1 agreement for both precisions.
  • Eval blocker: schema exits 2 and tuple runs exit 1 with the exact unsupported audio-classification evaluator error.
  • Analyze cross-check: with the same external rules source used internally, fresh fp32 analysis reproduced 163 ops / 15 types, dominant counts, full TensorRT/OpenVINO static classifications, QNN partial Div/Erf/Add/Mul, and all-unknown rule-less EP groups. The compact public summary agrees with exhaustive tester evidence; only its published command is not portable.
  • Lint/CI: uv run ruff check reports the same six pre-existing diagnostics on PR head and clean current main, all outside this JSON-only diff. All 9 GitHub checks are successful.
  • Public-body hygiene: hierarchy, frozen architecture tree, tuple matrices, compact component/op summaries, Draft state, and label are correct; no inaccessible local paths or uncommitted handoff hashes are exposed publicly.

This is fixable evidence/reporting and learner-capture work, not a structural recipe failure. Keep the PR in DRAFT after the loop-back.

@ssss141414

Copy link
Copy Markdown
Contributor Author

Resolved the explainer-facing REQUEST_CHANGES items without changing the Lane B recipe diff:

  • Tester → explainer: consumed refreshed verdict table 1ad5f4813ce2e919346b9106e18586523e3441f1a467b057396ff508178261be; restored portable version/HEAD, recipe-free baseline, and fp32/fp16 winml config commands; added the complete self-contained L2 reproduction; and replaced the local-rules analyze setup with the public v0.2.0 rule archive, SHA-256/extraction checks, and the exact exit-1/complete-JSON partial-success caveat.
  • Learner: refreshed wav2vec2-003 in Lane A only with Add=35, Mul=24, and Transpose=23 for both artifacts while retaining the analyzed EP set.
  • Explainer: updated the PR body in place, retained the required hierarchy and all seven evidence subsections, preserved the exact metrics and compact architecture/component/op summaries, and added No methodology friction observed.

The PR remains DRAFT with model-scale-by-skill; no self-approval or ready transition was performed.

@ssss141414

Copy link
Copy Markdown
Contributor Author

Independent reviewer verdict: APPROVE

Scope reviewed: only onecxi/open-vakgyata, PR #1131, head b67721d836041de1316707020eb361a53d46d3c2, against current origin/main e7509b1e908c74beff0a5655b8f8d7de69c5afae.

Remediation verification

  • Baseline/config reproduction — PASS. The live body now includes the exact portable winml --version, git rev-parse HEAD, detached current-main recipe-free build, and fp32/fp16 winml config commands. Independent reruns produced Build complete for the baseline and both recipes; both fresh generated configs are semantically identical to their checked-in precision recipes. fp32 remains identical to the starting auto-config; fp16 changes only /quant through WinML's generated fp16 configuration.
  • Durable L2 reproduction — PASS. I extracted the self-contained harness from the live PR body and ran it against fresh PR-head artifacts. It exactly reproduced fp32 cosine 0.9999999996964098, max/mean absolute error 0.0005121231079101562 / 0.00011473149061203003, and fp16 cosine 0.999999301707027, max/mean absolute error 0.015536665916442871 / 0.007226626574993134; both retained top-1 4/4.
  • Versioned analyze prerequisite and semantics — PASS. The public v0.2.0 archive downloaded successfully, SHA-256 matched 6dcabbe7f5493fbc2bd23de195ab6e35b3578d510f18c2e220bc5538a1f232cc, and extraction contained exactly 1,746 parquet files. Both published-style analyze reruns exited 1 because optional provider registration failed but emitted complete 11-row JSON, matching the body's honest partial-process-success caveat rather than claiming runtime execution.
  • Lane A learner correction — PASS. Refreshed wav2vec2-003 and its learner handoff retain the actual analyzed EP/device set and record Add=35, Mul=24, and Transpose=23 for both artifacts. Independent public-rules analysis reproduced fp32 163 ops / 15 types and fp16 165 ops / 16 types with those same dominant counts.
  • Methodology declaration — PASS. The live body contains the exact sentence No methodology friction observed. exactly once. No new methodology trigger is present: the reviewer-requested checks were already encoded by the current contract.

Full gate evidence

  • Conversation/review enumeration: 2 issue comments, 0 submitted reviews, 0 line comments, 0 GraphQL review threads; open threads: 0. The two comments are the prior structured REQUEST_CHANGES verdict and the remediation response.
  • Freshness/state: baseline commit equals current fetched origin/main; git rev-list --count e7509b1e...origin/main returned 0. PR is OPEN DRAFT, label model-scale-by-skill is present, and it was not promoted.
  • CI/lint: all 9 GitHub checks are successful. uv run ruff check reports the same six diagnostics on PR head and a clean current-main worktree, all in unchanged Python files; changed Python scope is empty.
  • Diff/scope: exactly two precision-suffixed JSON recipes under examples/recipes/onecxi_open-vakgyata/cpu/cpu/; no source/test/skill/production-README changes and git diff --check is clean.
  • Recipe/artifact validity: both JSON files satisfy the recipe schema and named-input contract input_values FLOAT[1,16000] -> logits FLOAT[1,10], IR 8 / opset 17. fp32 has 88 FLOAT initializers and 234,981,376 bytes of colocated external data; fp16 has 88 FLOAT16 initializers, preserved fp32 I/O, and 117,490,688 bytes of colocated external data.
  • L1: independent 100-iteration CPU reruns selected CPUExecutionProvider; p50 was 41.646 ms fp32 and 49.198 ms fp16, within 20% of the reported 35.356 ms / 41.124 ms. Precision and memory fields agree with the claims.
  • Eval blocker: schema exits 2 and both tuple evals exit 1 with the exact unsupported audio-classification evaluator error; this is correctly reported as supplementary CLI-BLOCKED, not a PASS or host defer.
  • Upstream model evidence: pinned revision f2754058e485dfc65cc62589b8d0e21c1d328399 still declares Wav2Vec2ForSequenceClassification, stable LayerNorm, two hidden layers, ten cited labels, normalized 16 kHz input, and audio-classification metadata. No random-head warning was observed.
  • Public/private separation: required compact metrics, commands, public rule source, and durable L2 harness are public; exhaustive node data and internal handoff paths/hashes remain private. The live body has no local workspace/scratch-path leakage.

APPROVE. Required CPU/cpu fp32 and fp16 tuples are freshly verified through Goal L2 with full coverage and no deferred tuples. Leave PR #1131 in DRAFT.

@ssss141414
ssss141414 marked this pull request as ready for review July 20, 2026 12:52
@ssss141414
ssss141414 requested a review from a team as a code owner July 20, 2026 12:52
@ssss141414
ssss141414 merged commit 38767ad into main Jul 20, 2026
9 checks passed
@ssss141414
ssss141414 deleted the ssss141414/add-onecxi-open-vakgyata-recipe branch July 20, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants