Skip to content

Enable Parakeet V3 download and full Intel NPU execution - #13

Merged
Alex-Wengg merged 1 commit into
FluidInference:mainfrom
derekszen:fix/parakeet-v3-npu-upstream
Jul 23, 2026
Merged

Enable Parakeet V3 download and full Intel NPU execution#13
Alex-Wengg merged 1 commit into
FluidInference:mainfrom
derekszen:fix/parakeet-v3-npu-upstream

Conversation

@derekszen

Copy link
Copy Markdown
Contributor

Closes #12

Summary

Enable Parakeet V3 download and full Intel NPU execution by fixing two blockers:

  • Point PARAKEET_V3 at the published FluidInference/parakeet-tdt-0.6b-v3-ov export instead of the missing 1.1b-v3-ov repository.
  • Normalize negative opset5 LogSoftmax axes before compiling OpenVINO IR models on Intel NPU.

The joint graph uses axis=-1 on a static rank-4 tensor. This is valid OpenVINO and semantically identical to axis=3, but Intel's NPU compiler fails in AlignDimensionsForDPU, causing Eddy to run the joint component through its CPU fallback.

Implementation

  • Correct the Parakeet V3 Hugging Face repository ID.
  • Correct the README model-card link to the same published repository.
  • Read non-blob IR models into memory when the requested device is exactly NPU.
  • Visit opset5 LogSoftmax nodes with a negative axis and static input rank.
  • Convert valid negative axes to their canonical non-negative value.
  • Revalidate the graph only when a node changed.
  • Leave CPU/GPU/AUTO compilation and compiled-blob imports unchanged.
  • Leave dynamic-rank or out-of-range axes untouched so the existing compile failure and CPU fallback remain authoritative.

This follows the existing in-memory NPU graph-rewrite precedent in the Nemotron backend while keeping the workaround narrowly scoped to the affected operator and device.

Reproduction before the fix

Environment:

  • Intel Panther Lake NPU5010
  • Intel NPU driver 1.32.1
  • OpenVINO 2026.2.1
  • FluidInference/parakeet-tdt-0.6b-v3-ov
AlignDimensionsForDPU Pass failed : Got negative index -1 for Dim

The failing node is opset5 LogSoftmax logits, input shape [1, 1, 1, 8198], axis=-1.

Validation

  • Built all Eddy targets successfully on Linux with OpenVINO 2026.2.1.
  • Added openvino_utils_test with five hardware-independent cases: -1 -> 3 with CPU output equivalence, -4 -> 0, positive-axis no-op, dynamic-rank no-op, and non-LogSoftmax no-op.
  • CTest: 1/1 passed.
  • Confirmed the Parakeet V3 joint graph compiles and runs on NPU on an Intel Panther Lake NPU5010 with driver 1.32.1.
  • Full 14.24-second sample: exact 83-token match against CPU and the previous hybrid path.
  • Five speech slices: exact CPU/NPU transcript and token-count matches.
  • Ten-second silence: empty on both CPU and NPU.
  • Ten randomized joint inputs: identical winning token and duration heads on CPU and NPU.
  • Median model inference: 196 ms with CPU joint fallback, 116 ms all-NPU, about 41% lower latency.

Scope

This does not modify model files on disk or broaden NPU fallback behavior. The graph workaround only canonicalizes a semantically equivalent axis representation before NPU compilation.

@Alex-Wengg
Alex-Wengg merged commit 07028cf into FluidInference:main Jul 23, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parakeet V3 uses missing model repo and falls back from Intel NPU

2 participants