sync: whisper support in base-convert (open-core source)

[prabod]

Brings the Whisper HF-safetensors mapper across so the public converter can build the .base bundles the 0.2.1 engine transcribes. base-arch had no whisper.rs at all, so basert convert could not produce a Whisper bundle — even though converted Whisper bundles are already published for people to pull.

Scope

Converter only. The whisper C API and its bindings are deliberately not here: they call into libbaseRT, and the released 0.2.0 binary has no transcription entry points, so shipping those bindings now would break against the engine users actually have. They belong with the 0.2.1 release sync — same ordering as the 0.1.7 and 0.2.0 syncs.

Included

• crates/base-arch/src/whisper.rs — the mapper. All twelve variants (tiny…large-v3-turbo, incl. .en), tensor renaming, and special-token IDs read from each model's own tokenizer.json rather than inferred from vocabulary size — the inference is what mis-mapped large-v3's ids.
• crates/base-arch/tests/fixtures/whisper-tiny-added-tokens.json — the genuine added_tokens array trimmed to ids ≤ 50364. whisper.rs include_bytes!s this, so it is a build dependency, not just test data.
• base-arch/src/lib.rs — dispatch entry, arch list, and the encoder-decoder ArchConfig fields (encoder layers/heads/ffn, mel bins, source/target positions) plus their header emission.
• base-format/src/header.rs — the top-level metadata map the whisper token ids are written into; three test files updated for the new struct field.
• base-convert/src/main.rs — the whisper convert path.
• profiles/whisper-{f16,q8,q4}.json — see the judgement call below.
• The whisper case in base-quant/src/profile.rs's tests.

Excluded (unchanged from the #37 policy)

The tuned per-family quant profiles stay private. Concretely, profile.rs upstream also gained tests for gemma4-moe-mlx, moe-q4mix and q3-aggressive; those load profile files that are not published, and the tests resolve profiles from disk at run time, so bringing them would fail here on a missing file. Only the whisper test came across.

Also excluded as unrelated drift: base-arch/src/gemma.rs and base-convert/src/hub.rs.

One judgement call

The whisper profiles are included. They are architecture-enabling rather than a tuned recipe — they say which tensor classes may be quantized for this architecture (block linear projections only; the convolutional front end, positional and token embeddings, norms and biases stay f16), in the same spirit as the published default-* profiles. Without them the public converter could only emit f16, while quantized Whisper bundles are published — users could pull them but never reproduce them. Say the word if you'd rather hold them back.

Validation

In this tree:

• cargo build --locked --workspace ✓
• cargo test --workspace — 218 passed, 0 failed (13 whisper, including the real-tokenizer test, which runs only because the fixture is here)
• cargo clippy --all-targets --workspace -- -D warnings ✓ clean
• Every profile referenced anywhere in crates/ is present in profiles/

End to end, with the binary built from this branch:

• openai/whisper-tiny → f16 .base (167 tensors, 75 MB)
• openai/whisper-tiny → q8 .base via the shipped profile (59 MB)
• the f16 bundle transcribes jfk.wav: "And so my fellow Americans ask not what your country can do for you, ask what you can do for your country."