tests/omnivoice: add OmniVoice weight-type benchmark, report, and Python-to-C++ tests

[kawshikbuet17]

Summary

Follows up on #158, where you suggested opening a PR to add my OmniVoice test script under tests/omnivoice/.

This adds the voice-clone weight-type benchmark that produced the q8_0 audio-tokenizer finding I reported in that issue, plus two Python-to-C++ integration tests for OmniVoice.

What is here

Benchmarks — tests/omnivoice/run_clone_benchmark.sh and run_clone_batch_benchmark.sh sweep every combination of --num-inference-steps (8 to 64), omnivoice.generator_weight_type, and omnivoice.audio_tokenizer_weight_type, for single requests and for --batch-text-file runs. Shared Bengali fixtures live in tests/omnivoice/assets/.

Report — docs/reports/omnivoice_weight_type_benchmark.md holds the measurements: 375 single requests and 375 batch runs of 10 prompts on an RTX 4090.

Integrations — two ways to drive OmniVoice from Python with inference staying in C++:

• tests/omnivoice/python_cpp_simple/ posts OpenAI-compatible requests to audiocpp_server using only the Python standard library, with a matching audiocpp_cli baseline.
• tests/omnivoice/python_cpp_so/ is a pybind11 extension that calls the session directly, no server and no subprocess. The model loads once and is reused across generate() calls.

Docs — docs/models/omnivoice.md did not list omnivoice.generator_weight_type or omnivoice.audio_tokenizer_weight_type, so both are now in the options table.

Findings

• omnivoice.generator_weight_type=f16 averaged 1.26x faster than native (up to 1.31x at 64 steps) and was fastest at every step count.
• Runtime omnivoice.audio_tokenizer_weight_type=q8_0 produced unusable audio at every step count while being slower than both f16 and bf16. This is the issue Help Wanted: OmniVoice Batch Infererence #158 finding, and it is runtime quantization of SafeTensors weights — not the prebuilt Q8 GGUF package, which as you noted quantizes far more conservatively.
• Wall time is 90-98% generator diffusion and scales linearly with step count; audio tokenizer decode stays flat near 9-10 ms. Step count is the largest latency lever, with 16 steps running 1.9x faster than the default 32.
• Each --batch-text-file run of 10 prompts emitted 10 separate session.wall_ms values across all 375 batch combinations, matching what you described in Help Wanted: OmniVoice Batch Infererence #158: batched CLI input runs as sequential requests in one session. The 1.20x-1.36x per-request gain over an isolated single request comes from session and graph reuse, not batching — the first request in each batch is consistently the slowest.

Recommended pairing for this route:

--session-option omnivoice.generator_weight_type=f16 \
--session-option omnivoice.audio_tokenizer_weight_type=f16

Build and run

python3 tools/model_manager.py install omnivoice

cmake -S . -B build -DENGINE_ENABLE_CUDA=ON
cmake --build build --parallel --target audiocpp_cli audiocpp_server

./tests/omnivoice/run_clone_benchmark.sh
./tests/omnivoice/run_clone_batch_benchmark.sh

The pybind11 extension is opt-in behind a new ENGINE_BUILD_OMNIVOICE_PYTHON_BINDING CMake option that defaults to OFF, so existing builds are unaffected:

./tests/omnivoice/python_cpp_so/build.sh
python3 tests/omnivoice/python_cpp_so/test_so.py

build.sh passes -DENGINE_BUILD_OMNIVOICE_PYTHON_BINDING=ON rather than editing the root CMakeLists.txt. The built .so is gitignored.

Server path:

./tests/omnivoice/python_cpp_simple/run_server.sh      # terminal 1
python3 tests/omnivoice/python_cpp_simple/test_python.py   # terminal 2

Environment tested

Item	Value
Backend	CUDA, NVIDIA GeForce RTX 4090, 24 GiB
Model	models/OmniVoice SafeTensors
Python	3.12 for the pybind11 extension
Task	tts voice cloning, Bengali reference audio and text

Known limitations

• Single-request rows in the report are one sample each after 10 warmup runs; only the aggregate means (75 runs per weight type) average out run-to-run noise.
• The q8_0 audio-tokenizer failure is a listening judgement. The sweep WAVs were not retained, so the report does not attach a numeric similarity metric. Reproducing it means listening to the generated audio.
• CUDA only. No CPU, Vulkan, or Metal coverage.
• Measured on one GPU with one reference voice and one short prompt, so absolute times are not a general baseline.
• test_so_2.py keeps a multi_infer() path that is skipped: binding.cpp does not expose generate_batch(), since there is no fused batch entry point to bind yet.
• Raw sweep logs are not included here, since they are about 4.6 MB of CLI output. They are on my dev_kawshik branch: single inference, batch. The report tables are parsed directly from these two files.

Refs #158

[0xShug0]

@kawshikbuet17 Thanks for the PR! The test harness is heavier than I expected. Could you keep the benchmark report/docs and the lightweight python_cpp_simple, but drop python_cpp_so from this PR?

[kawshikbuet17]

@0xShug0 thanks, now please check.

[0xShug0]

Thanks @kawshikbuet17! Merged!