test(stt): live parakeet http-remote integration tests by Coldaine · Pull Request #410 · Coldaine/ColdVox

Coldaine · 2026-04-20T14:14:49Z

Summary

Closes the test coverage gaps surfaced in the PR #409 review.

Before this PR, the only real behavioral coverage of the canonical HTTP-remote Parakeet path was the TCP-stub test inside `crates/coldvox-stt/src/plugins/http_remote.rs`. No test exercised the `Settings → runtime_plugin_selection → runtime_http_remote_config → HttpRemotePlugin` wiring, and the Windows Preflight check never touched the Rust plugin at all (it shelled out to curl).

This PR adds two `#[ignore]`-gated live integration tests that hit the real `parakeet-cpu` container and assert the exact transcript for `test_1.wav`:

`crates/coldvox-stt/tests/http_remote_live.rs` — plugin-level: `HttpRemoteConfig::canonical_parakeet_cpu()` → real container → `"On august twenty seventh, eighteen thirty seven, she writes."`
`crates/app/tests/http_remote_wiring_live.rs` — wiring: `Settings::from_path("config/windows-parakeet.toml")` → `runtime_plugin_selection()` → `runtime_http_remote_config()` → `HttpRemotePlugin::new()` → real container → same exact transcript assertion.

Both tests were regression-proven during authoring:

Mutating `api_path` to `/v1/BOGUS_REGRESSION_CHECK` produced `404 Not Found` and the plugin test FAILED as required. Reverted.
Mutating `preferred = "mock"` in `config/windows-parakeet.toml` produced `left: Some("mock"), right: Some("http-remote")` at the assertion before any network call, and the wiring test FAILED as required. Reverted.

A `just integration-parakeet` recipe + `scripts/integration_parakeet.ps1` bring up the container, wait for `/health`, and run both tests.

What this catches that existing tests don't

`api_path` drift (plugin URL path)
`model_name` drift (canonical model name)
Response parser regressions (exact-text equality, not substring)
`runtime_http_remote_config()` field drift (base_url, api_path, health_path, model_name)
Config → plugin wiring breakage — fails at the assertion step, proven

Not in scope (follow-up issues will be filed)

Linux/macOS `.sh` runner counterpart to `integration_parakeet.ps1`
`docs/domains/foundation/fdn-testing-guide.md` entry categorizing these as Level 3 external-service tests
GHA CI job on a Docker-capable runner running `just integration-parakeet`
Broader uncovered surface: chunked `process_audio`, double-`finalize`, non-200 / malformed JSON, bearer auth, actual `main` bootstrap config discovery
Shared `repo_root()` helper if more live tests land

Test plan

`just integration-parakeet` green on Windows against docker-compose parakeet-cpu
Mutation test: `api_path` → 404 → plugin test FAIL (reverted)
Mutation test: `preferred = mock` → assertion FAIL before network (reverted)
Same on Linux (requires `.sh` counterpart — tracked as follow-up)

Closes the test coverage gaps identified in the PR #409 review: the only real plugin-level coverage was the TCP stub test in coldvox-stt, and no test exercised the Settings -> runtime_plugin_selection -> runtime_http_remote_config -> HttpRemotePlugin wiring path. Adds two #[ignore]-gated live integration tests that assert the exact transcript for test_1.wav against the real parakeet-cpu container: - crates/coldvox-stt/tests/http_remote_live.rs covers the plugin alone. - crates/app/tests/http_remote_wiring_live.rs covers the full settings -> plugin wiring starting from config/windows-parakeet.toml. Both tests were regression-proven: mutating api_path caused a 404 that the plugin test caught, and mutating preferred to "mock" caused the wiring test to fail on the assertion before any network call. Adds scripts/integration_parakeet.ps1 and a just integration-parakeet recipe that brings up ops/parakeet/docker-compose.yml parakeet-cpu, waits for /health, and runs both tests with -- --ignored. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

coderabbitai · 2026-04-20T14:14:58Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: af0df1ef-ac1f-4543-8fe8-6880261902ab

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/parakeet-http-remote-live-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

Adds ignored-by-default live integration coverage for the canonical Parakeet HTTP-remote path, including both plugin behavior and app settings → plugin wiring, plus a local runner to bring up the docker fixture.

Changes:

Add a plugin-level live integration test that exercises HttpRemoteConfig::canonical_parakeet_cpu() against the real parakeet-cpu container and asserts an exact transcript.
Add an app-level live wiring test that exercises Settings -> runtime_plugin_selection -> runtime_http_remote_config -> HttpRemotePlugin against the same container and asserts the same transcript.
Add a just integration-parakeet workflow and PowerShell script to start the container, wait for /health, and run both ignored tests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
scripts/integration_parakeet.ps1	Brings up `parakeet-cpu`, waits for `/health`, and runs the ignored live tests.
justfile	Adds `integration-parakeet` recipe to invoke the runner script.
crates/coldvox-stt/tests/http_remote_live.rs	New ignored live test validating the canonical http-remote plugin transcript end-to-end.
crates/app/tests/http_remote_wiring_live.rs	New ignored live test validating Settings→plugin wiring and transcript correctness.
crates/app/Cargo.toml	Registers the new app integration test with `required-features = ["http-remote"]`.

Copilot · 2026-04-20T14:21:49Z

+    let config_path = repo_root().join("config/windows-parakeet.toml");
+    let settings = Settings::from_path(&config_path).expect("load settings");
+
+    // Step 1: the selection resolves to the canonical http-remote plugin.
+    let selection = settings
+        .runtime_plugin_selection()
+        .expect("runtime plugin selection must succeed");


Settings::from_path() merges COLDVOX__... environment overrides (and COLDVOX_PLUGIN_CONFIG_PATH / COLDVOX_CONFIG_PATH can affect plugin selection). This makes the live wiring test dependent on the caller’s environment and can cause hard-to-diagnose failures when running just integration-parakeet. Consider saving+clearing the relevant COLDVOX* env vars at the start of the test (then restoring in a drop/finally guard) so the assertions reflect only the checked-in config.

…gin + restore live integration tests (#422) * fix: tighten windows validation corrections * fix: address review feedback on windows corrections * fix: improve parakeet discovery and slim agent entrypoint * fix: tighten parakeet discovery follow-up * fix(windows): wire windows-parakeet.toml to canonical http-remote plugin + integration tests The Windows live override file selected preferred="parakeet" (local CUDA via parakeet-rs), but the canonical Windows path documented in docs/windows-live-runbook.md and exercised by ops/parakeet/docker-compose.yml is the OpenAI-compatible HTTP container at http://localhost:5092. The wiring live integration test windows_parakeet_settings_wire_through_to_real_transcript caught the mismatch immediately. Changes: - config/windows-parakeet.toml: preferred = "http-remote", require_local = false, expanded comments documenting the docker-compose bring-up flow. - Cherry-picks the live integration tests + integration_parakeet.ps1 from feat/parakeet-http-remote-live-tests (originally PR #410, whose test files did not actually land in the squashed merge): * crates/coldvox-stt/tests/http_remote_live.rs (plugin-level) * crates/app/tests/http_remote_wiring_live.rs (Settings -> plugin wiring) * scripts/integration_parakeet.ps1 + just integration-parakeet recipe Validation (this commit, on Windows 11 + RTX 5090 + Docker Desktop 29.2.1): docker compose -f ops/parakeet/docker-compose.yml up -d parakeet-cpu -> healthy on :5092 cargo test -p coldvox-stt --features http-remote --test http_remote_live -- --ignored -> 1 passed cargo test -p coldvox-app --features http-remote --test http_remote_wiring_live -- --ignored -> 1 passed * fix(test): preserve last health probe error in integration_parakeet Address CodeRabbit review on PR #422 by capturing the last health probe exception during the wait loop and surfacing it in the timeout message.

Copilot AI review requested due to automatic review settings April 20, 2026 14:14

github-actions Bot merged commit e18bffe into tauri-base Apr 20, 2026
9 of 21 checks passed

github-actions Bot deleted the feat/parakeet-http-remote-live-tests branch April 20, 2026 14:15

This was referenced Apr 20, 2026

test(stt): Linux/macOS runner for live parakeet integration tests #411

Open

docs(testing): categorize live parakeet integration tests in fdn-testing-guide #412

Open

Copilot started reviewing on behalf of Coldaine April 20, 2026 14:15 View session

This was referenced Apr 20, 2026

ci: GHA job running live parakeet http-remote integration tests #413

Open

test(stt): broaden http-remote plugin test surface #414

Open

test: shared repo_root() helper for live integration tests #415

Open

Copilot AI reviewed Apr 20, 2026

View reviewed changes

Coldaine mentioned this pull request Apr 27, 2026

fix(windows): wire windows-parakeet.toml to canonical http-remote plugin + restore live integration tests #422

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(stt): live parakeet http-remote integration tests#410

test(stt): live parakeet http-remote integration tests#410
github-actions[bot] merged 1 commit into
tauri-basefrom
feat/parakeet-http-remote-live-tests

Coldaine commented Apr 20, 2026

Uh oh!

coderabbitai Bot commented Apr 20, 2026

Review skipped

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Coldaine commented Apr 20, 2026

Summary

What this catches that existing tests don't

Not in scope (follow-up issues will be filed)

Test plan

Uh oh!

coderabbitai Bot commented Apr 20, 2026

Review skipped

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants