Skip to content

feat(desktop): add pitch-preserving TTS playback speed - #3194

Draft
johnmatthewtennant wants to merge 27 commits into
jtennant/pocket-tts-settings-v1from
jtennant/tts-playback-speed
Draft

feat(desktop): add pitch-preserving TTS playback speed#3194
johnmatthewtennant wants to merge 27 commits into
jtennant/pocket-tts-settings-v1from
jtennant/tts-playback-speed

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Context

Generated speech currently plays at a fixed 1x rate. Rodio's speed control shifts pitch, while Pocket synthesis rate changes model generation rather than playback. This keeps playback speed independent from Pocket-only voice settings and preserves the existing per-chunk streaming path.

Summary

Adds a locally persisted global 0.75x to 1.50x generated-speech playback setting, with a 1x reset and pitch-preserving Signalsmith time stretching. The setting layers into Voice settings while remaining separate from Pocket voice selection and synthesis.

Voice settings playback speed

Changes

  • Adds a 1x bypass and pitch-preserving Signalsmith processor for each complete Pocket playback chunk before rodio append.
  • Preserves the fixed 20 ms device lead-in outside time stretching, keeps one contiguous append per chunk, and retains the existing post-DSP barge-in and voice-change cancellation check.
  • Keeps Pocket time-to-first-playback streaming: the first synthesized chunk plays without waiting for the full response.
  • Adds global local persistence with a conservative 0.75x to 1.50x range, serialized native writes, cross-platform atomic replacement, initialization-race protection, intent coalescing, and Reset to 1x.
  • Places the control in Voice settings independently of Pocket enablement and voice selection, including preview playback and the desktop E2E bridge.
  • Adds DSP pitch, order, onset, tail, and latency tests, production lead-in and constructor integration tests, UI persistence-race tests, and an end-to-end processing benchmark.
  • Adds ssstretch 0.1.0 (Signalsmith Stretch), MIT licensed. It is bundled at build time; normal use performs no download.

Related issue

N/A. No related issue was supplied.

Testing

  • Normal pre-push hooks on the integrated head passed: branch-skew, Desktop static guards, 3,736 Desktop tests, 824 Mobile tests with 1 skipped, all root Rust unit suites, and full Desktop Tauri tests.
  • Integrated-head Desktop Tauri result: 1,833 unit tests and 3 diagnostics passed, 14 ignored, 0 failed.
  • Focused Rust playback tests: 8 passed; Settings voice transition and selection: 8 passed; Settings constructor: 1 passed.
  • just desktop-tauri-clippy: passed with warnings denied.
  • just desktop-test: 3,733 passed on the reviewed pre-restack head. The integrated pre-push run passed 3,736.
  • just desktop-check: passed, including file-size, px-text, and pubkey-truncation guards.
  • just desktop-typecheck: passed.
  • cargo deny check licenses: passed. ssstretch and Signalsmith are MIT licensed; warnings are existing unmatched allowances for MITNFA, OpenSSL, Unicode-DFS-2016, and bzip2-1.0.6.
  • Fresh E2E build plus pnpm exec playwright test tests/e2e/voice-settings.spec.ts --project=smoke: 2 of 2 passed in 3.8 seconds.
  • Full Builderbot, Kalvin, and minimize-diff review completed. Selective post-base Builderbot and minimize reviews found zero findings, and the prior final Kalvin rerun was clean. No findings remain.
  • Remaining manual validation: launch the real Tauri app; open Settings, then Voice; listen to Mary and Marius preview and active-huddle speech at 0.75x, 1.25x, and 1.50x; verify pitch, first phoneme, and tail; interrupt speech to verify immediate cancellation; restart to verify persistence; use Reset and confirm 1.00x. This has not been performed on the headless machine.

Reviewer-reproducible examples

Run the exact production-helper benchmark:

cargo run --release --manifest-path desktop/src-tauri/Cargo.toml --example tts_speed_probe

Observed output on the final DSP implementation:

0.75x: processing=20.63ms for 10s (0.206% realtime), output=320480, compensated lookahead=60.0ms
1.25x: processing=12.29ms for 10s (0.123% realtime), output=192480, compensated lookahead=60.0ms
1.50x: processing=11.56ms for 10s (0.116% realtime), output=160480, compensated lookahead=60.0ms

The 60 ms value is Signalsmith's compensated internal lookahead, not added audible delay; the measured end-to-end processing wall time is shown separately.

Run focused behavior checks:

cargo test --manifest-path desktop/src-tauri/Cargo.toml playback_speed
cargo test --manifest-path desktop/src-tauri/Cargo.toml voice_selection_tests

johnmatthewtennant added a commit that referenced this pull request Jul 27, 2026
@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

🤖 ### Speech playback speed

The global generated-speech control appears in Appearance, defaults to 1.00x, and exposes a reset action independently of Pocket voice settings.

Speech playback speed

@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 27, 2026 19:47
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner July 27, 2026 19:47
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft July 27, 2026 19:53
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from f62db77 to 8de8b63 Compare July 28, 2026 00:45
@johnmatthewtennant
johnmatthewtennant changed the base branch from main to jtennant/pocket-tts-settings-v1 July 28, 2026 00:45
johnmatthewtennant added a commit that referenced this pull request Jul 28, 2026
@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

🤖 ### Voice settings

Global pitch-preserving playback speed is layered beside the Pocket voice controls.

pocket-voices

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 5a78f33 to 6fca35f Compare July 28, 2026 02:10
johnmatthewtennant and others added 10 commits July 27, 2026 23:51
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from 4891a92 to c118f5a Compare July 28, 2026 04:48
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 6fca35f to f78b065 Compare July 28, 2026 05:12
johnmatthewtennant and others added 2 commits July 28, 2026 01:31
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>

johnmatthewtennant commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

🤖 New review-intent sweep completed on the frozen stacked candidate.

  • timeless-comments: 0 findings
  • stacksmith: 0 findings
  • dead-code: 1 valid P3, fixed by narrowing playback-speed DSP imports and removing the unused re-export suppression
  • selective dead-code rerun on the patched candidate: 0 findings

Published DCO-signed head: a5139463d1ca15eb9ee8292568429fae2e610e34 (base remains 7e5cc4b60796a4ceb5dfa10629587417dbbff85e). Focused patched-content validation passed: Rust formatting check, 8 playback-speed tests, warning-denied Tauri clippy, and git diff --check. GitHub CI finished green with 18 passing and 7 skipped checks. PR remains open and draft; no readiness or merge action was taken.

Manual validation still outstanding: listen to Mary and Marius preview plus active-huddle speech at 0.75x, 1.25x, and 1.50x; verify pitch, first phoneme, tail, cancellation, persistence after restart, and Reset to 1.00x.

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch 2 times, most recently from 7cc0804 to acb302e Compare July 29, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant