ElevenLabs: model-aware SpeechMarkdown dialects, modelId, voice_settings.speed - #34
Merged
Conversation
ElevenLabs previously routed SpeechMarkdown through the Alexa fallback and then stripped it to plain text in speak() — every break, tag and IPA hint was silently lost. Now: - preprocess_speech_markdown routes elevenlabs to the pre-v3 prompt dialect (<break time>) and elevenlabs-v3 to the audio-tag dialect ([pause], [whispers], "/IPA/"), returning is_ssml=false so the dialect text reaches the API verbatim instead of being stripped - speak() picks the dialect from the model: eleven_v3* (settable via the modelId credential, previously hardcoded to multilingual_v2) gets audio tags — v3 reads stray XML aloud, pre-v3 reads tags aloud, so the mapping must follow the model - rate (1.0 = normal) maps to the deterministic voice_settings.speed API parameter, clamped to the documented 0.7-1.2 range - ElevenLabs boundary offsets search the caller's original input when SpeechMarkdown was reformatted (injected tags shifted offsets) - live-cloud template: v3 audio-tag synthesis + timestamps tests, and fixed the stale template (speak() gained on_mark, boundary callbacks a bool) so it compiles again - [patch.crates-io] points speechmarkdown-rust at the sibling checkout until >= 0.4.14 (Platform::ElevenLabsV3) is published
…branch The relative-path patch only resolves in a sibling checkout; CI needs the pushed branch. Swap to a version bump after 0.4.14 is published.
…ecedence - extract elevenlabs_smd_platform() and assert against the production helper (the test previously checked a local copy that would keep passing if the real predicate flipped) - derived voice_settings no longer clobbers a config-supplied one (skipped when extra_body carries voice_settings; also inserted before the extra_body loop so config wins either way) - live template header lists the v3 dialect tests; README notes the 3s break clamp
v0.4.14 (published from the merged ElevenLabs PR) carries Platform::ElevenLabsV3 and the expressive/strict-break parser work — drop the temporary [patch.crates-io] git-branch override.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ElevenLabs previously routed SpeechMarkdown through the Alexa SSML fallback and then stripped it to plain text in
speak()— every break, tag and IPA hint was silently lost. This wires up the real dialects added in AACTools/speechmarkdown-rust#4:preprocess_speech_markdownroutes"elevenlabs"→ pre-v3 prompt markup (<break time>) and"elevenlabs-v3"→ audio-tag dialect ([pause],[whispers],"/IPA/"), returned withis_ssml=falseso the dialect text reaches the API verbatim instead of being strippedspeak()picks the dialect from the model:eleven_v3*→ audio tags. The model is now settable via themodelIdcredential (was hardcodedeleven_multilingual_v2). The mapping must follow the model — v3 reads stray XML aloud, pre-v3 reads audio tags aloudrate(1.0 = normal) maps to the deterministicvoice_settings.speedAPI parameter, clamped to the documented 0.7–1.2; only sent for explicit non-default ratesspeak()gainedon_mark, boundary callbacks abool).env.exampledocumentsELEVENLABS_MODEL_ID[patch.crates-io]points speechmarkdown-rust at thefeat/elevenlabs-dialectsgit branch (relative-path patch can't resolve in CI); swap to a version bump after 0.4.14 publishesTest plan
--no-default-features --features cloud: 134 lib + integration tests)cargo clippyclean (pedantic lints)ELEVENLABS_API_KEY)