Skip to content

feat(smallestai): add endpointing, keyword boosting, format, and sentence timestamps to Smallest STT - #6565

Merged
tinalenguyen merged 5 commits into
livekit:mainfrom
harshitajain165:feat/smallest-stt-endpointing
Jul 31, 2026
Merged

feat(smallestai): add endpointing, keyword boosting, format, and sentence timestamps to Smallest STT#6565
tinalenguyen merged 5 commits into
livekit:mainfrom
harshitajain165:feat/smallest-stt-endpointing

Conversation

@harshitajain165

@harshitajain165 harshitajain165 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds four Pulse STT streaming options newly documented by Smallest AI: endpointing (trailing-silence finalization, default True), format (punctuation/capitalization, default True), keywords (word-boost list, default off), and sentence_timestamps (sentence-level utterances surfaced via SpeechData.metadata, default off).
  • Adds redact_pii and redact_pci streaming options (default off), per the PII and PCI Redaction docs. redact_pii masks names, addresses, and phone numbers; redact_pci masks credit card numbers, CVVs, ZIP codes, and account numbers. Redacted placeholder tokens (e.g. [FIRSTNAME_1], [CREDITCARDCVV_1]) are surfaced via SpeechData.metadata["redacted_entities"]. Per the docs, redaction is only reliably supported for language="en" and "hi".
  • All six options are wired through STT.update_options() / SpeechStream.update_options() and trigger a reconnect like the existing options (eou_timeout_ms, diarize, etc).
  • Batch transcription now also surfaces utterances in SpeechData.metadata whenever word_timestamps is enabled (no separate flag needed there, per the API).

Test plan

  • uv run pytest tests/test_plugin_smallestai_stt.py --unit — 26/26 passing
  • ruff check / ruff format --check on changed files — clean
  • mypy --strict on stt.py — clean (one pre-existing, unrelated generic-type note on the STT class declaration)
  • Live end-to-end run against the real Pulse streaming API covering both states of every new option (endpointing, format, sentence_timestamps, keywords) individually, combined-enabled, combined-disabled, and a mid-stream update_options() reconnect — 17/17 checks passed

…ence timestamps to STT

Wires up four new Pulse STT streaming options: endpointing (trailing-silence
finalization, default true), format (punctuation/capitalization, default true),
keywords (word-boost list, default off), and sentence_timestamps (utterances
metadata, default off). All four flow through update_options() and trigger a
stream reconnect like the existing options.
@harshitajain165
harshitajain165 requested a review from a team as a code owner July 27, 2026 13:37

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@harshitajain165 harshitajain165 changed the title feat(smallestai): add endpointing, keyword boosting, format, and sentence timestamps to STT feat(smallestai): add endpointing, keyword boosting, format, and sentence timestamps to Smallest STT Jul 27, 2026
@harshitajain165

Copy link
Copy Markdown
Contributor Author

Hey @tinalenguyen 👋

Just added a few new streaming features to the Smallest AI STT plugin. Would love your eyes on it when you get a chance

harshitajain165 and others added 3 commits July 28, 2026 12:10
Wires up redact_pii and redact_pci streaming options (default false),
matching the format/sentence_timestamps pattern. Redacted entity
placeholders from the server surface via SpeechData.metadata["redacted_entities"].

@tinalenguyen tinalenguyen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR!

@tinalenguyen
tinalenguyen merged commit 394d916 into livekit:main Jul 31, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants