Skip to content

1.2.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Sep 17:37
· 38 commits to main since this release
1.2.1
71d11cb

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

Assets 4
Loading