Skip to content

Releases: mpconley/MudletSTT

1.6.0

Choose a tag to compare

@github-actions github-actions released this 21 Sep 22:38
4cfe721

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Leaving does not send: when listening stops because you switched to
    another profile or another application, a phrase you were partway through
    is thrown away rather than sent as a command. Needs a Mudlet with
    stt.cancel(); older ones can only stop, which sends it.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt test game [n]      phrases drawn from this game's own catalog
stt test scope [n]     phrases naming what is in this room and inventory
stt test phrases <a>; <b>; ... [n]   an explicit list, for one comparison
stt test repeat [n]    the last built or given list again
stt vocab [n|all]      grade the game's catalog for what can be said
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Ask one question at a time. stt test phrases score guild; guild score 3
runs exactly those two phrases three times each, so a server author deciding
between two spellings of a command measures that and nothing else. Load a
model before any of this - biasing needs a decoder to bias, and asked of an
engine with nothing loaded the command answers that no model is loaded. Turn
correction off (stt correct off) when the recogniser alone is the question,
and run the list once with stt bias on and once with stt bias off via stt test repeat.

Confirm the switch actually took effect before the second run, and read the
reply to stt bias itself to do it. It answers biasing on (N words) when the
decoder took them, and takes effect at the next model load when it did not -
the engine rebuilds its decoder to change what it biases toward and cannot
while it is listening, so it keeps the request and carries on with the old word
list. Stop listening and ask again. stt status does not settle this: it
reports the setting you asked for, not the words the decoder holds. Two runs
compared without that check can be the same condition twice, which is the
mistake that forced a measurement to be retracted once already.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.5.1

Choose a tag to compare

@github-actions github-actions released this 19 Sep 03:02
abd9c20

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt test game [n]      phrases drawn from this game's own catalog
stt test scope [n]     phrases naming what is in this room and inventory
stt test phrases <a>; <b>; ... [n]   an explicit list, for one comparison
stt test repeat [n]    the last built or given list again
stt vocab [n|all]      grade the game's catalog for what can be said
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Ask one question at a time. stt test phrases score guild; guild score 3
runs exactly those two phrases three times each, so a server author deciding
between two spellings of a command measures that and nothing else. Load a
model before any of this - biasing needs a decoder to bias, and asked of an
engine with nothing loaded the command answers that no model is loaded. Turn
correction off (stt correct off) when the recogniser alone is the question,
and run the list once with stt bias on and once with stt bias off via stt test repeat.

Confirm the switch actually took effect before the second run, and read the
reply to stt bias itself to do it. It answers biasing on (N words) when the
decoder took them, and takes effect at the next model load when it did not -
the engine rebuilds its decoder to change what it biases toward and cannot
while it is listening, so it keeps the request and carries on with the old word
list. Stop listening and ask again. stt status does not settle this: it
reports the setting you asked for, not the words the decoder holds. Two runs
compared without that check can be the same condition twice, which is the
mistake that forced a measurement to be retracted once already.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.5.0

Choose a tag to compare

@github-actions github-actions released this 15 Sep 23:56
08845d2

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt test game [n]      phrases drawn from this game's own catalog
stt test scope [n]     phrases naming what is in this room and inventory
stt test phrases <a>; <b>; ... [n]   an explicit list, for one comparison
stt test repeat [n]    the last built or given list again
stt vocab [n|all]      grade the game's catalog for what can be said
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Ask one question at a time. stt test phrases score guild; guild score 3
runs exactly those two phrases three times each, so a server author deciding
between two spellings of a command measures that and nothing else. Turn
correction off first (stt correct off) when the recogniser alone is the
question, and run the list once with stt bias on and once with stt bias off via stt test repeat. Confirm the switch actually took effect before the
second run - stt bias on|off can answer deferred, meaning the engine kept
the request for its next model load and is still decoding with the old word
list, and stt status reports how many words the engine has really taken. Two
runs compared without that check can be the same condition twice, which is the
mistake that forced a measurement to be retracted once already.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.4.1

Choose a tag to compare

@github-actions github-actions released this 08 Sep 11:21

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.4.0

Choose a tag to compare

@github-actions github-actions released this 07 Sep 23:21

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.3.3

Choose a tag to compare

@github-actions github-actions released this 07 Sep 23:17

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.3.2

Choose a tag to compare

@github-actions github-actions released this 07 Sep 23:06

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.3.1

Choose a tag to compare

@github-actions github-actions released this 07 Sep 22:48

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.3.0

Choose a tag to compare

@github-actions github-actions released this 07 Sep 22:32

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.

1.2.10

Choose a tag to compare

@github-actions github-actions released this 07 Sep 22:12

MudletSTT

Speech-to-text for Mudlet, built on the stt.*
speech bridge. Toggle the microphone, speak, and your words land in the
command line ready to send — corrected against the game's own vocabulary
when the game publishes one.

What it does

  • Toggle listening with the stt alias, or the microphone command on
    Mudlet builds that provide the addon command API - it appears both on the
    main toolbar and under Options -> Extensions -> Speech, so it is reachable
    even with the toolbar hidden, which is how a fresh profile starts.
  • Command-line routing: partial results preview live in the command
    line while you speak; each finished utterance replaces the line, ready
    for Return. stt autosend on sends utterances straight to the game
    instead.
  • Vocabulary correction: when the
    MudletMCVP package is installed
    and the game implements the
    MUD Client Vocabulary Protocol,
    recognised words are corrected against the game's command and target
    vocabulary. Corrections are conservative: only a unique, close match
    within a length-scaled edit distance replaces a word.
  • Typed the way you'd type it: recognisers that produce natural prose
    sentence-case their output ("Smile"), so the first letter is lowercased to
    match how commands are written. Only the first letter — proper nouns in
    arguments keep their case.
  • Tuned for commands, not dictation: sensitivity defaults to short, so a
    one-word command like look commits after a brief pause instead of waiting
    out a dictation-length silence. Engines that support it are also biased
    against dropping the quiet start of a phrase, which is where the verb lives.
  • Engine aware: prefers an installed sherpa-onnx streaming model
    (NVIDIA Nemotron - high accuracy, hands-free endpointing) and falls back
    to Vosk. Models install separately via engine packs.

Checking a live session

tools/mcvp-integration-pass.lua is not part of the package - it is a script
to paste into a connected profile when you want evidence rather than an
impression. _pass.check() walks the chain and names the layer that failed:
packages loaded, catalog received and its version, the biasing budget against
the server's cap, whether a context adapter is bound and what is in reach,
the engine and model, and finally how many words the engine actually took.

_pass.compare() speaks the same phrases twice, once with biasing and once
without, and prints the two scores side by side. It draws the order at random
so practice does not favour whichever ran second, reuses one phrase list so
moving between runs cannot skew it, and refuses to read anything into two
runs that did not arrive at comparable volume.

Commands

stt              toggle listening (also: stt on / stt off)
stt status       engine, state and settings
stt autosend on|off    send finals to the game instead of the command line
stt preview on|off     show partial results live in the command line
stt correct on|off     correct finals against the game vocabulary (MCVP)
stt lowercase on|off   lowercase the first letter, the way commands are typed
stt sensitivity short|default|long   how soon a phrase counts as finished
stt timeout <ms>       stop after this much silence; 0 keeps listening
stt focus stop|keep    whether to stop listening when Mudlet loses focus
stt test [n]     score recognition against set phrases, n passes (stt test stop)
stt model <name> load a different installed model, to compare them
stt models       list installed speech models

Measuring recognition quality

stt test prompts with a set of phrases, listens for each, and scores what
came back. Nothing is sent to the game during a run, so it is safe while
connected. The report gives three numbers worth comparing between settings:

  • exact — phrases that matched outright.
  • word error rate — substitutions, insertions and deletions over all
    expected words.
  • first word lost — tracked separately, because a command's first word is
    its verb: losing it costs the whole phrase however well the rest was heard.

Each report names the engine, model, sensitivity and how many vocabulary
words correction actually had, so results from different settings can be
compared honestly.

Run more than one pass. Repeated runs of the same ten phrases at identical
settings have scored anywhere from 50% to 80% exact, so a single run cannot
tell a real difference from the spread. stt test 3 runs three passes and
reports the total; the failure list at the end shows how consistently each
phrase failed, which is what separates a fault worth chasing from ordinary
variance.

Events

Event Arguments When
sttPackageResult corrected text, raw text Each finished utterance, after correction.
sttPackageState state string The recogniser changed state.

Other packages can consume these instead of the raw sysSTT* events to get
correction for free.

Requirements

  • A Mudlet build with the stt.* speech bridge.
  • An installed speech model (sherpa-onnx or Vosk). The package tells you
    where models belong (stt.getModelPath()) when none is found.
  • Optional: MudletMCVP + a game that implements Client.Vocabulary, for
    correction.

Development

Scripts live under src/scripts/STT/; the correction engine
(STTCorrect.lua) and the test scoring (STTTest.lua) are pure Lua and
tested with busted: busted spec.
Builds use muddler.

Run luacheck src spec before building. These scripts run inside Mudlet, so
a call to something that does not exist raises nothing until that line
executes — which in this package means during a live microphone test.
luacheck catches it in a second instead.