Optional live vocabulary provider for context-aware dictation

[nukeador]

Problem

Handy's Custom Words are useful for persistent vocabulary, but relevant terms can change while dictating: people's names, locations, project identifiers or specialised words associated with the current task.

External tools may already know that context, but I haven't found a supported way to supply temporary vocabulary before each transcription without changing Handy or editing its persistent settings.

This proposal is for future consideration and community feedback; I understand Handy is currently under a feature freeze.

Proposed solution

An optional external vocabulary provider that Handy queries immediately before transcription. A small initial implementation could read a local JSON file; a local socket or external command could be considered if a better fit.

Illustrative interface, not a prescribed schema:

{
  "version": 1,
  "expires_at": "2026-10-02T14:30:15Z",
  "vocabulary": ["Thalendir", "Bashal'Aran", "Moonwell"]
}

The terms would supplement permanent Custom Words without changing them and apply only to the current transcription. Expired, unavailable or malformed input should be ignored. File size, term count and any provider timeout should be bounded so failure never blocks normal dictation.

Vocabulary should remain independent of the selected speech language: English proper names can still be useful inside Spanish speech. How hints are applied would depend on the recognition engine; this does not assume every engine supports Whisper-style prompts.

Concrete example: live game vocabulary

I'm developing WoW Context Bridge, an experimental addon exposing player, NPC and location names through an on-screen pixel protocol. An external reader decodes the context locally; an experimental Decktation integration supplies fresh names as recognition hints.

A generic provider would let the same reader supply Handy with relevant names as the player explores, without putting WoW-specific logic in Handy. The addon/reader would remain a separate opt-in tool. This is an example, not a claim that Handy integration already exists.

Other potential uses

• Meetings: participant names and terminology from the current agenda.
• Software development: identifiers, dependencies and project names from an active workspace.
• Document editing: proper nouns and specialised terms from the current document.
• Customer support: product names and terminology from the active ticket.
• Accessibility: names and labels from the application the user is interacting with.

These would be independent providers. Handy would not need to collect context or inspect the screen itself.

Alternatives considered

• Manually updating Custom Words: useful for static vocabulary, but cumbersome for changing context.
• Exporting terms into the settings file and restarting: provides a snapshot but interrupts normal use and persists temporary names.
• A custom Handy fork: possible, but duplicates maintenance for an interface useful to multiple tools.
• Correcting the transcript afterwards: may help, but differs from giving the recognition engine hints during decoding and risks replacing the wrong words.

Related work: Whisper custom-word prompting #1035 and bulk import #1321. This proposal concerns temporary, expiring live vocabulary rather than importing a permanent dictionary.

Fit with Handy's philosophy and privacy

A small, generic local interface could support extensibility and accessibility while keeping application-specific logic outside Handy. It should be disabled by default, require explicit provider configuration, and avoid logging vocabulary or retaining it in permanent settings. Providers may expose sensitive names, so context should remain local unless a user separately chooses otherwise.

The initial scope could be vocabulary only: no arbitrary instructions, remote service or automatic screen inspection. Recognition benefits would need testing for each supported engine.

Would this fit Handy's direction? I'd welcome feedback on the smallest useful interface and whether others have similar live-context use cases.

[cjpais]

The settings file is already the json you're looking for?

[nukeador]

Can that file be modified externally while Handy is running and changes will be picked up in the next dicatation without having to close the app?

[cjpais]

give it a try, there have been reports for some settings it doesnt like it, but best way to find out exactly is try

for what its worth this is not something i plan to officially support at the moment. in the future we may do something over ipc