Custom Dictionary Support - Implemented but seeking discussion

[GrahamJenkins]

Hi, I recently wanted to implement custom dictionary support, allowing users to maintain a list of words/phrases that can be included in whisper's transcription. I have added support for this via a flat file dictionary file and wanted to see if there was any input before I wrap it up as a PR.

Implementation has two layers:

1. Dictionary support, a dictionary file located by default under .config/vocalinux/dictionary.txt. Simple flat file, one phrase per line. Read at the time of transcription so it can be dynamically updated anytime.
2. A separate accessibility service that does some primitive screen content recording, identifies non-dictionary words, and (after some simple logic/filtering) appends new words to dictionary.txt.

I'm not sure how the community feels about these topics, they are two sides of the same solution, but I think the accessibility service has wider implications than just identifying new words to insert into the database. I'm willing to share/include it if we wanted to built it into vocalinux, but imagine that it probably is cleaner to have the two interests separate.

Additionally, I have considered using a sqlite database rather than flat file, and/or some other storage backend.

I have both features/services running on my local system, currently under Ubuntu, I have not tested on other machines, and would want further cross-distro testing before being comfortable releasing the accessibility service. I think that the dictionary implementation should be fairly portable.

Any questions or input is welcome. I'm happy to push my local code for public review, but wanted to allow some review prior to opening an unsolicited PR.

I see a partially related request for dictionary support at #408

[jatinkrmalik]

@GrahamJenkins Thanks for putting this together. The dictionary approach sounds reasonable, and I am interested in seeing the code.

If you have something working locally, go ahead and open a PR. We can discuss the specifics there, look at the implementation, and figure out the best way to land it. The accessibility service piece is worth thinking about separately, but the dictionary itself is a good starting point.

Happy to review whenever you are ready.

[abishekmuthian]

Hey @GrahamJenkins , did you open the PR for dictionary?

[GrahamJenkins]

@jatinkrmalik @abishekmuthian Hi there, yeah, no, I meant to, I haven't created the PR, let me see if I can get that kicked over. I should probably make sure I can pull latest as my branch is fairly stale, stand by!

[GrahamJenkins]

Ok, PR is up at #767 Let me know if there's a better/cleaner direction and I can make some updates if needed.

[abishekmuthian]

Thanks @GrahamJenkins , I had also implemented custom dictionary so I've submitted a PR as well - #768 I will leave it to the maintainers and the community to decide which implementation is best suited.

Mine just adds the dictionary to the config.json , words can be added in the UI and the text is replaced during post-processing.

"text_injection": {
    "custom_dictionary": [
        {"spoken": "super base", "replacement": "Supabase"},
        {"spoken": "next door", "replacement": "Nextdoor"}
    ]
}

Works well for my use case so far.

[GrahamJenkins]

Personal functional observation between the two approaches:
@abishekmuthian Your approach implements a pretty UI, and seems intended for correcting mis-matched words.
My approach is intended to be paired with a secondary automated service (leveraging accessibility) to find words that aren't matched at all.

There may be a way that both approaches can be combined, to achieve:

• Pretty UI
• Ability to correct mis-transcribed words
• Support arbitrary list of words that aren't transcribed
• Friendly to external helper services

No personal bias here, I think both approaches have their own merits, I don't mind mine taking the bank burner if it's inferior, but a combination seems to offer the most utility. I'll review #768 and see if any other observations come up.

[abishekmuthian]

Thanks, I concur. Both are indeeded solving different problems and perhaps there's room for both the solutions.