Multiple keywords in Snowboy universal wake word model causes error/core dump

[prabbit237]

Some universal Snowboy models have multiple hotwords and will cause the following error (or at least one does. Jarvis was encoded with both "jarvis" and "Jarvis." Kitt-AI/snowboy#447 talks about this. )

DEBUG:SnowboyWakeListener:Loading snowboy model from /profiles/en/snowboy/jarvis.umdl 
ERROR (ClassifySensitivities():pipeline-detect.cc:744) PipelineDetect: number of hotwords and number of sensitivities mismatch, expecting sensitivities for 0 personal hotwords, and 2 universal hotwords, got 1 sensitivities instead.
terminate called after throwing an instance of 'std::runtime_error'
  what():  ERROR (ClassifySensitivities():pipeline-detect.cc:744) PipelineDetect: number of hotwords and number of sensitivities mismatch, expecting sensitivities for 0 personal hotwords, and 2 universal hotwords, got 1 sensitivities instead.

[stack trace: ]
Backtrace has not been implemented yet.

/run.sh: line 28:    10 Aborted                 (core dumped) python3 app.py "$@"

The following profile will trigger this error:

{
    "wake": {
        "snowboy": {
            "model": "snowboy/jarvis.umdl",
            "sensitivity": "0.8",
        },
        "system": "snowboy"
    }
}

After quite a bit of digging and searching, I found that in wake.py, you have the following code:

            sensitivity = float(self.profile.get("wake.snowboy.sensitivity", 0.5))
            audio_gain = float(self.profile.get("wake.snowboy.audio_gain", 1.0))

            self._logger.debug("Loading snowboy model from %s", model_path)

            self.detector = snowboydetect.SnowboyDetect(
                snowboydecoder.RESOURCE_FILE.encode(), model_path.encode()
            )

            assert self.detector is not None

            sensitivity_str = str(sensitivity).encode()
            self.detector.SetSensitivity(sensitivity_str)

Now I am by NO means proficient in python and, in fact, have avoided even TRYING to learn it till recently (and I've been messing with computers off and on probably for longer than you have been alive. Some of my first coding was in college on a time-shared IBM-360 writing in COBOL, Fortran, PL/1 and RPG in 1979 and later when I wrote a word-processing program in BASIC on a Tandy CoCo computer back in 1981 or so. But python, and PHP, drive me nuts with the un-typed variables and why does JSON need a , at the end of a list such as the one at the end of the "sensitivity" section above? Took me a while to figure out why the syntax highlighting was indicating an error in Joe. But I digress....)

Anyways, I figured out that if I changed profile.json to read ..."sensitivity": "0.8,0.80"... maybe it would work. But nope, it then gave me errors in the float() statement. Changing the code to the following seems to be working OK and allows the multiple hotwords/sensitivity without any errors either from the python or from the Snowboy library.

#            sensitivity = float(self.profile.get("wake.snowboy.sensitivity", 0.5))
            sensitivity = self.profile.get("wake.snowboy.sensitivity", 0.5)
            audio_gain = float(self.profile.get("wake.snowboy.audio_gain", 1.0))

            self._logger.debug("Loading snowboy model from %s", model_path)
                model_path,
                sensitivity,
                audio_gain,
            )
            self.detector = snowboydetect.SnowboyDetect(
                snowboydecoder.RESOURCE_FILE.encode(), model_path.encode()
            )

            assert self.detector is not None
            self._logger.debug("Asserted")

#            sensitivity_str = str(sensitivity).encode()
            sensitivity_str = sensitivity.encode()

So my burning question is: Is there a reason why you were taking the string returned from self.profile.get("wake.snowboy.sensitivity", 0.5) and turning it into a number and then turning it right back into a string again a few lines further down, other than maybe to generate an error if someone did put in a value that was not a number? Of course this way the user would be responsible for making sure the sensitivity string matched the number of hotwords but it does make it more flexible.

Also, I haven't changed/tested it yet but possibly allowing the model to be a list as well may allow for multiple personal hotwords trained for several different people:

"model":"snowboy/jarvis.umdl,snowboy/snowboy.umdl,snowboy/person1.pmld,snowboy/person2.pmld,snowboy/person3.pmld"

The user would just need to make sure there's enough sensitivity settings to match.

[prabbit237]

BTW, great job with it all from what I've seen so far. And with Snips going away (and not even being fully open-source to begin with) I think you'll have more people turning to Rhasspy in the future.

[synesthesiam]

Thanks for taking the time to dig into this issue! I did a float conversion because I was unaware there could be multiple sensitivity values. I had assumed the snowboy library was just using strings for all the parameters because it made something easier in calling their C library.

I can definitely get this fix in there and add support for multiple hotwords. I need to also upgrade my version of snowboy, so this will be a good chance to do it all at once. I'll post here once I've got that done.

Thanks again, and always glad to accept Snips refugees :)

[prabbit237]

The convert from the str returned by the .get() to a float and then the convert from a float back to a str just seemed unneeded altogether (whether it was breaking the code or not) and I couldn't figure out why it was being done and thought maybe I was overlooking something due to not being familiar with python. I even looked up what float() even did just in case it wasn't what I thought it was.

And, as I said, I'm not sure if multiple models will work but worth a shot, anyways.

And I did find one other thing that the multiple sensitivities "breaks." The web UI uses:

<input data-v-6a6550bb="" id="wake-snowboy-sensitivity" type="number" min="0" max="1" step="0.1" class="form-control">

for the sensitivity. So inputting multiple sensitivities is not possible directly (and I'm fine with that and having to put them in manually if needed.) So this last is just another feature request, not a bug report: How about a JSON editor box at the end of the GUI that would allow the profile to be editted completely free-hand? The forms are great and I wouldn't want them to go away but there's always that occasional setting that the forms just don't have setup and then the user has to go hunting for the files. I don't know if you've used the configurator add-in for HassIO but I'm thinking something along that line but just for the one profile.json file.

[synesthesiam]

Should be fixed in 2.4.14