Minor Kaldi updates #302

daanzu · 2020-09-30T09:57:49Z

@LexiconCode Curious if this reduces spurious messages with disconnect.

drmfinlay · 2020-10-02T06:06:58Z

This looks good to me. What are the spurious disconnect messages you mentioned?

LexiconCode · 2020-10-02T06:56:57Z

I mentioned in another thread Kaldi with Linux and possibly Windows error messages on disconnect related to the audio library. I've been quite busy with work so I haven't had a chance to test this yet. I can post examples tomorrow or so.

daanzu · 2020-10-02T09:26:10Z

Previously, depending on timing during disconnect, the audio thread could raise an error. Now, the disconnect thread will wait and join with the audio thread being closed. Actually, I should probably add a time out to the join call. If the audio thread is broken some how, we would probably rather hurry up and disconnect, and accept any later error messages.

@LexiconCode No rush!

LexiconCode · 2020-10-05T05:32:50Z

Here are the error messages for Linux

Using your latest daanzu:kaldi dragonfly branch.

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 110, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)

Using standard dragonfly 0.27.1

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 100, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)

daanzu · 2020-10-05T06:12:58Z

@LexiconCode And this is when you just call disconnect, right?

LexiconCode · 2020-10-13T13:06:36Z

@daanzu hmmm. Test Rule without Caster. Test with subprocess.Popen and without just "disconnect()".

import subprocess, sys, time

from dragonfly import (Choice, Dictation, Function, Grammar, IntegerRef, Key,
                       MappingRule, Text, get_current_engine)

def reboot():
    engine = get_current_engine()
    if engine.name == 'kaldi':
        engine.disconnect()
        # time.sleep(3)
        subprocess.Popen([sys.executable, '-m', 'dragonfly', 'load', '_*.py', '--engine', 'kaldi',  '--no-recobs-messages'])


class TestDragonflyRule(MappingRule):
    mapping = {
        "test reboot":
            Function(reboot),
    } 

    extras = [

    ]

    defaults = {

    }

grammar = Grammar('TestExamples')
grammar.add_rule(TestDragonflyRule())
grammar.load()

However there seems to be issues calling Popen on Linux after disconnect.

This problem is semi-intermittent with subprocess.Popen.

INFO:engine:Loading grammar TestExamples
INFO:engine:Listening...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 110, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)

semi-intermittent Stream is stopped [PaErrorCode -9983] with subprocess.Popen and without.

INFO:engine:Listening...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 110, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)
sounddevice.PortAudioError: Stream is stopped [PaErrorCode -9983]
main@pop-os:~/Caster$ Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 110, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)
sounddevice.PortAudioError: Stream is stopped [PaErrorCode -9983]
INFO:engine:Loading Kaldi-Active-Grammar v1.8.1 in process 8286.
INFO:engine:Kaldi options: {'model_dir': None, 'tmp_dir': None, 'audio_input_device': None, 'audio_self_threaded': True, 'audio_auto_reconnect': True, 'audio_reconnect_callback': None, 'retain_dir': None, 'retain_audio': False, 'retain_metadata': False, 'retain_approval_func': None, 'vad_aggressiveness': 3, 'vad_padding_start_ms': 150, 'vad_padding_end_ms': 200, 'vad_complex_padding_end_ms': 600, 'auto_add_to_user_lexicon': True, 'lazy_compilation': True, 'invalidate_cache': False, 'expected_error_rate_threshold': None, 'alternative_dictation': None, 'cloud_dictation_lang': 'en-US', 'decoder_init_config': None}
Kaldi-Active-Grammar v1.8.1: 
    If this free, open source engine is valuable to you, please consider donating 
    https://github.com/daanzu/kaldi-active-grammar 
    Disable message by calling `kaldi_active_grammar.disable_donation_message()`
INFO:engine:Loading Kaldi-Active-Grammar v1.8.1 in process 8303.
INFO:engine:Kaldi options: {'model_dir': None, 'tmp_dir': None, 'audio_input_device': None, 'audio_self_threaded': True, 'audio_auto_reconnect': True, 'audio_reconnect_callback': None, 'retain_dir': None, 'retain_audio': False, 'retain_metadata': False, 'retain_approval_func': None, 'vad_aggressiveness': 3, 'vad_padding_start_ms': 150, 'vad_padding_end_ms': 200, 'vad_complex_padding_end_ms': 600, 'auto_add_to_user_lexicon': True, 'lazy_compilation': True, 'invalidate_cache': False, 'expected_error_rate_threshold': None, 'alternative_dictation': None, 'cloud_dictation_lang': 'en-US', 'decoder_init_config': None}
Kaldi-Active-Grammar v1.8.1: 
    If this free, open source engine is valuable to you, please consider donating 
    https://github.com/daanzu/kaldi-active-grammar 
    Disable message by calling `kaldi_active_grammar.disable_donation_message()`
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms
INFO:module:CommandModule('_DragonflyExamplesRule.py'): Loading module: '/home/main/Caster/_DragonflyExamplesRule.py'
INFO:engine:Loading grammar TestExamples
INFO:engine:Listening...
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms
INFO:module:CommandModule('_DragonflyExamplesRule.py'): Loading module: '/home/main/Caster/_DragonflyExamplesRule.py'
INFO:engine:Loading grammar TestExamples
INFO:engine:Listening...

Sometimes initializes twice with subprocess.Popen and (without only once).

main@pop-os:~/Caster$ python3 -m dragonfly load _*.py --engine kaldi  --no-recobs-messages
INFO:engine:Loading Kaldi-Active-Grammar v1.8.1 in process 11548.
INFO:engine:Kaldi options: {'model_dir': None, 'tmp_dir': None, 'audio_input_device': None, 'audio_self_threaded': True, 'audio_auto_reconnect': True, 'audio_reconnect_callback': None, 'retain_dir': None, 'retain_audio': False, 'retain_metadata': False, 'retain_approval_func': None, 'vad_aggressiveness': 3, 'vad_padding_start_ms': 150, 'vad_padding_end_ms': 200, 'vad_complex_padding_end_ms': 600, 'auto_add_to_user_lexicon': True, 'lazy_compilation': True, 'invalidate_cache': False, 'expected_error_rate_threshold': None, 'alternative_dictation': None, 'cloud_dictation_lang': 'en-US', 'decoder_init_config': None}
Kaldi-Active-Grammar v1.8.1: 
    If this free, open source engine is valuable to you, please consider donating 
    https://github.com/daanzu/kaldi-active-grammar 
    Disable message by calling `kaldi_active_grammar.disable_donation_message()`
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms
INFO:module:CommandModule('_DragonflyExamplesRule.py'): Loading module: '/home/main/Caster/_DragonflyExamplesRule.py'
INFO:engine:Loading grammar TestExamples
INFO:engine:Listening...
^CException in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/main/.local/lib/python3.8/site-packages/dragonfly/engines/backend_kaldi/audio.py", line 110, in _reader_thread
    in_data, overflowed = self.stream.read(self.stream.blocksize)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 1196, in read
    _check(err)
  File "/home/main/.local/lib/python3.8/site-packages/sounddevice.py", line 2653, in _check
    raise PortAudioError(errormsg, err)
sounddevice.PortAudioError: Stream is stopped [PaErrorCode -9983]
main@pop-os:~/Caster$ INFO:engine:Loading Kaldi-Active-Grammar v1.8.1 in process 12113.
INFO:engine:Kaldi options: {'model_dir': None, 'tmp_dir': None, 'audio_input_device': None, 'audio_self_threaded': True, 'audio_auto_reconnect': True, 'audio_reconnect_callback': None, 'retain_dir': None, 'retain_audio': False, 'retain_metadata': False, 'retain_approval_func': None, 'vad_aggressiveness': 3, 'vad_padding_start_ms': 150, 'vad_padding_end_ms': 200, 'vad_complex_padding_end_ms': 600, 'auto_add_to_user_lexicon': True, 'lazy_compilation': True, 'invalidate_cache': False, 'expected_error_rate_threshold': None, 'alternative_dictation': None, 'cloud_dictation_lang': 'en-US', 'decoder_init_config': None}
Kaldi-Active-Grammar v1.8.1: 
    If this free, open source engine is valuable to you, please consider donating 
    https://github.com/daanzu/kaldi-active-grammar 
    Disable message by calling `kaldi_active_grammar.disable_donation_message()`
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms
INFO:module:CommandModule('_DragonflyExamplesRule.py'): Loading module: '/home/main/Caster/_DragonflyExamplesRule.py'
INFO:engine:Loading grammar TestExamples
INFO:engine:Listening...
WARNING:engine:<dragonfly.engines.backend_kaldi.audio.VADAudio object at 0x7fe9bdac3040>: no good block received recently, so reconnecting audio
WARNING:engine:<dragonfly.engines.backend_kaldi.audio.VADAudio object at 0x7f5355cad100>: no good block received recently, so reconnecting audio
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms
INFO:engine:streaming audio from 'default' using ALSA: 16000 sample_rate, 10 block_duration_ms, 40 latency_ms

The last bit is keyboard interrupt ctrl-c for ^C to end some times after for a short bit i can stlil execute commands.

drmfinlay · 2020-10-23T12:21:08Z

I'll merge this now for dragonfly2 release version 0.28.0. I have verified that the issues I had running the tests before are now resolved. I have also simplified the Kaldi engine test file (test_engine_kaldi.py).

Regarding the above errors, perhaps these should be documented in a separate issue?

daanzu · 2020-10-23T14:35:21Z

@Danesprite Merge sounds good! I am planning on working on fixing the audio, but putting it in an issue doesn't hurt.

This was referenced Sep 30, 2020

Kaldi is too fast daanzu/kaldi-active-grammar#36

Open

Fix Kaldi grammar loading/unloading while in phrase #304

Merged

drmfinlay assigned daanzu Oct 2, 2020

drmfinlay added Bug Unexpected replicated behavior Kaldi Issue related to the Kaldi engine labels Oct 2, 2020

daanzu and others added 5 commits October 22, 2020 22:47

Fix Kaldi disconnect() logic

fa14274

Change Kaldi default vad_padding_end_ms and vad_complex_padding_end_ms

c4a014e

Cleanup Kaldi

458cb77

Cleanup Kaldi audio handling when mimicking with wav files

ecc457c

Remove unnecessary code from the Kaldi engine tests

4f7c5ca

drmfinlay force-pushed the kaldi branch from 6d5d2bd to 4f7c5ca Compare October 22, 2020 12:03

drmfinlay merged commit 020f26f into dictation-toolbox:master Oct 23, 2020

LexiconCode mentioned this pull request Oct 23, 2020

Issues with restarting Kaldi subprocess.Popen and/or disconnect(). #309

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Minor Kaldi updates #302

Minor Kaldi updates #302

daanzu commented Sep 30, 2020

drmfinlay commented Oct 2, 2020

LexiconCode commented Oct 2, 2020 •

edited

Loading

daanzu commented Oct 2, 2020

LexiconCode commented Oct 5, 2020 •

edited

Loading

daanzu commented Oct 5, 2020

LexiconCode commented Oct 13, 2020 •

edited

Loading

drmfinlay commented Oct 23, 2020

daanzu commented Oct 23, 2020

Minor Kaldi updates #302

Minor Kaldi updates #302

Conversation

daanzu commented Sep 30, 2020

drmfinlay commented Oct 2, 2020

LexiconCode commented Oct 2, 2020 • edited Loading

daanzu commented Oct 2, 2020

LexiconCode commented Oct 5, 2020 • edited Loading

daanzu commented Oct 5, 2020

LexiconCode commented Oct 13, 2020 • edited Loading

drmfinlay commented Oct 23, 2020

daanzu commented Oct 23, 2020

LexiconCode commented Oct 2, 2020 •

edited

Loading

LexiconCode commented Oct 5, 2020 •

edited

Loading

LexiconCode commented Oct 13, 2020 •

edited

Loading