Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming too fast from Streaming API sample code (unmodified) #742

Closed
chan71 opened this issue Dec 30, 2016 · 9 comments
Closed

Streaming too fast from Streaming API sample code (unmodified) #742

chan71 opened this issue Dec 30, 2016 · 9 comments
Assignees

Comments

@chan71
Copy link

chan71 commented Dec 30, 2016

In which file did you encounter the issue?

transcribe_streaming.py

Did you change the file? If so, how?

No

Describe the issue

We have witnessed a very inconsistent behavior of streaming API functionality of google cloud speech service during the last week in our testing in different environments. I have raised couple of tickets for each observation. However, we didn't realize that issue was even in the original script as we did our initial testing with a modified script to use arecord instead of pyaudio to circumvent an issue in NAO robot OS (see #738 ). As none of the fixes worked out, we tried testing the original transcribe_streaming.py file on windows 8 and python 2.7.11 but ended up with the inconsistent behavior. During our testing, we ran into streaming too fast error couple of times (see following stack trace). This error didn't occur consistently with every attempt we made to run the script. We tried running script for multiple times at 4 different times of the day out of which the error was thrown on two times of the day. This script was used as is with no changes at all. Is there any reason for the script to throw this error at different attempts? What does it depend on? Is there a way to get rid of this error?

C:\google stt integration>python transcribe_streaming.py
I want to open an account
Traceback (most recent call last):
  File "transcribe_streaming-changed.py", line 230, in <module>
    main()
  File "transcribe_streaming-changed.py", line 221, in main
    listen_print_loop(recognize_stream)
  File "transcribe_streaming-changed.py", line 170, in listen_print_loop
    raise RuntimeError('Server error: ' + resp.error.message)
RuntimeError: Server error: Audio data is being streamed too fast. Please stream audio data approximately at real time.

C:\google stt integration>python transcribe_streaming.py
E1230 18:57:05.532000000 15128 src/core/lib/security/transport/handshake.c:128] Security handshake failed: {"created":"@1483104425.532000000","description":"Handshake read failed","file":"src/core/lib
/security/transport/handshake.c","file_line":237,"referenced_errors":[{"created":"@1483104425.532000000","description":"End of TCP stream","file":"src/core/lib/iomgr/tcp_windows.c","file_line":180}]}
I want to open an account
Traceback (most recent call last):
  File "transcribe_streaming.py", line 230, in <module>
    main()
  File "transcribe_streaming.py", line 221, in main
    listen_print_loop(recognize_stream)
  File "transcribe_streaming.py", line 170, in listen_print_loop
    raise RuntimeError('Server error: ' + resp.error.message)
RuntimeError: Server error: Audio data is being streamed too fast. Please stream audio data approximately at real time.
@chan71
Copy link
Author

chan71 commented Jan 4, 2017

@jerjou Is this due to an issue from GCP speech service or the way sample client code is written?

@jerjou
Copy link
Contributor

jerjou commented Jan 4, 2017

Huh... interesting. The "streamed too fast" error you're getting occurs when the sample rate specified in the initial config request doesn't match the rate at which the api is receiving the data. I've seen it happen when the device was opened with a different sample rate than was sent in the config request (See #364). Did you check that the audio device supports the sample rate specified? The sample code assumes a sample rate of 16000, but not all devices support that.

@chan71
Copy link
Author

chan71 commented Jan 5, 2017

@jerjou I have set the sampling rate of input device to 2 channel, 16 bit, 48000 Hz. The options available from my laptop (hp elitebook 8570w with windows 8) limits to 2 channels and higher sampling rates. Unfortunately I don't see mono channel and lower sampling rate options in audio recording properties.

elitebook-8570w-win8-sampling-rates

When rate is set to 48000 and channel = 2 is passed to initial config, getting the following error.

C:\Users\chandana_kithalagama>cd C:\Users\chandana_kithalagama\Documents\1. HLB\google stt integration

C:\>python transcribe_streaming.py
Traceback (most recent call last):
  File "transcribe_streaming.py", line 230, in <module>
    main()
  File "transcribe_streaming.py", line 221, in main
    listen_print_loop(recognize_stream)
  File "transcribe_streaming.py", line 170, in listen_print_loop
    raise RuntimeError('Server error: ' + resp.error.message)
RuntimeError: Server error: Invalid 'audio_content': too long.

@chan71
Copy link
Author

chan71 commented Jan 5, 2017

@jerjou another observation is that when I changed rate back to 16 kHz and channels = 1, I could get the transcripts for 1 min without 'streaming errors'. I wasn't able to get any transcripts yesterday around this time having tried running the sample for 3 times. I noticed this behavior multiple times during last two weeks. Doesn't this suggest a problem at server side? If the sampling rate is the issue, it should consistently throw the error.

@jerjou
Copy link
Contributor

jerjou commented Jan 5, 2017

Per https://goo.gl/z757pE, the api only supports 1-channel (and per https://g.co/cloud/speech/limits#content streaming is limited to 1 minute)

Looking into whether there was a server outage yesterday..

@chan71
Copy link
Author

chan71 commented Jan 5, 2017

@jerjou check with back end guys on how stable the speech API is. I'm asking this as this is in beta. Could it have issues that lead to 'streaming too fast' error at times during a day. I have seen this issue time to time during the past two weeks. I tested it once more now and it works fine which means channels = 2 is working too despite the disclaimer as you pointed out.

@ToryHwang
Copy link

ToryHwang commented Jan 26, 2017

Hi. Tory.
The same issue occurred.

I am using ubuntu 16 on Raspberry pi 3.
And audio uses usb audio.
Is the audio input only 16Khz?
Is there any solution?

=========================================

ras@ras:~$ aplay -l


List of PLAYBACK Hardware Devices 
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
**card 1: Device [USB PnP Sound Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0**


```=============================================
**aplay -L**
null
    Discard all samples (playback) or generate zero samples (capture)
pulse
    PulseAudio Sound Server
default
    Playback/recording through the PulseAudio sound server
sysdefault:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
dmix:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct sample mixing device
dmix:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct sample mixing device
dsnoop:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct sample snooping device
dsnoop:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct sample snooping device
hw:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct hardware device without any conversions
hw:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct hardware device without any conversions
plughw:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Hardware device with all software conversions
plughw:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Hardware device with all software conversions
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
front:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Front speakers
surround21:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct sample mixing device
dsnoop:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct sample snooping device
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions

=========================================
**system_info.py**
	--------------------------------
	defaultSampleRate: 44100.0
	defaultLowOutputLatency: 0.00868480725624
	defaultLowInputLatency: 0.00868480725624
	maxInputChannels: 1
	structVersion: 2
	hostApi: 0 (ALSA)
	index: 2
	defaultHighOutputLatency: 0.0348299319728
	maxOutputChannels: 2
	name: USB PnP Sound Device: Audio (hw:1,0)
	defaultHighInputLatency: 0.0348299319728
	**Input rates: [44100.0, 48000.0]
	Output rates: [44100.0, 48000.0]
	Full duplex: [44100.0, 48000.0]**

=========================================
**/usr/share/alsa/alsa.conf**

defaults.ctl.card 1
defaults.pcm.card 1
defaults.pcm.device 0
defaults.pcm.subdevice -1
defaults.pcm.nonblock 0
defaults.pcm.compat 0
defaults.pcm.minperiodtime 5000         # in us
defaults.pcm.ipc_key 5678293
defaults.pcm.ipc_gid audio
defaults.pcm.ipc_perm 0660
defaults.pcm.dmix.max_periods 0
defaults.pcm.dmix.rate 44100
#defaults.pcm.dmix.rate 16000
defaults.pcm.dmix.format "unchanged"
defaults.pcm.dmix.card defaults.pcm.card
defaults.pcm.dmix.device defaults.pcm.device
defaults.pcm.dsnoop.card defaults.pcm.card
defaults.pcm.dsnoop.device defaults.pcm.device
defaults.pcm.front.card defaults.pcm.card
defaults.pcm.front.device defaults.pcm.device
defaults.pcm.rear.card defaults.pcm.card
defaults.pcm.rear.device defaults.pcm.device
defaults.pcm.center_lfe.card defaults.pcm.card
defaults.pcm.center_lfe.device defaults.pcm.device
defaults.pcm.side.card defaults.pcm.card
defaults.pcm.side.device defaults.pcm.device
defaults.pcm.surround21.card defaults.pcm.card
defaults.pcm.surround21.device defaults.pcm.device
defaults.pcm.surround40.card defaults.pcm.card
defaults.pcm.surround40.device defaults.pcm.device
defaults.pcm.surround41.card defaults.pcm.card
defaults.pcm.surround41.device defaults.pcm.device
defaults.pcm.surround50.card defaults.pcm.card
defaults.pcm.surround50.device defaults.pcm.device
defaults.pcm.surround51.card defaults.pcm.card
defaults.pcm.surround51.device defaults.pcm.device
defaults.pcm.surround71.card defaults.pcm.card
defaults.pcm.surround71.device defaults.pcm.device
defaults.pcm.iec958.card defaults.pcm.card
defaults.pcm.iec958.device defaults.pcm.device
defaults.pcm.modem.card defaults.pcm.card
defaults.pcm.modem.device defaults.pcm.device
# truncate files via file or tee PCM
defaults.pcm.file_format        "raw"
defaults.pcm.file_truncate      true
defaults.rawmidi.card 1
defaults.rawmidi.device 0
defaults.rawmidi.subdevice -1
defaults.hwdep.card 1
defaults.hwdep.device 0
defaults.timer.class 2
defaults.timer.sclass 0
defaults.timer.card 1
defaults.timer.device 0
defaults.timer.subdevice 0

=========================================
** transcribe_steaming.py** 
** < Modify >**
# Audio recording parameters
#RATE = 8000
**RATE = 44100** 
CHUNK = int(RATE / 10)  # 100ms

=========================================
** < Result >**
(env) ras@ras:~/src/python-docs-samples/speech/grpc$ python transcribe_streaming.py
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
Traceback (most recent call last):
  File "transcribe_streaming.py", line 239, in <module>
    main()
  File "transcribe_streaming.py", line 228, in main
    listen_print_loop(recognize_stream)
  File "transcribe_streaming.py", line 176, in listen_print_loop
    raise RuntimeError('Server error: ' + resp.error.message)
**RuntimeError: Server error: Audio data is being streamed too fast. Please stream audio data approximately at real time**.

@jerjou
Copy link
Contributor

jerjou commented Feb 28, 2017

This seems to be the same issue as #738, plus perhaps an intermittent server outage? Closing for now - please comment with clarification if this is a separate issue.

@jerjou jerjou closed this as completed Feb 28, 2017
@chan71
Copy link
Author

chan71 commented Feb 28, 2017

Thanks @jerjou. Yes, it is more or less the same issue but with the original sample code from git. @ToryHwang could you confirm if this is fixed? I don't have a laptop with me (just migrated to Australia) to confirm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants