[googletts] Only white noise #10015

DJPsycho82 · 2021-01-31T11:55:27Z

Platform information:
Hardware: Raspberry Pi 4B 2GB
OS: Openhabian 1.6.2 64Bit
Java Runtime Environment: Java 11
openHAB version: 3.0.1 release build

When openhab service is stopped the voice played bij Google TTS service via 3.5mm jack output, i only hear white noise. Sometimes rebooting the raspberry pi solves the problem. Every time I restart the openhab service, this problem occurs again.

What I checked:

All google voices are present
I can select them in the UI, i tried a couple Wavenet and the ‘default-ones’, doesn’t change
VoiceRSS works fine
The wav files (temp folder) play back nicely when opened on another device
Google limits is ok
I’m just using openhab:voice say test on karaf > no difference
openhab:audio play barking.mp3 < works fine
playback via chromecast is fine

Not sure how to fix it.

***- At this very moment, I had to reboot the Pi > White noise >restarted openhab > white noise when 'saying' something... I cleared the cache with [openhab-cli clean-cache] >>> This solved the problem this time

DJPsycho82 · 2021-03-12T15:44:30Z

Just a bump. Can I help in any kind of way to sort this problem. It really is a problem for me, so I want to help any way i can. But I don't know that to do anymore. :)

cweitkamp · 2021-03-12T16:58:01Z

The add-on stores the sound files generated by Googles TTS in a local folder. You can find them in the $OPENHAB_USERDATA/cache/org.openhab.voice.googletts. Did you purge them over there too? If not please remove the folder and try it again.

DJPsycho82 · 2021-03-12T17:12:24Z

Yes, when I clear cache the folder is empty afterwards but still white noise. Strange thing now is. I can't get it to work anymore. I rebooted, clean cache, restarted openhab service, stopped the bundle via karaf, stopped all other voice related bundles, uninstalled it via add-ons.cfg. but it just gives noise, but only via the 3mm port, via Chromecast the voice works just normal. The sound files in the cache play just fine, so the whole "get Voice from Google"-part works. Just the playback via 3mm Jack gives noise. And only Google TTS. Other mp3's just work fine when Google TTs fails.

cweitkamp · 2021-03-12T17:18:12Z

Are you using "System Speaker" as a default audio sink? Do you have other audio sinks for testing available? Does it work with them?

DJPsycho82 · 2021-03-12T17:20:59Z

Yes, I meant "System speaker" when I said 3mm jack. I tested it with Chromecast audio sink and then it just works. No more noise, but when I change to System speaker i only hear noise.

cweitkamp · 2021-03-12T17:27:17Z

I have the slight feeling it is not related to Google TTS. But javasound. Can you enable DEBUG logging for org.openhab.voice.googletts and org.openhab.core.audio while trying to "say" something and provide the logs over here?

DJPsycho82 · 2021-03-12T17:35:49Z

I think you're right. I didn't know javasound could be the problem, so I only was able to focus on Google TTS.

==> /var/log/openhab/openhab.log <==

2021-03-12 18:32:37.388 [DEBUG] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Synthesize 'h ' for voice 'googletts:nlNLWavenetA' in format AudioFormat [codec=PCM_SIGNED, container=WAVE, bigEndian=true, bitDepth=16, bitRate=705600, frequency=44100]

2021-03-12 18:32:37.390 [DEBUG] [org.openhab.voice.googletts.internal.GoogleCloudAPI     ] - Audio file nlNLWavenetA_e706061669e0a3ec967386145b286b47.wav was found in cache.

2021-03-12 18:32:37.410 [WARN ] [openhab.core.audio.internal.javasound.JavaSoundAudioSink] - Cannot determine master volume level - assuming 100%

DJPsycho82 · 2021-03-12T17:38:05Z

Hmm, when i check the file it outputs it has these properties: 16bit,1 channel, 24000hz. But that's not what the log says.

cweitkamp · 2021-03-12T19:04:54Z

I think we got a track. My log is different - TTS service uses a different audio format (mp3):

2021-03-12 19:51:48.738 [DEBUG] [.googletts.internal.GoogleTTSService] - Synthesize 'Hoe is het met je? ' for voice 'googletts:nlNLWavenetA' in format AudioFormat [codec=MP3, container=NONE, bitDepth=16, bitRate=64000, frequency=44100]
2021-03-12 19:51:49.085 [DEBUG] [ce.googletts.internal.GoogleCloudAPI] - Caching audio file nlNLWavenetA_d5764ad47f5d391466b832c4ea159c5e.mp3
2021-03-12 19:51:49.087 [DEBUG] [ce.googletts.internal.GoogleCloudAPI] - Caching text file nlNLWavenetA_d5764ad47f5d391466b832c4ea159c5e.txt
2021-03-12 19:51:49.106 [WARN ] [nternal.javasound.JavaSoundAudioSink] - Cannot determine master volume level - assuming 100%

When increasing logging to TRACE for googletts and restarting the bundle do you see something like this:

2021-03-12 20:05:34.957 [TRACE] [.googletts.internal.GoogleTTSService] - Initializing audio formats
2021-03-12 20:05:34.958 [WARN ] [.googletts.internal.GoogleTTSService] - Audio format OGG_OPUS is not yet supported.
2021-03-12 20:05:34.958 [TRACE] [.googletts.internal.GoogleTTSService] - Audio format not supported: OGG_OPUS
2021-03-12 20:05:34.959 [TRACE] [.googletts.internal.GoogleTTSService] - Audio format supported: MP3
2021-03-12 20:05:34.960 [TRACE] [.googletts.internal.GoogleTTSService] - Audio format supported: LINEAR16

DJPsycho82 · 2021-03-12T19:19:11Z

I'm not at home right now, but I'm sure I have this in my logs every time Google TTS is used:

format not supported: OGG_OPUS

For the other TRACE logs I have to check my system. Will do this evening. Looks like the system(javasound I guess) is playing the file in a wrong format. Funny thing is, you can hear the words kind of when noise is being played.

(By the way, I'm also from Holland, and (het gaat goed) I'm doing fine haha.

DJPsycho82 · 2021-03-12T20:36:19Z

Here is my output with TRACE, no strange extra's when using voice (noise lol):

2021-03-12 21:24:47.365 [DEBUG] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Synthesize 'avondklok, nu weer thuis ' for voice 'googletts:nlNLWavenetA' in format AudioFormat [codec=PCM_SIGNED, container=WAVE, bigEndian=true, bitDepth=16, bitRate=705600, frequency=44100]

2021-03-12 21:24:47.620 [DEBUG] [org.openhab.voice.googletts.internal.GoogleCloudAPI     ] - Caching audio file nlNLWavenetA_8e78b61c0c80806749139c532a96be5d.wav

2021-03-12 21:24:47.621 [DEBUG] [org.openhab.voice.googletts.internal.GoogleCloudAPI     ] - Caching text file nlNLWavenetA_8e78b61c0c80806749139c532a96be5d.txt

2021-03-12 21:24:47.636 [WARN ] [openhab.core.audio.internal.javasound.JavaSoundAudioSink] - Cannot determine master volume level - assuming 100%

And below, same output as yours.

2021-03-12 21:26:01.944 [DEBUG] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Using cache folder /var/lib/openhab/cache/org.openhab.voice.googletts

2021-03-12 21:26:01.949 [DEBUG] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Updating configuration

2021-03-12 21:26:01.950 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - New configuration: GoogleTTSConfig{pitch=0.0, speakingRate=1.0, volumeGainDb=0.0, purgeCache=false}

2021-03-12 21:26:02.042 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Initializing voices

2021-03-12 21:26:02.045 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Google Cloud TTS voice: pl-PL-Standard-E
.
.
.
2021-03-12 21:26:02.292 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Initializing audio formats

2021-03-12 21:26:02.293 [WARN ] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Audio format OGG_OPUS is not yet supported.

2021-03-12 21:26:02.294 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Audio format not supported: OGG_OPUS

2021-03-12 21:26:02.295 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Audio format supported: MP3

2021-03-12 21:26:02.296 [TRACE] [org.openhab.voice.googletts.internal.GoogleTTSService   ] - Audio format supported: LINEAR16

Get's me thinking, could there be anywhere a config file where the parameters could be set? Maybe there is somewhere a place where the output settings can change. Because the mp3's file frequency is 24000hz, but the log says: [codec=PCM_SIGNED, container=WAVE, bigEndian=true, bitDepth=16, bitRate=705600, frequency=44100]. Not sure if that's cosmetical or just wrong.

[edit: nope, that's not the problem, when it's a config error, the problem should always exist. This problem, mostly exist, not always. How can we check Javasound?]

DJPsycho82 · 2021-03-12T20:56:40Z

Some extra info:

	<Logger level="INFO" name="org.openhab.voice.googletts"/>
	<Logger level="TRACE" name="org.openhab.core.audio"/>
	<Logger level="OFF" name="core.audio.internal.AudioManagerImpl"/>
	<Logger level="OFF" name="openhab.core.audio.internal.javasound.JavaSoundAudioSink"/>

I set the log levels from above to TRACE via Karaf, but when playing, there are no errors. I don't think the problem is within the GoogleTTS binding. But I don't know where to look to check the Enhanced Javasound for errors or settings...

By the way, i'm using a Raspberry Pi 4b. Maybe it's hardware related in some kind of way?

cweitkamp · 2021-03-13T15:26:55Z

All involved bundles (javasound, googletts, chromecast in my case and webaudio) are supporting both audio formats mp3 and wav whereas mp3 is the preferred one. I am not aware of any configuration setting to override the default values. Currently I do not have a clue where to look next. I do not think that it is a hardware problem.

ftr. I am not from the Netherlands. 😉 Ik ben Duits (I am German).

DJPsycho82 · 2021-03-14T08:33:59Z

Me neither... Hopefully someone will find a solution for this after reading this thread. If I stumble upon something I will post it here but I'm out of options too.

(Not many Germans speak Dutch. Nice to see our language 'across the borders' :D.)

BobMiles · 2021-03-17T16:34:08Z

I'm pretty sure it's not a hardware problem either, I'm on a rpi3b+.
I would even say the problems for me began without even restarting openhab. Suddenly it was just white noise and ever since I couldn't get it back to work. Alsa & Co seem fine...
It must be a playback problem on the raspberry, as I can play the voice normally on other speakers...

DJPsycho82 · 2021-03-17T20:28:20Z

With hardware I meant driver for certain Pi model but Nice to hear it isn't only the pi4 but also other models. So it had to do something with playback, but how and where it goes wrong...

jankaluza · 2021-03-22T07:38:33Z

Also hit this and second restart fixed it. It seems whenever TTS uses .wav it does not work. If it uses mp3 it does work. I've started digging into code more. This is just dump of my process. I hope in the end I will find out what's wrong :).

How does google-tts openhab service work?

tts.getSupportedFormat returns two formats for googletts - AudioFormat.CONTAINER_WAVE for PCM (wav) and AudioFormat.CONTAINER_NONE for mp3.
getBestMatch is called here.
getBestMatch calls getPrefferedFormat.
getPreferredFormat prefers CONTAINER_WAVE (it does not return anything else than WAVE).
getPreferredFormat at first checks if that CONTAINER_WAVE has all the values set. If it does, it returns the format and uses it. If it does not, it sets some values to defaults and returns it. I think that's the root issue here:

GoogleTTSService creates the CONTAINER_WAVE like this:

                return new AudioFormat(AudioFormat.CONTAINER_WAVE, AudioFormat.CODEC_PCM_SIGNED, null, bitDepth, null,
                        frequency);

The null values there are for bigEndian and bitRate.

In this case when bigEndian is null, the getPreferredFormat set it the TRUE here.
In this case when bitRate is null, it also computes bitRate as bitRate = Integer.valueOf(bitDepth.intValue() * frequency.intValue());

What is the .wav produced by it?

I've unfortunately removed the cache, but I have this is my terminal history. I have no reason to not believe file for now:

# file csCZStandardA_f80e4f8a8040ff6f79f28a81429514b5.wav 
csCZStandardA_f80e4f8a8040ff6f79f28a81429514b5.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 24000 Hz

What does google API says for text.synthesize?

https://cloud.google.com/text-to-speech/docs/reference/rest/v1/text/synthesize

LINEAR16 | Uncompressed 16-bit signed little-endian samples (Linear PCM). Audio content returned as LINEAR16 also contains a WAV header.

It also seems that sampleRateHertz is by default set according to voice, which for my voice is 24000Hz. This is also what file returns for that wav. It is possible to override it, but they say it might not be supported for all possible values. Not sure if setting it to 44100 would work.

Conclusion

I think there are issues in GoogleTTSService plugin when creating LINEAR16 audioformat in GoogleTTSService.getAudioFormat:

I think it should set frequency according to naturalSampleRateHertz of voice (preferrably) or it should try using 44100 in AudioConfig.sampleRateHertz or it should examine the resulting .wav and construct the format based on that (well, this might not be possible, because you need to create format before requesting the .wav) or it should stop using CONTAINER_WAVE completely and just use MP3. Speaking about MP3, I think the AudioContainer is also not right, but the whathever Openhab uses to play it is probably smart enough when it comes to MP3 to just play it. ffprobe says Stream #0:0: Audio: mp3, 24000 Hz, mono, fltp, 32 kb/s it is not what is set for MP3 audioFormat.
I think it should set bigEndian to FALSE.

Why does it sometimes work?

My theory is that getBestMatch sometimes handles MP3 before WAV from outputs. If that happens mp3 is used, if WAV appears first in output, then WAV is used and it's broken.

This is quite possible, because MP3/WAV is defined in Set here: https://github.com/openhab/openhab-core/blob/ee1d3f3a73c728973e8bb374641615f42f1dc2f9/bundles/org.openhab.core.audio/src/main/java/org/openhab/core/audio/internal/javasound/JavaSoundAudioSink.java#L68

Reading some doc about Java sets, it seems The elements are returned in no particular order (unless this set is an instance of some class that provides a guarantee).

openhab-bot · 2021-03-22T07:50:17Z

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/oh3-google-tts-only-white-noise/112714/22

sjcliffe · 2021-05-09T07:41:04Z

I've just upgraded from 2.5.8 to 3.0.2 and am experiencing this issue on my Raspberry Pi 4.

The generated WAV file in the cache folder play fine using aplay on the machine.

Gransi · 2021-11-15T12:27:54Z

I have the same issue. After several service restarts google TTS works normal.

Hardware: AMD Ryzen 3200G System
OS: Ubuntu 20.04
Java Runtime Environment: Java 11
openHAB version: 3.1.0 release build

jankaluza · 2021-11-15T13:01:03Z

FYI, I've switched to voicerss openhab addon which works as expected for my use-cases. It supports lot of languages and does not have this issue. With google tts, I also had an issue when I had to add the token after few days to get it work. I suggest people to do the same.

DJPsycho82 · 2021-11-15T20:38:23Z

I didn't like voicerss because of the bad sound quality, it sounded as if the speaker was packed in a large blanket, a verry low sample/bit rate. But I checked again today and saw 2 new voices for dutch. I tried one of them and it sounds pretty good. Ok, wavenet from google is just better but this is good enough for me. I still hope someone can fix google TTS one day but for the time being VoiceRSS is fine for me.

sjcliffe · 2021-11-16T07:47:05Z

Voicerss looks and sounds good - I tried it via their website. Unfortunately after installing the binding it seems you can't change from the default voice :-(

DJPsycho82 · 2021-11-16T11:28:41Z

It's a bit off topic. But it's right, voicerss is at a high nitrate on their website. But you can change the voice in the settings of openhab. I did it yesterday. For Dutch there are 3 voices, 1 default, 1 female and 1 male voice. Only the male voice sounds clear for Dutch. I'm pretty sure for English there are more then 1 voices to choose from.

DJPsycho82 · 2021-12-23T10:44:47Z

I updated my openhab to version 3.2 just an hour ago and the white noise problem still exists. Did not try anything fancy but just booted the new version, changed voice to google TTS and executed openhab:voice say h via Karaf and it gave a white noise.

dalgwen · 2021-12-28T09:51:52Z

Hello all.

[DELETED]

EDIT : the bug I mentioned is unrelated, as the method in the TTS service doesn't use the play file method.

…#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com>

dalgwen · 2021-12-28T18:51:19Z

@hanzz was right in this analysis, and it seems that the solution is to parse the resulting audio file to get correct information before trying to play it. I opened a pull request with the fix.

* [googletts] Use real sound returned to get play informations (#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close #10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com>

* [googletts] Use real sound returned to get play informations (openhab#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com> Signed-off-by: Michael Schmidt <mi.schmidt.83@gmail.com>

* [googletts] Use real sound returned to get play informations (openhab#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com>

* [googletts] Use real sound returned to get play informations (openhab#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com> Signed-off-by: Nick Waterton <n.waterton@outlook.com>

* [googletts] Use real sound returned to get play informations (openhab#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com>

* [googletts] Use real sound returned to get play informations (openhab#10015) When google tts returns a wav file, it is now parsed to get correct informations about it.(mandatory for playing it with at least the System speaker audio sink) Close openhab#10015 * Fix a regression, as the WAV fix was incorrectly applied to the MP3 file Signed-off-by: Gwendal Roulleau <gwendal.roulleau@gmail.com> Signed-off-by: Andras Uhrin <andras.uhrin@gmail.com>

cweitkamp transferred this issue from openhab/openhab-core Feb 1, 2021

cweitkamp changed the title ~~OH3 - Google TTS - Only white noise~~ [googletts] Only white noise Mar 12, 2021

DJPsycho82 closed this as completed Mar 13, 2021

DJPsycho82 reopened this Mar 13, 2021

dalgwen mentioned this issue Dec 28, 2021

[googletts] Use returned sound to get play informations #11877

Merged

cweitkamp added bug An unexpected problem or unintended behavior of an add-on PR pending There is a pull request for resolving the issue labels Jan 1, 2022

lolodomo closed this as completed in #11877 Jan 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[googletts] Only white noise #10015

[googletts] Only white noise #10015

DJPsycho82 commented Jan 31, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021 •

edited

DJPsycho82 commented Mar 12, 2021 •

edited

cweitkamp commented Mar 13, 2021 •

edited

DJPsycho82 commented Mar 14, 2021

BobMiles commented Mar 17, 2021 •

edited

DJPsycho82 commented Mar 17, 2021

jankaluza commented Mar 22, 2021 •

edited

openhab-bot commented Mar 22, 2021

sjcliffe commented May 9, 2021

Gransi commented Nov 15, 2021

jankaluza commented Nov 15, 2021

DJPsycho82 commented Nov 15, 2021

sjcliffe commented Nov 16, 2021

DJPsycho82 commented Nov 16, 2021

DJPsycho82 commented Dec 23, 2021

dalgwen commented Dec 28, 2021 •

edited

dalgwen commented Dec 28, 2021

[googletts] Only white noise #10015

[googletts] Only white noise #10015

Comments

DJPsycho82 commented Jan 31, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

cweitkamp commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021

DJPsycho82 commented Mar 12, 2021 • edited

DJPsycho82 commented Mar 12, 2021 • edited

cweitkamp commented Mar 13, 2021 • edited

DJPsycho82 commented Mar 14, 2021

BobMiles commented Mar 17, 2021 • edited

DJPsycho82 commented Mar 17, 2021

jankaluza commented Mar 22, 2021 • edited

How does google-tts openhab service work?

What is the .wav produced by it?

What does google API says for text.synthesize?

Conclusion

Why does it sometimes work?

openhab-bot commented Mar 22, 2021

sjcliffe commented May 9, 2021

Gransi commented Nov 15, 2021

jankaluza commented Nov 15, 2021

DJPsycho82 commented Nov 15, 2021

sjcliffe commented Nov 16, 2021

DJPsycho82 commented Nov 16, 2021

DJPsycho82 commented Dec 23, 2021

dalgwen commented Dec 28, 2021 • edited

dalgwen commented Dec 28, 2021

DJPsycho82 commented Mar 12, 2021 •

edited

DJPsycho82 commented Mar 12, 2021 •

edited

cweitkamp commented Mar 13, 2021 •

edited

BobMiles commented Mar 17, 2021 •

edited

jankaluza commented Mar 22, 2021 •

edited

dalgwen commented Dec 28, 2021 •

edited