audio key error in text to stream #136

bbekdemir · 2023-11-09T17:15:02Z

>>> def text_stream():
...     yield "Hi there, I'm Eleven "
...     yield "I'm a text to speech API "
...
>>> audio_stream = generate(
...     text=text_stream(),
...     voice="Nicole",
...     model="eleven_monolingual_v1",
...     stream=True
... )
>>>
>>> stream(audio_stream)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bbekdemir/Developer/sandbox/venv/lib/python3.11/site-packages/elevenlabs/utils.py", line 74, in stream
    for chunk in audio_stream:
  File "/Users/bbekdemir/Developer/sandbox/venv/lib/python3.11/site-packages/elevenlabs/api/tts.py", line 99, in generate_stream_input
    if data["audio"]:
       ~~~~^^^^^^^^^
KeyError: 'audio'

The text was updated successfully, but these errors were encountered:

vincentwi · 2023-11-09T18:21:06Z

change File "/Users/bbekdemir/Developer/bookie_pepper/venv/lib/python3.11/site-packages/elevenlabs/api/tts.py", line 99, in generate_stream_input

to
if data.get("audio")

bbekdemir · 2023-11-09T19:17:20Z

How does that help - audio key does not exist.

natashaaasmi · 2023-11-12T05:27:31Z

Having the same problem

kgovind0001 · 2023-11-17T15:49:38Z

from elevenlabs import generate, stream, play 

audio = generate(
    text="Hi! My name is Bella, nice to meet you!",
    voice="Bella",
    model='eleven_monolingual_v1',
    stream=True
)

# Stream the audio directly 
stream(audio)



# Saving the audio for later 
with open('output.mp3', 'wb') as f:
    for i,chunk in enumerate(audio):
        if chunk: f.write(chunk)

Please try this

nikolas-n · 2023-11-22T20:00:58Z

I just tried in the tts.py file to print the data in order to see what it contained and I saw that I was getting a 401 error, because no api key was set.
So, i just update the imports with
from elevenlabs import generate, stream, set_api_key
and later in the script added
set_api_key("<myactualapikey>")
hope that helps
i believe it'd be good that the README is updated with that

Mascobot · 2023-11-23T03:36:36Z

This worked (adding the set_api_key("") just before 'generate', but only the first time for some reason. I re-ran the exact same code and it gave me again the same error: KeyError: 'audio'

Changing data["audio"] to data.get("audio") doesn't help (there's no 'audio')

nikolas-n · 2023-11-24T14:48:47Z

i'd suggest to add in tts.py before the if data["audio"]: that something like print(data) to see the API's response in case it can give you a lead

nitishymtpl · 2023-11-27T19:49:46Z

@bbekdemir @nikolas-n @Mascobot : Did anyone fixed this bug? It seems, whenever user is sending request, elevenlab is not sending result through websocket. Seems more of server issue or webhook url seems incorrect.

kgovind0001 · 2023-11-27T20:09:54Z

@nitishymtpl I just had a close look and found that it is coming from server, the output is
{'message': 'Unusual activity detected. Free Tier usage disabled. If you are using proxy/VPN you might need to purchase a Paid Plan to not trigger our abuse detectors. Free Tier only works if users do not abuse it, for example by creating multiple free accounts. If we notice that many people try to abuse it, we will need to reconsider Free Tier altogether. Please play fair.\nPlease purchase any Paid Subscription to continue.', 'error': 'quota_exceeded', 'code': 1008}

This worked (adding the set_api_key("") just before 'generate', but only the first time for some reason. I re-ran the exact same code and it gave me again the same error: KeyError: 'audio'

Changing data["audio"] to data.get("audio") doesn't help (there's no 'audio')

This is why a the audio is streamed for the first time but not on rerunning the code.

Mascobot · 2023-11-27T20:12:03Z

@kgovind0001 thx. I am actually using a paid plan...

nitishymtpl · 2023-11-27T20:20:51Z

@Mascobot : Are you having same issues or working perfect?

kgovind0001 · 2023-11-27T20:24:16Z

@Mascobot It should work for you then. I just subscribed to a paid membership.

Can you please give the below a try and let me know if you face any issue.

#https://github.com/elevenlabs/elevenlabs-python/issues/136
from elevenlabs import generate, stream

def text_stream():
    yield "Hi there"
    yield "I am a text"


audio = generate(
    text=text_stream(),
    voice="Nicole",
    model='eleven_monolingual_v1',
    stream=True, 
    api_key = "put api key here"
)

# Stream the audio directly 
stream(audio)

cdubiel · 2023-11-28T07:29:48Z

getting this problem too and I don't want to have to change the library code.

kgovind0001 · 2023-11-28T07:32:36Z

@cdubiel Can you please provide the code and the error ? The below code snippet worked for me. Make sure you have paid subscription.

@Mascobot It should work for you then. I just subscribed to a paid membership.

Can you please give the below a try and let me know if you face any issue.

#https://github.com/elevenlabs/elevenlabs-python/issues/136
from elevenlabs import generate, stream

def text_stream():
    yield "Hi there"
    yield "I am a text"


audio = generate(
    text=text_stream(),
    voice="Nicole",
    model='eleven_monolingual_v1',
    stream=True, 
    api_key = "put api key here"
)

# Stream the audio directly 
stream(audio)

nitishymtpl · 2023-12-05T03:32:35Z

@kgovind0001: Even with the paid plan, this is the error:

stream(audio)
Traceback (most recent call last):
File "", line 1, in
File "/home/codespace/.local/lib/python3.10/site-packages/elevenlabs/utils.py", line 74, in stream
for chunk in audio_stream:
File "/home/codespace/.local/lib/python3.10/site-packages/elevenlabs/api/tts.py", line 85, in generate_stream_input
with connect(
File "/home/codespace/.local/lib/python3.10/site-packages/websockets/sync/client.py", line 266, in connect
sock = ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 1071, in _create
self.do_handshake()
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

RicardoEscobar · 2023-12-05T07:03:30Z

@kgovind0001: Even with the paid plan, this is the error:

stream(audio)
Traceback (most recent call last):
File "", line 1, in
File "/home/codespace/.local/lib/python3.10/site-packages/elevenlabs/utils.py", line 74, in stream
for chunk in audio_stream:
File "/home/codespace/.local/lib/python3.10/site-packages/elevenlabs/api/tts.py", line 85, in generate_stream_input
with connect(
File "/home/codespace/.local/lib/python3.10/site-packages/websockets/sync/client.py", line 266, in connect
sock = ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 1071, in _create
self.do_handshake()
File "/usr/local/python/3.10.8/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

This error occurs when the SSL certificate of the server cannot be verified.
Maybe the API is having issues resolving SSL certificate.

RicardoEscobar · 2023-12-05T07:04:20Z

I do get this error quite often now ~50% of the time.

Error: 'audio'
Traceback (most recent call last):
  File "C:\Users\Jorge\git\core-ai\controller\waifuai\conversation.py", line 328, in stream_conversation
    response = stream_completion.generate_completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jorge\git\core-ai\controller\stream_completion.py", line 247, in generate_completion
    completed_audio_stream = stream(audio_stream)
                             ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jorge\git\core-ai\venv\Lib\site-packages\elevenlabs\utils.py", line 74, in stream
    for chunk in audio_stream:
  File "C:\Users\Jorge\git\core-ai\venv\Lib\site-packages\elevenlabs\api\tts.py", line 99, in generate_stream_input
    if data["audio"]:
       ~~~~^^^^^^^^^
KeyError: 'audio'

peterhanlon · 2023-12-08T19:22:18Z

Running the test code fails for me as well.

from elevenlabs import generate, stream, set_api_key

def text_stream():
    yield "Hi there"
    yield "I am a text"


audio = generate(
    text=text_stream(),
    voice="Nicole",
    model='eleven_monolingual_v1',
    stream=True,
    api_key = "put api key here"
)

stream(audio)

(Output)

Traceback (most recent call last):
  File "/Users/peterhanlon/code/test_audio.py/main.py", line 20, in <module>
    stream(audio)
  File "/Users/peterhanlon/anaconda3/envs/test_audio.py/lib/python3.11/site-packages/elevenlabs/utils.py", line 74, in stream
    for chunk in audio_stream:
  File "/Users/peterhanlon/anaconda3/envs/test_audio.py/lib/python3.11/site-packages/elevenlabs/api/tts.py", line 122, in generate_stream_input
    if data["audio"]:
       ~~~~^^^^^^^^^
KeyError: 'audio'

santoshlite · 2023-12-13T03:11:20Z

Same here, any updates?

RicardoEscobar · 2023-12-13T18:00:29Z

I edited the elevenlabs\api\tts.py file at line 99:

if data["audio"]:

The elevenlabs package is assuming "audio" is present in every websocket message, which is not the case if you want to fix it in the meantime, just change that line to:

if data.get("audio"):

Since get() throws back a None rather than erroring out, this allows the script to continue or throw the actual exeption.

rs2247 · 2024-01-17T03:55:05Z

@RicardoEscobar your suggestion and changing to a paid plan worked for me! thx!

rosscado · 2024-01-19T23:19:24Z

I also ran into this issue while using the streaming text-to-speech API.
Here's a description of the problem, and some ways around it.

The generate_stream_input function in the tts.py module of the ElevenLabs Python library assumes successful audio data responses from the text-to-speech API. This assumption is unsafe, as the API can respond with error messages under various conditions (e.g., missing API key, unended/timed-out input stream, account usage exceeded, etc.).

Proposed Solution:
Rather than assuming success or simply ignoring the error, the maintainers should modify the function to robustly handle error responses from the API. This includes checking for the presence of the 'audio' key and handling cases where it's absent.

Workarounds:

Debug the tts.py module to inspect actual API responses.
End text input streams with an empty string ('') to avoid timeouts.
Upgrade to a paid account or add Two-Factor Authentication to mitigate usage-related errors.

GadflyR · 2024-03-06T00:30:01Z

make sure you subscribed

roperi · 2024-03-07T17:39:26Z

Debugging the tts file as @rosscado suggested, I was able to see the root cause of the issue:

{'message': 'Unusual activity detected. Free Tier usage disabled. If you are using a proxy/VPN you might need to purchase a Paid Plan to not trigger our abuse detectors. Free Tier only works if users do not abuse it, for example by creating multiple free accounts. If we notice that many people try to abuse it, we will need to reconsider Free Tier altogether. \nPlease play fair and purchase any Paid Subscription to continue.', 'error': 'detected_unusual_activity', 'code': 1008}

Changing to a paid subscription solved it.

cdubiel · 2024-03-07T20:46:11Z

thank you

…

On Thu, Mar 7, 2024 at 11:39 AM roperi ***@***.***> wrote: Debugging the tts file as @rosscado <https://github.com/rosscado> suggested, I was able to see the root cause of the issue: {'message': 'Unusual activity detected. Free Tier usage disabled. If you are using a proxy/VPN you might need to purchase a Paid Plan to not trigger our abuse detectors. Free Tier only works if users do not abuse it, for example by creating multiple free accounts. If we notice that many people try to abuse it, we will need to reconsider Free Tier altogether. \nPlease play fair and purchase any Paid Subscription to continue.', 'error': 'detected_unusual_activity', 'code': 1008} Changing to a paid subscription solved it. — Reply to this email directly, view it on GitHub <#136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDLCWG7Z4P7OSCI2T5WARLYXCQ6JAVCNFSM6AAAAAA7E5EJMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBUGA4TCMZYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

dsinghvi · 2024-03-12T19:52:38Z

This is fixed in the latest SDK (snippet). Please reopen if the issue still persists!

KoljaB mentioned this issue Mar 3, 2024

KeyError: 'audio' KoljaB/RealtimeTTS#49

Closed

dsinghvi closed this as completed Mar 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

audio key error in text to stream #136

audio key error in text to stream #136

bbekdemir commented Nov 9, 2023 •

edited

Loading

vincentwi commented Nov 9, 2023

bbekdemir commented Nov 9, 2023

natashaaasmi commented Nov 12, 2023

kgovind0001 commented Nov 17, 2023

nikolas-n commented Nov 22, 2023

Mascobot commented Nov 23, 2023 •

edited

Loading

nikolas-n commented Nov 24, 2023 •

edited

Loading

nitishymtpl commented Nov 27, 2023

kgovind0001 commented Nov 27, 2023

Mascobot commented Nov 27, 2023

nitishymtpl commented Nov 27, 2023

kgovind0001 commented Nov 27, 2023

cdubiel commented Nov 28, 2023

kgovind0001 commented Nov 28, 2023

nitishymtpl commented Dec 5, 2023

RicardoEscobar commented Dec 5, 2023

RicardoEscobar commented Dec 5, 2023

peterhanlon commented Dec 8, 2023 •

edited

Loading

santoshlite commented Dec 13, 2023

RicardoEscobar commented Dec 13, 2023

rs2247 commented Jan 17, 2024

rosscado commented Jan 19, 2024

GadflyR commented Mar 6, 2024

roperi commented Mar 7, 2024

cdubiel commented Mar 7, 2024 via email

dsinghvi commented Mar 12, 2024

audio key error in text to stream #136

audio key error in text to stream #136

Comments

bbekdemir commented Nov 9, 2023 • edited Loading

vincentwi commented Nov 9, 2023

bbekdemir commented Nov 9, 2023

natashaaasmi commented Nov 12, 2023

kgovind0001 commented Nov 17, 2023

nikolas-n commented Nov 22, 2023

Mascobot commented Nov 23, 2023 • edited Loading

nikolas-n commented Nov 24, 2023 • edited Loading

nitishymtpl commented Nov 27, 2023

kgovind0001 commented Nov 27, 2023

Mascobot commented Nov 27, 2023

nitishymtpl commented Nov 27, 2023

kgovind0001 commented Nov 27, 2023

cdubiel commented Nov 28, 2023

kgovind0001 commented Nov 28, 2023

nitishymtpl commented Dec 5, 2023

RicardoEscobar commented Dec 5, 2023

RicardoEscobar commented Dec 5, 2023

peterhanlon commented Dec 8, 2023 • edited Loading

santoshlite commented Dec 13, 2023

RicardoEscobar commented Dec 13, 2023

rs2247 commented Jan 17, 2024

rosscado commented Jan 19, 2024

GadflyR commented Mar 6, 2024

roperi commented Mar 7, 2024

cdubiel commented Mar 7, 2024 via email

dsinghvi commented Mar 12, 2024

bbekdemir commented Nov 9, 2023 •

edited

Loading

Mascobot commented Nov 23, 2023 •

edited

Loading

nikolas-n commented Nov 24, 2023 •

edited

Loading

peterhanlon commented Dec 8, 2023 •

edited

Loading