Open
Description
What is the current behavior?
I'm trying to use ElevenLabs as an external speech provider. Following the documentation, I encountered an error when applying the settings.
After spending considerable time debugging, I managed to get it working—but only by patching the SDK. Below are the changes I had to make (I marked it with comments) :
@dataclass
class Endpoint(BaseResponse):
"""
Define a custom endpoint for the agent.
"""
method: Optional[str] = field(default="POST") # This line seems unnecessary
url: str = field(default="")
headers: Optional[List[Header]] = field(
default=None, metadata=dataclass_config(exclude=lambda f: f is None)
) # Using List[Header] results in incorrect JSON format. I had to replace this with Optional[dict] and define headers as raw key-value pairs.So in summary:
-
The
methodfield appears unused or unnecessary. -
The
headers(Headerclass) must be serialized correctly to JSON. Using a dictionary worked.
Here’s the working pseudo code after patching:
speak = Speak(
provider=SpeakProvider(
type="eleven_labs",
model=None,
model_id='eleven_multilingual_v2',
),
endpoint=Endpoint(
url=f'https://api.elevenlabs.io/v1/text-to-speech/{message.dev_options.voice_id}/stream',
headers={'xi-api-key': <11labs api key>}
)
)- I removed
method: Optional[str] = field(default="POST")because it was causing errors. - I changed the type of the headers field to Optional[dict] so it would serialize to JSON correctly:
headers: Optional[dict] = field(
default=None, metadata=dataclass_config(exclude=lambda f: f is None)
)Please consider updating the SDK to handle this case properly.
Of course, it's possible I did something wrong—let me know if that's the case.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Milestone
Relationships
Participants
Activity
lukeocodes commentedon May 14, 2025
Hi @ArturZhukovets, can you provide some more details on what you were originally trying to do. I appreciate your attempt at a workaround here but we want to avoid unions on generics in a type safe environment.
To best figure out the issue, it would really help to see your original code that was not working.
Thanks
ArturZhukovets commentedon May 15, 2025
@lukeocodes I initially tried using the Python SDK to run an agent with an external speech provider (elenen_labs). While doing so, I encountered two issues during the serialization of dataclasses to a dictionary when preparing the settings configuration for Deepgram.
The final JSON structure ended up being invalid due to the Header class and possibly the method field. Unfortunately, I don't have the original code before my changes, but after serializing the SettingsOptions class, the result looked like this:
As you can see, the headers field is not formatted correctly, which causes issues when sending the request.