[@elevenlabs/react] ConversationProvider state broken on iOS Safari 18+ — tools don't fire, status stuck at disconnected

[amarcher]

Summary

ConversationProvider from @elevenlabs/react is broken on iOS Safari 18.7. When using useConversation() + useConversationClientTool() inside a ConversationProvider, the session connects at the WebSocket level (the onConnect callback fires, audio streams in both directions), but React state never updates:

• status reports disconnected within 1ms of briefly showing connected
• isSpeaking never updates
• Client tools registered via useConversationClientTool() never fire — the SDK logs "Client tool not defined" errors for every tool call

The same agent, same device, same browser works perfectly when calling Conversation.startSession() from @elevenlabs/client directly, bypassing the React provider entirely.

Environment

Component	Version
Device	iPhone 16 Pro (iOS 18.7)
Browser	Safari 26.4 / WebKit
@elevenlabs/react	1.0.3
@elevenlabs/client	1.1.2
React	19.x
Vite	8.x

Desktop browsers (Chrome, Firefox, Safari on macOS) work correctly with ConversationProvider.

Steps to reproduce

// main.tsx
import { ConversationProvider } from '@elevenlabs/react';

createRoot(document.getElementById('root')!).render(
  <ConversationProvider>
    <App />
  </ConversationProvider>
);

// App.tsx
import { useConversation, useConversationClientTool } from '@elevenlabs/react';

function App() {
  const { status, isSpeaking, startSession, endSession } = useConversation();

  useConversationClientTool('navigate_to_planet', (params) => {
    console.log('Tool called:', params); // Never fires on iOS
    return 'OK';
  });

  return (
    <button onClick={() => {
      if (status === 'connected') endSession();
      else startSession({ agentId: 'your-agent-id' });
    }}>
      {status} {isSpeaking ? '(speaking)' : ''}
    </button>
  );
}

1. Deploy to a public URL (localhost won't have mic permissions on iOS)
2. Open on iPhone running iOS 18.7 in Safari
3. Tap the button to start a session
4. Observe: button shows "disconnected" even though the agent is audibly speaking
5. Ask the agent to use a tool — observe "Client tool not defined" in the console

Expected behavior

1. status transitions: disconnected → connecting → connected
2. isSpeaking toggles as the agent speaks
3. Client tools fire when the agent invokes them

Actual behavior (iOS Safari 18.7)

1. startSession() is called
2. status briefly flashes connected, then immediately reverts to disconnected (~1ms)
3. The WebSocket connection IS alive — the agent speaks, audio plays (after the first message), transcripts appear in onMessage
4. isSpeaking never changes from false
5. When the agent tries to call a client tool, the SDK logs: "Client tool not defined: navigate_to_planet" — the tool registrations from useConversationClientTool() are not reaching the live session
6. The session is effectively headless: audio works, but all React state and tool dispatch is broken

Root cause analysis

The provider's internal state management breaks on iOS Safari. Based on extensive debugging:

1. ConversationProvider wraps Conversation.startSession() in a .then(success, failure) chain that manages React context state
2. On iOS Safari, either:
   • The promise rejects silently (the failure handler in ConversationProvider.js:85 runs but only sets lockRef.current = null without logging), or
   • The provider's internal state machine enters an inconsistent state where the WebSocket session is alive but the React context layer believes it's disconnected
3. Because the context reports disconnected, the tool dispatch layer (which is gated on connection status) never forwards tool calls to the registered handlers
4. The useConversationClientTool() registrations are stored in React context state that is stale/disconnected from the live Conversation instance

Evidence

When using ConversationProvider on iOS Safari:

onConnect fired (+3982ms)                      ← SDK callback fires
onStatusChange: {status: "disconnected"}       ← React state immediately reverts (same ms)
isSpeaking: false (never changes)
"Client tool not defined: navigate_to_planet"  ← tools not registered on live session

When bypassing the provider via Conversation.startSession() from @elevenlabs/client on the SAME device:

onConnect fired (+3982ms)
onStatusChange: {status: "connected"}          ← stays connected
onModeChange: {mode: "speaking"}               ← updates correctly
onMessage: {source: "user", message: "Go to Mars."}
navigate_to_planet called: {name: "Mars"}      ← tool works

This confirms the bug is in ConversationProvider's state management, not in the underlying client SDK or iOS Safari's WebSocket support.

Workaround

Bypass ConversationProvider and use Conversation from @elevenlabs/client directly:

import { Conversation } from '@elevenlabs/client';

const convRef = useRef<Conversation | null>(null);
const [status, setStatus] = useState('disconnected');

const start = async () => {
  setStatus('connecting');
  const conv = await Conversation.startSession({
    agentId: 'your-agent-id',
    clientTools: {
      navigate_to_planet: (params) => {
        // Works reliably on iOS
        return 'OK';
      },
    },
    onConnect: () => setStatus('connected'),
    onDisconnect: () => { setStatus('disconnected'); convRef.current = null; },
    onModeChange: (m) => setIsSpeaking(m.mode === 'speaking'),
  });
  convRef.current = conv;
};

Related issues

• isSpeaking and status values ​​from useConversation are not changing. #422 — isSpeaking not updating on iOS (same root cause)
• iOS: Agent not receiving microphone audio on first call after app launch (React Native elevenlabs SDK) #361 — iOS mic not initialized on first call (related iOS lifecycle issue)

Additional iOS issue (separate)

There is a second, unrelated issue on iOS Safari: the first agent message audio is inaudible. Subsequent messages play normally.

• AudioContext.state is running
• The SDK's hidden <audio> element has currentTime advancing past 15+ seconds
• paused is false, volume is 1, readyState is 4
• No audible output from the speaker

This appears to be related to MediaDeviceOutput.create() in output.ts using audioElement.autoplay = true with srcObject = MediaStream but never calling audioElement.play() explicitly. On iOS Safari, autoplay with srcObject does not produce audible output without an explicit play() call in the user gesture chain. Additionally, getUserMedia() switches iOS to AVAudioSessionCategoryPlayAndRecord which routes initial audio output to the earpiece at near-zero volume.

[amarcher]

Minimal reproduction

https://github.com/amarcher/elevenlabs-ios-repro

Two panels side-by-side, same agent:

• Left panel: ConversationProvider + useConversation + useConversationClientTool → broken on iOS Safari (status stuck at disconnected, tools don't fire)
• Right panel: Conversation.startSession() from @elevenlabs/client directly → works on iOS Safari

Setup

git clone https://github.com/amarcher/elevenlabs-ios-repro.git
cd elevenlabs-ios-repro
npm install
echo "VITE_ELEVENLABS_AGENT_ID=your-agent-id" > .env
npx vercel  # deploy to test on iPhone

Open the Vercel URL on iPhone Safari 18+. Tap "Start" on both panels and compare the event logs at the bottom. The left panel's status stays "disconnected" while the right panel correctly shows "connected" and fires tool calls.

ConversationProvider	Direct Conversation

[kraenhansen]

First of all - thanks a lot for the detailed issue report and reproduction! Definitely an issue we'd want to fix.

I suspect #659 might actually help surface more information, as we had a bug where errors thrown from startSession wouldn't correctly end up in the onError callback.

[amarcher]

Thanks @kraenhansen — upgraded the minimal repro to @elevenlabs/react@1.1.0 and the error is now surfaced. On iOS Safari 18.7 the ConversationProvider path fails with:

onError: could not establish pc connection

The WebRTC PeerConnection fails when going through the provider, but succeeds on the same device/browser when calling Conversation.startSession() from @elevenlabs/client directly with identical options.

So PR #659 confirms the error was being swallowed — but the underlying issue is still there: something about the provider's session setup path breaks WebRTC PeerConnection establishment on iOS Safari.

One caveat: the error only surfaced on the second click

Looking at the event log, the first startSession went to "connecting" and then hung — no status change, no error — for 12 seconds. I had to tap the button a second time to get the error:

+0.0s  [provider] calling startSession...
+0.0s  [provider] onStatusChange: {"status":"connecting"}
+0.0s  [provider] startSession returned
                                          ← 12 seconds of silence
+12.3s [provider] calling startSession...
+12.3s [provider] startSession returned
+17.8s [provider] onStatusChange: {"status":"disconnected"}
+17.8s [provider] onError: could not establish pc connection

The second startSession wasn't blocked by lockRef, so the first attempt's promise must have resolved or rejected — clearing the lock — but without firing onError or any status callback.

This suggests there may be a second failure path that #659 doesn't cover: if Conversation.startSession() resolves successfully (WebSocket connects) but the WebRTC PeerConnection fails after that as an async event, the .then(success, failure) handler from #659 won't catch it because the promise didn't reject. The error would need to come through the onError callback composed onto the session options instead.

Updated repro is deployed at https://elevenlabs-ios-repro.vercel.app if you want to test directly.

[kraenhansen]

Thanks for the continued debugging on this, @amarcher. Since your last test on v1.1.0, we merged PR #672 (shipped in @elevenlabs/react@1.1.1) which reworked the ConversationProvider lifecycle to fix a race condition where onConnect fired before conversationRef was set. Could you try upgrading to the latest @elevenlabs/react (currently 1.6.0) and let us know if the issue persists on iOS Safari? If you don't have capacity to do the upgrade and test right now, that's perfectly fine: we'll eventually get to this on our end.

[amarcher]

Thanks @louisjoecodes — upgraded to @elevenlabs/react@1.6.0 (with @elevenlabs/client@1.7.0) and re-ran the repro on iOS Safari 18.7. PR #672 resolves the original issue. ✅

Verified on iOS Safari with both transports:

Path	Status → connected	onConnect order	Bidirectional audio	Client tool dispatch
ConversationProvider + useConversation	✅	✅ (after status)	✅	✅
Conversation.startSession directly (control)	✅	✅	✅	✅

The previously-silent provider path now reaches connected, fires onConnect, streams events, and dispatches tool calls — verified end-to-end with useConversationClientTool handlers receiving real tool invocations (navigate_to_planet, navigate_to_moon) and returning values to the agent.

Happy to have this closed out from our side.

While testing I noticed the agent's first_message is inaudible on iOS Safari in both paths — filed separately as #777 so this issue stays focused.

[kraenhansen]

@amarcher thats great news! Thank you for verifying the fix!