Skip to content

Conversation

@AjayKumbham
Copy link
Contributor

Summary

Fixes issue #339 where initializing a RealtimeSession with any config object would break tool registration, making it impossible to use tools with audio customization.

Problem

When a RealtimeSession was initialized with session config (even just voice settings), the tool registration became broken because:

  • The _getMergedSessionConfig method would include tools: undefined in session data when no tools were explicitly provided in the config
  • The OpenAI API interpreted this as a request to disable all tools
  • This created a mutual exclusivity between session configuration and tool functionality

Solution

  • Modified _getMergedSessionConfig to only include the tools field when tools are actually present and non-empty
  • This prevents sending empty/undefined tools arrays which disable tool calls
  • Users can now customize audio settings (voice, turn detection, noise reduction, etc.) while maintaining tool functionality

Changes

  • Core Fix: Updated packages/agents-realtime/src/openaiRealtimeBase.ts
  • Tests: Added comprehensive tests in packages/agents-realtime/test/realtimeSession.test.ts
  • Changeset: Added proper release documentation

Testing

Added tests covering:

  • Tools working with session config provided
  • Tools working without session config
  • Tools preserved in updateSessionConfig calls
  • No tools field included when no tools are provided
  • Original issue scenario reproduction

Fixes #339

…audio customization

- Fix _getMergedSessionConfig to only include tools field when tools are present
- Prevent sending empty/undefined tools array which disables tool calls
- Add comprehensive tests covering the issue scenarios
- Resolves mutual exclusivity between session config and tool functionality

Fixes #339
@changeset-bot
Copy link

changeset-bot bot commented Aug 26, 2025

🦋 Changeset detected

Latest commit: c352690

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@openai/agents-realtime Patch
@openai/agents Patch
@openai/agents-extensions Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

…roperty.call

- Resolves ESLint error: Do not access Object.prototype method 'hasOwnProperty' from target object
- Uses safer Object.prototype.hasOwnProperty.call() pattern to avoid prototype pollution issues
- Maintains same test functionality while following ESLint no-prototype-builtins rule
@AjayKumbham
Copy link
Contributor Author

Hi @seratch, I’ve pushed the changes to resolve the CI issues. Could you please review the updates when you have a chance? Thank you.

Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't checked if the change is relevant yet, but can you simplify the changeset like I suggest?

AjayKumbham and others added 2 commits August 26, 2025 14:58
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
@seratch seratch requested a review from dkundel-openai August 26, 2025 09:32
- Fix voice config precedence: session config voice now overrides agent default voice
- Fix updateSessionConfig test expectations: add agent updates to trigger updateSessionConfig calls
- Tests now properly trigger updateSessionConfig by calling updateAgent()
- Preserves session config voice setting instead of always using agent default

Fixes 4 failing tests:
- includes tools in session config when session config is provided
- preserves tools when updateSessionConfig is called
- does not include tools field when no tools are provided
- reproduces the original issue - tools work with config provided
@seratch
Copy link
Member

seratch commented Aug 26, 2025

@dkundel-openai Although the CI jobs are still failing, the change itself looks good to me for resolving the reported issue. Can you take a look?

…ols are present

- Modified #getSessionConfig to only include tools field when #currentTools.length > 0
- Fixes failing test 'does not include tools field when no tools are provided'
- Preserves existing behavior for agents with actual tools
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AjayKumbham
Copy link
Contributor Author

Hi @dkundel-openai, this PR has passing checks and has already been approved by @seratch. Could you please review when you get a chance? Thanks!

@dkundel-openai
Copy link
Collaborator

Thanks for your contribution. Since I worked on the GA launch we couldn't merge this directly but I ported your change into #402

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session config breaks tool registration - cannot use tools with audio customization

3 participants