Skip to content

Conversation

@toubatbrian
Copy link
Contributor

@toubatbrian toubatbrian commented Jan 16, 2026

  1. Fixed type assertion for text content (line 1644)

    • The content object was always cast to InputTextContent, but when role === 'assistant', the type is 'output_text' which corresponds to OutputTextContent
    • Changed to InputTextContent | OutputTextContent to accurately reflect both cases
  2. Fixed return type assertion for message items (line 1663)

    • The return was cast as UserItem, but this function can return AssistantItem or SystemItem depending on the role
    • Changed to UserItem | AssistantItem | SystemItem for accuracy

Summary by CodeRabbit

  • Refactor

    • Improved type compatibility in the realtime message handling to support a broader range of content and item variations.
  • Chores

    • Added a changeset entry recording a patch-level fix to the plugin's type handling.

✏️ Tip: You can customize this high-level summary in your review settings.

@changeset-bot
Copy link

changeset-bot bot commented Jan 16, 2026

🦋 Changeset detected

Latest commit: af47938

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-xai Patch
@livekit/agents Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai
Copy link

coderabbitai bot commented Jan 16, 2026

📝 Walkthrough

Walkthrough

Type assertions in the OpenAI realtime model were broadened to accept additional message content and item types (including output text and assistant/system items); a changeset file documenting the fix was added.

Changes

Cohort / File(s) Summary
Type casting expansion
plugins/openai/src/realtime/realtime_model.ts
Broadened message content cast to `api_proto.InputTextContent
Changeset
.changeset/full-cities-knock.md
Added patch changeset noting the fix: "correct type assertions in livekitItemToOpenAIItem".

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested reviewers

  • lukasIO

Poem

🐰 I nibbled at types, widened the gate,

Output and Assistant now fit in the crate,
A tiny changeset tucked in with care,
Realtime messages hop through the air! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description check ⚠️ Warning The description lacks required sections from the template (Changes Made, Pre-Review Checklist, Testing) and is missing critical checklist confirmations. Complete the PR description template by adding all required sections including Changes Made list, Pre-Review Checklist, Testing details, and Additional Notes sections.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: fixing type assertions in the livekitItemToOpenAIItem function.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f0d4013 and af47938.

📒 Files selected for processing (1)
  • .changeset/full-cities-knock.md
✅ Files skipped from review due to trivial changes (1)
  • .changeset/full-cities-knock.md

✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/openai/src/realtime/realtime_model.ts (1)

1658-1663: Fix type mismatch: SystemItem expects single InputTextContent, not array.

SystemItem.content is defined as InputTextContent (a single object), but the code always returns contentList as an array for all roles including 'system'. UserItem and AssistantItem correctly expect arrays, but SystemItem does not.

When role === 'system' (converted from 'developer' at line 1637), the returned object does not match the SystemItem structure. Consider handling the system role separately by returning only the first content item, or adjust the return statement to reflect the actual type being returned.

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d4b0e16 and f0d4013.

📒 Files selected for processing (1)
  • plugins/openai/src/realtime/realtime_model.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)

Add SPDX-FileCopyrightText and SPDX-License-Identifier headers to all newly added files with '// SPDX-FileCopyrightText: 2025 LiveKit, Inc.' and '// SPDX-License-Identifier: Apache-2.0'

Files:

  • plugins/openai/src/realtime/realtime_model.ts
**/*.{ts,tsx}?(test|example|spec)

📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)

When testing inference LLM, always use full model names from agents/src/inference/models.ts (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')

Files:

  • plugins/openai/src/realtime/realtime_model.ts
**/*.{ts,tsx}?(test|example)

📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)

Initialize logger before using any LLM functionality with initializeLogger({ pretty: true }) from '@livekit/agents'

Files:

  • plugins/openai/src/realtime/realtime_model.ts
🧠 Learnings (1)
📚 Learning: 2026-01-16T14:33:39.551Z
Learnt from: CR
Repo: livekit/agents-js PR: 0
File: .cursor/rules/agent-core.mdc:0-0
Timestamp: 2026-01-16T14:33:39.551Z
Learning: Applies to **/*.{ts,tsx}?(test|example|spec) : When testing inference LLM, always use full model names from `agents/src/inference/models.ts` (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')

Applied to files:

  • plugins/openai/src/realtime/realtime_model.ts
🧬 Code graph analysis (1)
plugins/openai/src/realtime/realtime_model.ts (1)
plugins/openai/src/realtime/api_proto.ts (5)
  • InputTextContent (155-158)
  • OutputTextContent (170-173)
  • UserItem (206-210)
  • AssistantItem (212-216)
  • SystemItem (200-204)
🔇 Additional comments (1)
plugins/openai/src/realtime/realtime_model.ts (1)

1641-1644: LGTM!

The union type correctly reflects that the content type depends on the role: output_text for assistant messages (matching OutputTextContent) and input_text for other roles (matching InputTextContent).

✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.

@toubatbrian
Copy link
Contributor Author

@codex

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Breezy!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@toubatbrian toubatbrian merged commit b7876c8 into main Jan 16, 2026
9 checks passed
@toubatbrian toubatbrian deleted the brian/fix-oai-realtime-type branch January 16, 2026 19:41
@github-actions github-actions bot mentioned this pull request Jan 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants