Skip to content

feat(openai): add logprobs support for chat completions (request encoding, response decoding, tests)#645

Merged
mikehostetler merged 3 commits into
agentjido:mainfrom
www-zaq-ai:main
Apr 25, 2026
Merged

feat(openai): add logprobs support for chat completions (request encoding, response decoding, tests)#645
mikehostetler merged 3 commits into
agentjido:mainfrom
www-zaq-ai:main

Conversation

@jat10
Copy link
Copy Markdown
Contributor

@jat10 jat10 commented Apr 24, 2026

Description

Adds openai_logprobs and openai_top_logprobs provider options to the OpenAI schema. When enabled, logprobs are encoded in the request body and extracted from choices[0].logprobs.content in the shared decode_response_body_openai_format, making them available as response.provider_meta[:logprobs] for all OpenAI Chat Completions responses.

Type of Change

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Breaking Changes

NA

Testing

  • Tests pass (mix test)
  • Quality checks pass (mix quality)

Checklist

  • My code follows the project's style guidelines
  • I have updated the documentation accordingly
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass
  • My commits follow conventional commit format
  • I have NOT edited CHANGELOG.md (it is auto-generated by git_ops)

Related Issues

@mikehostetler mikehostetler added the needs_work Changes requested before merge label Apr 25, 2026
@mikehostetler
Copy link
Copy Markdown
Contributor

Thanks for adding the logprobs support and the solid test coverage around request encoding, non-stream decoding, and streaming accumulation. I marked this needs_work for one small validation gap: openai_top_logprobs is documented as 0-20, matching the OpenAI API, but the schema currently accepts any non-negative integer, so values like 21 pass local option processing and get encoded into the request. Please tighten the schema or add validation for the 0-20 range. Nice addition overall; this should be straightforward to finish.

@jat10
Copy link
Copy Markdown
Contributor Author

jat10 commented Apr 25, 2026

@mikehostetler Fixed, changed openai_top_logprobs type from :non_neg_integer to {:in, 0..20} so NimbleOptions rejects out-of-range values before they ever reach encoding. Added boundary tests for 0, 20 (valid), and -1, 21 (rejected).

Once this merges, I'll open a PR on jido_ai that bumps the req_llm dep and wires up logprobs

@mikehostetler mikehostetler added ready_to_merge and removed needs_work Changes requested before merge labels Apr 25, 2026
@mikehostetler mikehostetler merged commit 071849d into agentjido:main Apr 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants