Skip to content

Conversation

@nicohrubec
Copy link
Member

@nicohrubec nicohrubec commented Nov 10, 2025

The main OpenAI APIs are the completions and responses API. Currently we truncate messages sent to the API only if the user input is an array of strings. This works for the completions, but not for the responses API (where the input is a plain string).

Updated the truncation logic to also truncate if the input is a plain string (+ test to verify everything works now as expected).

@nicohrubec nicohrubec changed the title Add failing test to confirm responses api string inputs do not get tr… feat(core): Truncate request string inputs in OpenAI integration Nov 10, 2025
@github-actions
Copy link
Contributor

node-overhead report 🧳

Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.

Scenario Requests/s % of Baseline Prev. Requests/s Change %
GET Baseline 8,693 - 8,797 -1%
GET With Sentry 1,323 15% 1,360 -3%
GET With Sentry (error only) 5,996 69% 6,159 -3%
POST Baseline 1,174 - 1,199 -2%
POST With Sentry 493 42% 504 -2%
POST With Sentry (error only) 1,039 89% 1,049 -1%
MYSQL Baseline 3,223 - 3,257 -1%
MYSQL With Sentry 486 15% 451 +8%
MYSQL With Sentry (error only) 2,613 81% 2,654 -2%

View base workflow run

@nicohrubec nicohrubec marked this pull request as ready for review November 10, 2025 13:32
Copy link
Member

@andreiborza andreiborza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for taking care of this!

@nicohrubec nicohrubec merged commit c2a53e7 into develop Nov 11, 2025
195 checks passed
@nicohrubec nicohrubec deleted the nh/openai-requests-api-truncation branch November 11, 2025 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants