feat(core): simplify truncation logic to only keep the newest message #18906
Conversation
size-limit report 📦
|
node-overhead report 🧳Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
RulaKhaled
left a comment
RulaKhaled
left a comment
There was a problem hiding this comment.
Overall looks really good. Two things we should do:
- Take care of embeddings input
- Remove the media filtering logic entirely (lmk if that's something you want to do as a follow up, we can create a ticket and tackle it after)
| let bytesUsed = 0; | ||
| let startIndex = stripped.length; // Index where the kept suffix starts | ||
| // Strip inline media from the single message | ||
| const stripped = stripInlineMediaFromMessages([lastMessage]); |
There was a problem hiding this comment.
We should entirely remove inline media logic
There was a problem hiding this comment.
will do in a follow-up if need be
| 'gen_ai.operation.name': 'embeddings', | ||
| 'sentry.op': 'gen_ai.embeddings', | ||
| 'gen_ai.system': 'openai', | ||
| 'gen_ai.request.messages': '["This is a small input that fits within the limit"]', |
There was a problem hiding this comment.
so yah we don't want this format, we should map this to https://getsentry.github.io/sentry-conventions/generated/attributes/gen_ai.html#gen_aiembeddingsinput instead of gen_ai.request.messages, let's take care of this here so that request messages type is consistent with the type listed in https://getsentry.github.io/sentry-conventions/generated/attributes/gen_ai.html#gen_aiinputmessages
for truncate message logic for embeddings, according to TEL experience team, we don't need to worry about this just yet.
There was a problem hiding this comment.
Should be good now. When setting the request attributes I now set embeddings inputs separately and it is excluded from the truncation
Changes
gen_ai.embeddings.inputand do not truncate that.Test Updates
Closes #18916