Reducing memory foot print for vercel AI messages

[JPeer264]

Problem Statement

When sendDefaultPii: true, requestMessagesFromPrompt parses and re-serializes ai.prompt.messages on every AI span, even when no transformation is needed.

For a 4mb prompt, this briefly doubles memory usage and has inside the function another memory footprint (which is unavoidable I guess):

1. Original JSON string on the span attribute
2. Parsed array from JSON.parse
3. Re-serialized string from getJsonString/getTruncatedJsonString

The parse + reserialize is only necessary when:

• System instructions need to be extracted (messages contain role: "system")
• Truncation is enabled

In the common case (no system messages, no truncation), we could skip parsing entirely and reuse the original string.

Solution Brainstorm

Not sure if there is a good solution for it or if it is worth fixing as 4mb is unusually big, but if there is a high traffic this could spike real fast.

Additional Context

No response

Priority

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it.}

[linear-code]

JS-2640

[HazAT]

👍
It can be especially tricky in constrained environments like CF or serverless where memory can be quite limited.
Holding multiple unnecessary copies of large strings in memory could be the difference between OOM and nothing.