add truncate option for oversized events

[vparfonov]

Question

Hi Vector team,
Following up on the addition of max_merged_line_bytes in PR #22582, I noticed that when an event exceeds the maximum allowed size, the current behavior is to drop the event entirely.

I'd like to propose building on that API by adding an option to truncate huge events instead of dropping them. This would allow us to still send the partial (truncated) event payload downstream to sinks, ensuring we don't completely lose visibility into these oversized logs.

Does it make sense to add this functionality? If the team thinks this is a good direction, I would be happy to start working on a PR for it.

Let me know your thoughts!

Vector Config

No response

Vector Logs

No response

[pront]

Hi @vparfonov,

Truncation as an alternative to dropping is a reasonable addition. There are cases where a partial event with context is more useful than no event at all. We would welcome a contribution here. A few things to consider for this PR:

• we should mark the event as truncated
• add a new counter metric to count the truncated events
• truncation should be a new opt-in behavior e.g. overflow_behavior: drop | truncate where drop is the default value

[vparfonov]

#25567