feat(obs): structure-preserving truncation for captured prompt/response content - #747
Conversation
…se content
Captured content is usually serialized JSON (the chat prompt is the
serialized request), so the old blunt byte cut at content_max_bytes
left invalid JSON behind — a log console can't render field hierarchy
of a large payload, which is exactly the complaint in
AISIX-Cloud#1014.
Oversized content that parses as JSON is now reduced structurally so
the logged field stays valid JSON:
- long strings keep a prefix + inline '...[aisix: truncated, N bytes
total]' marker inside the value;
- base64 data URIs collapse to a compact placeholder (exact detection
only — bare base64 is indistinguishable from opaque tokens and keeps
the generic prefix+marker);
- long arrays keep head+tail samples around an
{"_aisix_truncated": true, "omitted_items": N} placeholder;
- objects keep every key.
Reduction walks a cap ladder until the result fits; non-JSON content
and JSON that cannot fit even at the tightest rung fall back to the
historical UTF-8-safe byte cut, so the byte cap is always honored.
Applies at both capture stages (handler-side largest cap, per-exporter
re-truncate), which also means the OTLP/Datadog gen_ai.* content
attributes get the same treatment. Re-serialization orders object keys
alphabetically; accepted for over-cap payloads only.
Runs only when content exceeds the cap, so the hot path for normal
requests is unchanged.
Fixes api7/AISIX-Cloud#1014
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 25 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
… skip no-op rungs Audit findings on the initial revision: - two-stage capture->exporter truncation dropped the stage-1 array placeholder into the sampled-out middle, so the stage-2 accounting reported only the surviving elements. A dropped placeholder now folds its omitted_items count into the new placeholder (composition test asserts kept+omitted == original across both stages). - structurally irreducible payloads paid all ladder rungs for nothing; a one-pass measure (largest string / largest array) now skips rungs that cannot change anything, and whitespace-heavy input that fits once compacted returns early. - a string barely over the per-string cap could come out LONGER with the marker appended; the original is kept when the marker form isn't strictly shorter.
1 participant
Problem
When a captured prompt/response exceeds an exporter's
content_max_bytes, the cut was a plain UTF-8-boundary byte slice. The chat prompt is the serialized request JSON, so an over-cap payload landed in SLS as broken JSON — half an object, a dangling string escape — and the console can no longer show field hierarchy, array contents, or where the problem sits.Change
Oversized content that parses as JSON is now reduced structurally, so the logged field stays valid JSON:
...[aisix: truncated, N bytes total]marker inside the value;data:<mime>;base64,payloads collapse to[aisix: base64 data uri omitted, N bytes](detection is exact; bare base64 is indistinguishable from long opaque tokens and keeps the generic prefix+marker);{"_aisix_truncated": true, "omitted_items": N}placeholder element;Reduction walks a fixed cap ladder (per-string cap × array head/tail keeps) until the serialized result fits
content_max_bytes. Non-JSON text keeps the existing UTF-8-safe byte cut, and JSON that cannot fit even at the tightest rung falls back to it too — the byte cap is always honored. The record-levelcontent_truncatedflag is unchanged.Only the
prompt/responsecontent fields are affected; metadata fields are untouched. The logic lives in the shared capture layer (SinkContent::capture/CapturedContent::new), so the same guarantee holds at both capture stages (handler-side largest cap → per-exporter re-truncate; the composition is covered by a test) and for the OTLP/Datadoggen_ai.*content attributes, not just SLS.Notes
Tests
sls-structured-truncation-e2e.test.ts: realaisix+ etcd +aliyun_slsexporter (content_mode=full, 2 KiB cap) against a mock SLS endpoint; decodes the delivered protobuf andJSON.parses thepromptfield back, asserting head/tail sentinels survive, the placeholder accounts for all omitted messages, and the cap is honored.Fixes api7/AISIX-Cloud#1014
🤖 Generated with Claude Code