Decode UTF-8 stream chunks#1852
Merged
karthikscale3 merged 9 commits intomainfrom Apr 29, 2026
Merged
Conversation
Contributor
🦋 Changeset detectedLatest commit: addd614 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
workflow with 1 step💻 Local Development
workflow with 10 sequential steps💻 Local Development
workflow with 25 sequential steps💻 Local Development
workflow with 50 sequential steps💻 Local Development
Promise.all with 10 concurrent steps💻 Local Development
Promise.all with 25 concurrent steps💻 Local Development
Promise.all with 50 concurrent steps💻 Local Development
Promise.race with 10 concurrent steps💻 Local Development
Promise.race with 25 concurrent steps💻 Local Development
Promise.race with 50 concurrent steps💻 Local Development
workflow with 10 sequential data payload steps (10KB)💻 Local Development
workflow with 25 sequential data payload steps (10KB)💻 Local Development
workflow with 50 sequential data payload steps (10KB)💻 Local Development
workflow with 10 concurrent data payload steps (10KB)💻 Local Development
workflow with 25 concurrent data payload steps (10KB)💻 Local Development
workflow with 50 concurrent data payload steps (10KB)💻 Local Development
Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
stream pipeline with 5 transform steps (1MB)💻 Local Development
10 parallel streams (1MB each)💻 Local Development
fan-out fan-in 10 streams (1MB each)💻 Local Development
SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
karthikscale3
requested review from
TooTallNate,
VaguelySerious and
pranaygp
and removed request for
TooTallNate
karthikscale3
force-pushed
the
codex/decode-typed-array-stream-chunks
branch
from
5a2386e to
03e6a65
Compare
VaguelySerious
approved these changes
Apr 27, 2026
Member
VaguelySerious
left a comment
VaguelySerious
left a comment
There was a problem hiding this comment.
(AI) I'm not sure how https://github.com/vercel/front/pull/68191 is a mirror of this, but as long as both ends work, LGTM. I haven't tried it out locally, so I'd want to re-test once the live deployment is back online, or before release
…yped-array-stream-chunks
…lapseRefs tests - Remove unused formatStreamChunkForDisplay/sanitizeStreamChunkForDisplay exports; keep only the formatArrayBufferViewForDisplay path actually used by DataInspector. - Replace broken role=tablist/role=tab on the Decoded/Bytes switcher with aria-pressed toggle-button semantics. - Export collapseRefs/isBytesDisplay and add regression tests covering typed-array detection (top-level, nested in object/array/Map/Set, DataView exclusion).
Contributor
Author
Front to close but not exactly 1:1. I will make a separate PR to close the gap. |
* Replace eval with JSON.parse in serialization revive helper devalue.stringify() always produces valid JSON — special values (undefined, NaN, Infinity, -0) are encoded as negative integer sentinels. JSON.parse yields the same flattened array form that unflatten() expects, without the eval anti-pattern (VULN-918). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Drop redundant workflow package from changeset Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Emits Uint8Array chunks containing multi-byte UTF-8 (Latin Extended,
CJK, emoji, RTL Arabic) plus a UTF-8 encoded JSON document, and
asserts each chunk round-trips through TextDecoder({ fatal: true }).
Exercises the same decode path the web inspector relies on for
typed-array stream values.
Made-with: Cursor
1 task
This was referenced Apr 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
formatArrayBufferViewForDisplay+summarizeArrayBufferViewhelpers in@workflow/web-shared(src/lib/stream-display.ts). They UTF-8 decode typed-array values viaTextDecoder({ fatal: true })and fall back to a compactUint8Array(N) [..]summary for binary data.DataInspector(src/components/ui/data-inspector.tsx) now collapses anyArrayBufferView(other thanDataView) into a non-expandableBytesDisplayopaque ref viacollapseRefs, then renders it with aDecodedBytesChunkthat shows decoded text by default and aDecoded/Bytestoggle to inspect the summarized raw value. Decoded text that parses as JSON is re-rendered through a nestedDataInspector.stream-viewer.tsxno longer formats stream chunks itself — chunks flow as raw hydrated values straight intoDataInspector, so Events tab payloads, Trace detail fields, and stream chunks all share the same typed-array rendering path.packages/web/app/lib/hooks/use-stream-reader.tsis dramatically simplified (95 lines removed): it no longer pre-formats chunks, it just keeps them hydrated for the inspector.Decoded/Bytestoggle usesaria-pressed+aria-labeltoggle-button semantics (notrole="tab", since the panel content lives above the buttons rather than as a sibling).src/index.tsexposes only theDecodedStreamChunkSourcetype from this module; the rendering helpers are intentionally internal.test/collapse-refs.test.ts(9 cases): top-level typed array, nested in plain object, array,Mapvalues,Setentries, non-Uint8Arrayviews (Int16Array),DataViewexclusion, binary fallback summary, and primitive passthrough.Why
AI agent workflow stream deltas (e.g.
helloAgentWorkflow) arrive asUint8Arrayvalues serialized through devalue. The previous viewer collapsed them into compact byte summaries likeUint8Array(87) [65, 73, ...], which made the actual text content unreadable. Decoded text alone would be lossy for binary payloads. Routing everything throughDataInspector → collapseRefskeeps the readable default, makes the byte-like source explicit and inspectable, and gives Events / Trace / stream views consistent behavior.Validation
pnpm --filter @workflow/web-shared test→ 22 passed (22)pnpm --filter @workflow/web-shared typecheck→ cleanpnpm --filter @workflow/web-shared build→ cleanpnpm exec biome checkon touched files → only pre-existing complexity warnings on untouched codeReference