Description
Version
24.0.1
Platform
Darwin macbook-air.taile5a6b.ts.net 24.4.0 Darwin Kernel Version 24.4.0: Fri Apr 11 18:34:14 PDT 2025; root:xnu-11417.101.15~117/RELEASE_ARM64_T8122 arm64
Subsystem
webstreams
What steps will reproduce the bug?
Run this in Node.js and browsers
deflate
:
;(async () => {
const valid = new Uint8Array([120, 156, 75, 4, 0, 0, 98, 0, 98]) // deflate('a')
const empty = new Uint8Array(1)
const invalid = new Uint8Array([...valid, ...empty])
const double = new Uint8Array([...valid, ...valid])
for (const chunks of [[valid], [invalid], [valid, empty], [valid, valid], [double]]) {
try {
const stream = new Blob(chunks).stream().pipeThrough(new DecompressionStream('deflate'))
const useFrom = Array.fromAsync && stream.values
console.log(await (useFrom ? Array.fromAsync(stream) : new Response(stream).blob()))
} catch (e) {
console.error(e)
}
}
})()
gzip
:
;(async () => {
const valid = new Uint8Array([31, 139, 8, 0, 0, 0, 0, 0, 0, 19, 75, 4, 0, 67, 190, 183, 232, 1, 0, 0, 0]) // gzip('a')
const empty = new Uint8Array(1)
const invalid = new Uint8Array([...valid, ...empty])
const double = new Uint8Array([...valid, ...valid])
for (const chunks of [[valid], [invalid], [valid, empty], [valid, valid], [double]]) {
try {
const stream = new Blob(chunks).stream().pipeThrough(new DecompressionStream('gzip'))
const useFrom = Array.fromAsync && stream.values
console.log(await (useFrom ? Array.fromAsync(stream) : new Response(stream).blob()))
} catch (e) {
console.error(e)
}
}
})()
How often does it reproduce? Is there a required condition?
Always
What is the expected behavior? Why is that the expected behavior?
First one should be valid and produce an blob/array with [97]
The other four with input past the end should throw a TypeError
(see refs)
Chrome/Firefox/Safari are consistent with the spec and are throwing a TypeError
on any input past the compressed stream.
- Chrome:
TypeError: Junk found after end of compressed data.
- Firefox:
TypeError: Unexpected input after the end of stream
- Safari:
TypeError: Extra bytes past the end.
What do you see instead?
All cases do not throw and produce [ 97 ]
for deflate
.
For gzip
, concatenating inputs concatenates outputs, and [valid, valid]
and [double]
produce [97, 97]
(all other produce [97]
).
Additional information
Refs:
-
https://compression.spec.whatwg.org/#decompression-stream
If the end of the compressed input has not been reached, then throw a TypeError.
-
https://nodejs.org/api/webstreams.html#class-decompressionstream
-
https://github.com/nodejs/node/blob/264cad75ce08dabb6ddc8cc56c/src/node_zlib.cc#L1061-L1072
Activity
0hmX commentedon May 9, 2025
will work on this!
ChALkeR commentedon May 9, 2025
Note that
zlib
anddeflate
ignore that for different reasons.Which is why
gzip
returns[97, 97]
anddeflate
returns[97]
on double buffer.gzip check comes from here and continues to decompress: https://github.com/nodejs/node/blob/264cad75ce08dabb6ddc8cc56c/src/node_zlib.cc#L1061-L1072
deflate just stops and ignores the rest
ChALkeR commentedon May 9, 2025
Also any change likely shouldn't affect default
zlib
behavior, only webstreams.ChALkeR commentedon May 11, 2025
Re: zlib -- once this is fixed in webstreams, perhaps an option for zlib for throwing on garbage after the end could also be added using the same underlying impl. But likely shouldn't be a blocker for the webstreams bugfix.
webstreams: add error handling for trailing junk after stream end
zlib: add error handling for trailing junk after stream end
zlib: add error handling for trailing junk after stream end
lib: add error handling for trailing junk after stream end
stream: making DecompressionStream spec compilent for trailing junk
stream: making DecompressionStream spec compilent for trailing junk
stream: making DecompressionStream spec compilent for trailing junk
stream: making DecompressionStream spec compilent for trailing junk