Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blob.stream().pipeTo() never resolves #48916

Closed
alanshaw opened this issue Jul 25, 2023 · 1 comment
Closed

Blob.stream().pipeTo() never resolves #48916

alanshaw opened this issue Jul 25, 2023 · 1 comment
Labels
confirmed-bug Issues with confirmed bugs.

Comments

@alanshaw
Copy link

Version

20.5.0

Platform

Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

async function main () {
  const blob = new Blob([
    new Uint8Array([
       58, 162, 101, 114, 111, 111, 116, 115, 129, 216,  42,
       88,  37,   0,   1,  85,  18,  32,  95, 102, 197,  19,
      218,  30, 103,  78, 133, 139,  57, 105,  28,  11,   0,
      246, 124, 167,  41, 140, 239, 220, 248, 168, 136,  35,
      196,  72, 236, 184, 232,  89, 103, 118, 101, 114, 115,
      105, 111, 110,   1
    ]),
    new Uint8Array([ 68 ]),
    new Uint8Array([
        1,  85,  18,  32,  95, 102, 197,  19,
      218,  30, 103,  78, 133, 139,  57, 105,
       28,  11,   0, 246, 124, 167,  41, 140,
      239, 220, 248, 168, 136,  35, 196,  72,
      236, 184, 232,  89
    ]),
    new Uint8Array([
       90, 189, 174, 145, 144, 134, 214,
      117,  47, 251, 231, 254,  29, 238,
        0,  29, 212, 201, 123, 107,  95,
      130,  24, 168, 207, 139, 134, 177,
      187,  88, 167,  36
    ])
  ])
  await blob.stream().pipeTo(new WritableStream({
    write (chunk) {
      console.log(chunk)
    }
  }))
}

main().then(() => console.log('done')).catch(console.error)

How often does it reproduce? Is there a required condition?

Every run.

What is the expected behavior? Why is that the expected behavior?

Expected to see all the chunks printed to console and then "done".

What do you see instead?

Uint8Array(59) [
   58, 162, 101, 114, 111, 111, 116, 115, 129, 216,  42,
   88,  37,   0,   1,  85,  18,  32,  95, 102, 197,  19,
  218,  30, 103,  78, 133, 139,  57, 105,  28,  11,   0,
  246, 124, 167,  41, 140, 239, 220, 248, 168, 136,  35,
  196,  72, 236, 184, 232,  89, 103, 118, 101, 114, 115,
  105, 111, 110,   1
]
Uint8Array(1) [ 68 ]

i.e. missing 2 chunks and no "done"

Additional information

No response

@bellbind
Copy link
Contributor

bellbind commented Jul 25, 2023

This code is OK on node-v20.3.1-darwin-arm64.tar.gz, and failed on node-v20.4.0-darwin-arm64.tar.gz

other case:

// blob.mjs
import {describe, it, before, after} from "node:test";
import {strict as assert} from "node:assert";

describe("blob", async () => {
  it("stream", async () => {
    const bufs = [];
    const blob = new Blob(["A", "B", "C"]);
    await blob.stream().pipeTo(new WritableStream({
      write(chunk) {
        bufs.push(new TextDecoder().decode(chunk));
      }
    }));
    assert.equal(bufs.join(""), "ABC");
    //assert.deepEqual(bufs, ["A", "B", "C"]); //node-20.3.1
    //assert.deepEqual(bufs, ["ABC"]); //chrome and firefox
  });
});
$ asdf global nodejs 20.3.1
$ node blob.mjs
▶ blob
  ✔ stream (6.642917ms)
▶ blob (7.957ms)

ℹ tests 1
ℹ suites 1
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 0.057125
$ asdf global nodejs 20.4.0
$ node blob.mjs
▶ blob
  ✖ stream (6.048208ms)
    'Promise resolution is still pending but the event loop has already resolved'

▶ blob (6.810833ms)

ℹ tests 1
ℹ suites 1
ℹ pass 0
ℹ fail 0
ℹ cancelled 1
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 1.912083

✖ failing tests:

✖ stream (6.048208ms)
  'Promise resolution is still pending but the event loop has already resolved'

✖ blob (6.810833ms)
  'Promise resolution is still pending but the event loop has already resolved'

I encountered the problem when using Blob with helia:

import {describe, it} from "node:test";
import {strict as assert} from "node:assert";

import {createHelia} from "helia";
import {unixfs} from "@helia/unixfs";

describe("helia", async () => {
  it("unixfs.addByteStream", async () => {
    const node = await createHelia();
    const nodefs = unixfs(node);

    const blob = new Blob(["Hello World!"], {type: "text/plain"});
    const cid = await nodefs.addByteStream(blob.stream()); //never finished from nodejs>=20.4.0
    console.log(cid);

    await node.stop();
  });
});

From the blame list of lib/internal/blob.js, the bug spawned by the change of 8cc1438 .

This issue may be same as #48668

bellbind added a commit to bellbind/node that referenced this issue Jul 26, 2023
bellbind added a commit to bellbind/node that referenced this issue Jul 26, 2023
…8916

Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
@daeyeon daeyeon added the confirmed-bug Issues with confirmed bugs. label Jul 26, 2023
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 27, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
bellbind added a commit to bellbind/node that referenced this issue Jul 28, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
debadree25 pushed a commit to bellbind/node that referenced this issue Aug 12, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
Ceres6 pushed a commit to Ceres6/node that referenced this issue Aug 14, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Ceres6 pushed a commit to Ceres6/node that referenced this issue Aug 14, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
RafaelGSS pushed a commit that referenced this issue Aug 15, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
RafaelGSS pushed a commit to RafaelGSS/node that referenced this issue Aug 15, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
rluvaton pushed a commit to rluvaton/node that referenced this issue Aug 15, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
RafaelGSS pushed a commit that referenced this issue Aug 16, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
RafaelGSS pushed a commit that referenced this issue Aug 17, 2023
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
vasco-santos pushed a commit to storacha/w3up that referenced this issue Sep 6, 2023
This PR reverts the workaround for the
[bug](nodejs/node#48916) in nodejs 20.5 (and
below) which was [resolved](nodejs/node#48935)
in nodejs 20.6.

---------

Co-authored-by: Travis Vachon <travis.vachon@protocol.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
confirmed-bug Issues with confirmed bugs.
Projects
None yet
3 participants