Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uploading file to GCP with bucket.upload in @google-cloud/storage hangs #24232

Closed
andyjp opened this issue Jun 17, 2024 · 1 comment · Fixed by #24390
Closed

Uploading file to GCP with bucket.upload in @google-cloud/storage hangs #24232

andyjp opened this issue Jun 17, 2024 · 1 comment · Fixed by #24390
Assignees
Labels
bug Something isn't working correctly node compat node:http Issues with `node:http` module - to be fixed with a rewrite

Comments

@andyjp
Copy link

andyjp commented Jun 17, 2024

deno --version output:
deno 1.44.2 (release, aarch64-apple-darwin)
v8 12.6.228.9
typescript 5.4.5

When uploading a file > 128 kib with the bucket.upload function in @google-coud/storage, the function hangs.

Tested with the script found here: #18730 (comment) and @google-cloud/storage@7.11.2

Steps to reproduce

  1. Update the script to log progress, await storage.bucket(bucketName).upload(filePath, { destination: fileName, onUploadProgress: console.log })
  2. Create file greater than 128 kib, e.g. head -c 131073 </dev/random > myfile
  3. Run script deno run -A script.ts upload-file [bucketName] myfile ./myfile
  4. This outputs the below and hangs
{ bytesWritten: 65536, contentLength: "*" }
{ bytesWritten: 131072, contentLength: "*" }
{ bytesWritten: 131073, contentLength: "*" }

With a 128 kib file, it finishes:

{ bytesWritten: 65536, contentLength: "*" }
{ bytesWritten: 131072, contentLength: "*" }
File myfile uploaded to bucket

It works fine with tsx:

npx tsx script.ts upload-file [bucketName] myfile ./myfile
{ bytesWritten: 65536, contentLength: '*' }
{ bytesWritten: 131072, contentLength: '*' }
{ bytesWritten: 131073, contentLength: '*' }
File myfile uploaded to bucket

Related: #24215

@satyarohith
Copy link
Member

I narrowed the issue down to node-fetch. The example below works with Node but not with Deno.

import fetch from "node-fetch";
import { createReadStream, statSync } from "node:fs";

const url = "https://httpbin.org/put";

// head -c 131073 </dev/random > myfile # a byte more than 128KiB
const readableStream = createReadStream("./myfile");

const headers = {
  "Content-Type": "application/octet-stream",
  "Content-Length": statSync("./myfile").size,
};

const options = {
  method: "PUT",
  headers: headers,
  body: readableStream,
};

const res = await fetch(url, options);
console.log("Response:", await res.json());

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working correctly node compat node:http Issues with `node:http` module - to be fixed with a rewrite
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants