Typescript error when using lib-storage upload

[inian]

Describe the bug

const data = await request.file()

const paralellUploads3 = new Upload({
      client,
      params: {
        Bucket,
        Key,
        Body: data.file
      },
    })

In the above code example, data.file is of type NodeJS.ReadableStream.

When I pass that to Upload, it errors out with the following error

error TS2322: Type 'ReadableStream' is not assignable to type 'string | Uint8Array | Blob | ReadableStream<any> | Buffer | Readable | undefined'.
  Type 'ReadableStream' is missing the following properties from type 'Readable': readableEncoding, readableEnded, readableFlowing, readableHighWaterMark, and 7 more.

193         Body: data.file,
            ~~~~

  node_modules/@aws-sdk/client-s3/types/commands/PutObjectCommand.d.ts:7:5
    7     Body?: PutObjectRequest["Body"] | string | Uint8Array | Buffer;
          ~~~~
    The expected type comes from property 'Body' which is declared here on type 'PutObjectCommandInput'

I think this is happening because it is taking in the type definition of WHATWG streams and trying to match with NodeJS.Readable stream.

I had to include DOM types in my tsconfig lib because of this. If I remove "DOM" types from tsconfig, I am getting other errors like this

node_modules/@aws-sdk/client-s3/types/models/models_0.d.ts:5440:23 - error TS2304: Cannot find name 'ReadableStream'.

5440     Body?: Readable | ReadableStream | Blob;
                           ~~~~~~~~~~~~~~

node_modules/@aws-sdk/client-s3/types/models/models_0.d.ts:5440:40 - error TS2304: Cannot find name 'Blob'.

5440     Body?: Readable | ReadableStream | Blob;
                                            ~~~~

How do I force typescript to assume it is a NodeJS.Readable stream here and not a WHATWG stream?

I was able to fix the error by changing the type definition here to NodeJS.ReadableStream instead of ReadableStream and adding "DOM" to my tsconfig lib.

Your environment

SDK version number

@aws-sdk/lib-storage@3.6.1

Is the issue in the browser/Node.js/ReactNative?

Node.js

Details of the browser/Node.js/ReactNative version

Node v12.20.2

[alexforsyth]

@inian this doesnt seem like a bug with the SDK, the idea is to bring in the same types that S3 says it can take. To expand those types we'd need to expand the putObject types and that carries a bunch of additional work (mostly around testing) to make sure everything works as intended.

Also, sorry about having to include DOM types in the tsconfig! We made this choice internally, i know its not ideal and I'm happy to revisit if its a huge issue with the community. I see you linked #1688 where I touched on this.

Since this is just a type error, to get rid of the typescript error in your code immediately you can do something like:

const data = await request.file()

const paralellUploads3 = new Upload({
     client,
     params: {
       Bucket,
       Key,
       Body: (data.file as any) or (data.file as Readable)
     },
   })

I know its a bit ugly, but we probably wont be able to prioritize adding NodeJS.ReadableStream as a valid data input type.

[G-Rath]

@inian you can remove dom lib by declaring the missing interfaces like so:

// @aws-sdk/client-s3 references these DOM lib interfaces,
// so we need them to exist to compile without having DOM.
declare global {
  /* eslint-disable @typescript-eslint/no-empty-interface */
  interface Blob {}
  interface ReadableStream {}
  /* eslint-enable @typescript-eslint/no-empty-interface */
}

This will mean you don't need to do any casting nor have to risk false negatives due to interfaces and globals defined in the DOM lib :)

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.