AWS CRT binary not present in any of the following locations

[AllanZhengYP]

Describe the bug

Transfered from aws/aws-sdk-js-v3#2753

When upgrading to 3.29.0, and compiling a typescript file to a single javascript file with esbuild, and running that with node, you get:

AWS CRT binary not present in any of the following locations:

       /codebuild/output/bin/native/aws-crt-nodejs
        /codebuild/output/bin/linux-x64/aws-crt-nodejs

Your environment

First saw this in a codebuild job, but happens everywhere. So for example AWS Linux Workspace.

UPDTATE:

I saw this on AWS CodeBuild jobs, so that's Ubuntu 20 I think.

SDK version number

@aws-sdk/client-s3@3.29.0

Is the issue in the browser/Node.js/ReactNative?

node.js

Seen with version 14 and 16.

Expected behavior

As per #2750 you should not pull in aws-crt and 200MB of dependencies.

[Created on behalf of @berenddeboer ]

[yaronvhop]

I am having this similar issue with M1 mac.

Error: AWS CRT binary not present in any of the following locations:
        /project/node_modules/aws-crt/dist/bin/native/aws-crt-nodejs
        /project/node_modules/aws-crt/dist/bin/darwin-arm64/aws-crt-nodejs

[KLewin23]

I had the same issue whilst I'm using it in NextJs on Vercel, my temporary fix was just to go back to 3.28.0.

[klingenm]

I have the same issue while bundling my build scripts using rollup.js. I can't roll back to 3.28, since I need things from the newer releases.

Is there any way to circumvent the requirement aws-crt? This sucks.

[bretambrose]

There appear to be two separate issues here: mac M1 and esbuild.

M1 support was added here: https://github.com/awslabs/aws-crt-nodejs/releases/tag/v1.12.2

esbuild is not yet supported by the CRT. It can be made to work if you understand the CRT internals but it is a confusing and terrible experience. I am going to close this issue and open a new feature request scoped to esbuild.

[johannnishant]

@bretambrose It can be made to work if you understand the CRT internals but it is a confusing and terrible experience.
Any examples on making it work?

[bretambrose]

There aren't any existing examples, hence the feature request to include that in its scope (#467).

You can tease out the basic structure needed from https://github.com/awslabs/aws-crt-nodejs/blob/main/lib/native/binding.js#L22-L37

The expectation is that the napi module (aws-crt-nodejs) exists in a directory (relative to binding.js) of '../../dist/bin/<os-arch>'
When using esbuild or other unity/packing transformations, this expectation is broken since all the JS gets dumped in one file. So right now, if you use esbuild to generate your unity JS file, the napi module needs to exist at the same relative path from that JS file (and be bundled with it appropriately if you're deploying to Lambda).

At a bare minimum, we should allow arbitrary flexibility (environment variables or similar) when searching for the napi module, which would allow you to place it wherever you wanted, presumably side-by-side with esbuild output JS file. It would be nice (but I'm not sure how possible it is) to have an off-the-shelf solution for bundling the CRT with esbuild (or webpack or similar) that did not require you to manually manage the native addon in any way.

[mitgol]

@johannnishant Look at my comment on #467