Sharp error on deployment

[5o50]

Hello I encounter this error when deploying a simple nextjs app

john@JOHN:~/workspace/app$ npx sst deploy --profile prod --stage prod
SST v2.30.3

➜  App:     app
   Stage:   prod
   Region:  X
   Account: X

[WARNING] aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Monorepo detected at /home/john/workspace/app
Next.js v13.5.6
OpenNext v2.2.4

┌─────────────────────────────────┐
│ OpenNext — Building Next.js app │
└─────────────────────────────────┘


> @app/web@0.1.0 build
> next build

 ✓ Creating an optimized production build
 ✓ Compiled successfully
   Skipping validation of types
 ✓ Linting
 ✓ Collecting page data
 ✓ Generating static pages (9/9)
 ✓ Collecting build traces
 ✓ Finalizing page optimization

Route (app)                              Size     First Load JS
┌ ○ /                                    228 B          80.9 kB
├ ○ /_not-found                          874 B          81.5 kB
├ ○ /about                               204 B          80.8 kB
├ ○ /community-guidelines                205 B          80.8 kB
├ ○ /privacy                             204 B          80.8 kB
├ ○ /support                             228 B          80.9 kB
└ ○ /terms                               204 B          80.8 kB
+ First Load JS shared by all            80.6 kB
  ├ chunks/1dd3208c-b2615f357c962e21.js  51.1 kB
  ├ chunks/53-86c33e625b50f980.js        27.5 kB
  ├ chunks/main-app-ee4882900a7f2a20.js  230 B
  └ chunks/webpack-22732b761b854a9f.js   1.86 kB


○  (Static)  automatically rendered as static HTML (uses no initial props)


┌──────────────────────────────┐
│ OpenNext — Generating bundle │
└──────────────────────────────┘

Bundling static assets...
Bundling cache assets...
Bundling server function...
Applying plugins:: [opennext-13.5-serverHandler,opennext-13.5-util,opennext-13.5-default] for Next version: ^13.5.6
Open-next plugin opennext-13.5-serverHandler -- Applying override for imports from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for handler from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-default -- Applying override for imports from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for processInternalEvent from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for postProcessResponse from ./default.replacement.js
Open-next plugin opennext-13.5-util -- Applying override for requireHooks from ./13.5/util.js
Open-next plugin opennext-13.5-util -- Applying override for requestHandler from ./util.replacement.js
Bundling revalidation function...
Bundling image optimization function...
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8

npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-20T21_43_45_512Z-debug-0.log
node:internal/errors:932
  const err = new Error(message);
              ^

Error: Command failed: npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" sharp@0.32.5
    at checkExecSyncError (node:child_process:890:11)
    at Object.execSync (node:child_process:962:15)
    at createImageOptimizationBundle (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:231:8)
    at build (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:33:5)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 10917,
  stdout: null,
  stderr: null
}

Node.js v21.0.0

Error: There was a problem building the "Site" site.

Need help with this error? Post it in #help on the SST Discord https://sst.dev/discord

Thank you

[5o50]

Also I tried running the failling command it give more info about the error

john@JOHN:~/workspace/app$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" sharp@0.32.5
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Detected globally-installed libvips v8.14.4
npm ERR! sharp: Building from source via node-gyp
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.4.0
npm ERR! gyp info using node@21.0.0 | linux | x64
npm ERR! gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Invalid version number: 18
npm ERR! gyp ERR! stack     at getNodeDir (/home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:57:25)
npm ERR! gyp ERR! stack     at /home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:31:7
npm ERR! gyp ERR! stack     at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
npm ERR! gyp ERR! command "/home/john/.nvm/versions/node/v21.0.0/bin/node" "/home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! gyp ERR! node -v v21.0.0
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-20T21_45_32_141Z-debug-0.log

[sladg]

Error seems pretty clear Error: Invalid version number: 18. You are no .nvm/versions/node/v21.0.0

[5o50]

@sladg thank you for you time!

here is the same run failling on Node.js v18.18.2

john@JOHN:~/workspace/app$ yarn sst deploy --profile prod --stage prod
yarn run v1.22.19
$ /home/john/workspace/app/node_modules/.bin/sst deploy --profile prod --stage prod
SST v2.31.0

➜  App:     app
   Stage:   prod
   Region:  eu-west-3
   Account: 004799217655

[WARNING] aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Monorepo detected at /home/john/workspace/app
Next.js v13.5.6
OpenNext v2.2.4

┌─────────────────────────────────┐
│ OpenNext — Building Next.js app │
└─────────────────────────────────┘

$ next build
 ✓ Creating an optimized production build
 ✓ Compiled successfully
   Skipping validation of types
 ✓ Linting
 ✓ Collecting page data
 ✓ Generating static pages (9/9)
 ✓ Collecting build traces
 ✓ Finalizing page optimization

Route (app)                              Size     First Load JS
┌ ○ /                                    227 B          80.7 kB
├ ○ /_not-found                          880 B          81.4 kB
├ ○ /about                               206 B          80.7 kB
├ ○ /community-guidelines                205 B          80.7 kB
├ ○ /privacy                             203 B          80.7 kB
├ ○ /support                             227 B          80.7 kB
└ ○ /terms                               206 B          80.7 kB
+ First Load JS shared by all            80.5 kB
  ├ chunks/1dd3208c-b2615f357c962e21.js  50.9 kB
  ├ chunks/53-86c33e625b50f980.js        27.5 kB
  ├ chunks/main-app-ee4882900a7f2a20.js  231 B
  └ chunks/webpack-22732b761b854a9f.js   1.86 kB


○  (Static)  automatically rendered as static HTML (uses no initial props)


┌──────────────────────────────┐
│ OpenNext — Generating bundle │
└──────────────────────────────┘

Bundling static assets...
Bundling cache assets...
Bundling server function...
Applying plugins:: [opennext-13.5-serverHandler,opennext-13.5-util,opennext-13.5-default] for Next version: ^13.5.6
Open-next plugin opennext-13.5-default -- Applying override for imports from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for processInternalEvent from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for postProcessResponse from ./default.replacement.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for imports from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for handler from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-util -- Applying override for requireHooks from ./13.5/util.js
Open-next plugin opennext-13.5-util -- Applying override for requestHandler from ./util.replacement.js
Bundling revalidation function...
Bundling image optimization function...
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8

npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-23T15_51_06_478Z-debug-0.log
node:internal/errors:865
  const err = new Error(message);
              ^

Error: Command failed: npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" sharp@0.32.5
    at checkExecSyncError (node:child_process:890:11)
    at Object.execSync (node:child_process:962:15)
    at createImageOptimizationBundle (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:231:8)
    at build (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:33:5)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 29241,
  stdout: null,
  stderr: null
}

Node.js v18.18.2

Error: There was a problem building the "Site" site.

Need help with this error? Post it in #help on the SST Discord https://sst.dev/discord
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[5o50]

here is the log for the failling sub command on Node.js v18.18.2

john@JOHN:~/workspace/app$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" sharp@0.32.5
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Detected globally-installed libvips v8.14.4
npm ERR! sharp: Building from source via node-gyp
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.4.0
npm ERR! gyp info using node@18.18.2 | linux | x64
npm ERR! gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Invalid version number: 18
npm ERR! gyp ERR! stack     at getNodeDir (/home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:57:25)
npm ERR! gyp ERR! stack     at /home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:31:7
npm ERR! gyp ERR! stack     at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
npm ERR! gyp ERR! command "/home/john/.nvm/versions/node/v18.18.2/bin/node" "/home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! gyp ERR! node -v v18.18.2
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-23T15_51_46_398Z-debug-0.log

[5o50]

when I replace "18" by "18.0.0" the command runs well.

the change is needed just here
npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="${nodeOutputPath}" sharp@${sharpVersion},

[sladg]

npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8

This seems like sharp-related thing. Can you try installing libvips yourself instead of relying on Sharp? Check out https://formulae.brew.sh/formula/vips. I would encourage you to open issue on sharp's repo, it seems like they are using libvips version which was not pre-compiled for ARM :/

[5o50]

I'm on Windows WSL 2 Ubuntu (x86_64), I've installed brew install vips successfully but I get the exact same error.
It seems like an arm64 compiled libvips is needed to package the lambda.

Am I the only one not being able to deploy my nextjs app on a standard x86 linux system ?

Is everyone using open-next on an arm64 M1 macbook thus being able to get the arm64 libvips straight from brew cmd ?

thank you for your time

[sladg]

Well, it seems WSL-related I think. Either way, I would recommend changing the --arch flag to x86 and subsequently change architecture of image optimiser in CDK.

@khuezy would it make sense to pre-package whole lambda code so it's just copy-paste? I went with that approach in https://github.com/sladg/imaginex-lambda/tree/master as it was too much headache to deal with different OS-es and settings, so prepackaging everything and uploading ready code to Lambda was easiest.

[khuezy]

Well, it seems WSL-related I think. Either way, I would recommend changing the --arch flag to x86 and subsequently change architecture of image optimiser in CDK.

@khuezy would it make sense to pre-package whole lambda code so it's just copy-paste? I went with that approach in https://github.com/sladg/imaginex-lambda/tree/master as it was too much headache to deal with different OS-es and settings, so prepackaging everything and uploading ready code to Lambda was easiest.

"replacement of shitty build-in NextJS image optimizer." lol

Yea, let's try to use your imagex-lambda. Btw, how shitty is the NextJS image optimizer compared to the python one?

[5o50]

I've found a quick working workaround for the time being
instead of fetching the missing arm libvips from default stechstudio/libvips-lambda
I've been able to override the source to the more richer lovell/sharp-libvips repo
using the npm flag sharp_libvips_binary_host

like this:

$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/nemesis/workspace/app/apps/web/.open-next/image-optimization-function" sharp@0.32.5 --sharp_libvips_binary_host=https://github.com/lovell/sharp-libvips/releases/download/

added 51 packages in 7s

10 packages are looking for funding
  run `npm fund` for details

that could be a quicker fix for open-next

[sladg]

@khuezy it's okay-ish, but there is overhead of starting up Next on image request. Also, trying to separate it efficiently and/or handle dependencies is rather painful. The API is rather simple so recreating in Python with minimal overhead seemed like win-win.

[khuezy]

Since the process is async and result is cached at CDN, the extra latency shouldn't be a big deal.
edit oh I see, the existing solution has the extra latency due to the next dependency.