Deployment to Fly.io failed. Sqlite3 cannot be installed.

[markustripp]

Issue summary

• Create a new Shopify app (node) with @shopify/shopify-api@5.2.0 dependency.
• Deploy to Fly.io as described here https://shopify.dev/apps/deployment/web
• Deployment fails with an error that Sqlite3 cannot be installed

The latest version uses node:18-alpine docker image on Fly.io. npm cannot install Sqlite3 and falls back to the source build. In this image, there is no Python available and the deployment fails.

I could not find a good fix for the problem, so I ended up forking @shopify/shopify-api, removing Sqlite3 from the library, and deploying it as a custom module dependency in my package.json.

"dependencies": {
    "@shopify/shopify-api": "file:customized_modules/shopify-api-node-5.2.0",
    ...
}

This is not a long-term solution and I want to get back to a standard dependency as soon as possible.

Expected behavior

Successful deployment to Fly.io

Actual behavior

Deployment failed to Fly.io with the following exception message:

➜  test-fly-01 git:(main) ✗ flyctl deploy --build-arg SHOPIFY_API_KEY=<KEY> --remote-only
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-xxx ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 5.6s (0/1)                                                                                                                            
[+] Building 372.4s (7/8)                                                                                                                          
 => [internal] load remote build context                                                                                                      0.0s
 => copy /context /                                                                                                                           2.1s
 => [internal] load metadata for docker.io/library/node:18-alpine                                                                             0.9s
 => [1/5] FROM docker.io/library/node:18-alpine@sha256:425697..........................................................                     228.4s
 => [2/5] WORKDIR /app                                                                                                                        0.3s
 => [3/5] COPY web .                                                                                                                          0.0s
 => ERROR [4/5] RUN npm install                                                                                                             140.2s
------                                                                                                                                             
 > [4/5] RUN npm install:
#7 140.1 npm ERR! code 1
#7 140.1 npm ERR! path /app/node_modules/sqlite3
#7 140.1 npm ERR! command failed
#7 140.1 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#7 140.1 npm ERR! Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64 --napi_version=8 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
#7 140.1 npm ERR! node-pre-gyp info it worked if it ends with ok
#7 140.1 npm ERR! node-pre-gyp info using node-pre-gyp@1.0.10
#7 140.1 npm ERR! node-pre-gyp info using node@18.12.0 | linux | x64
#7 140.1 npm ERR! node-pre-gyp info check checked for "/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64/node_sqlite3.node" (not found)
#7 140.1 npm ERR! node-pre-gyp http GET https://github.com/TryGhost/node-sqlite3/releases/download/v5.1.2/napi-v6-linux-musl-x64.tar.gz
#7 140.1 npm ERR! node-pre-gyp ERR! install request to https://github.com/TryGhost/node-sqlite3/releases/download/v5.1.2/napi-v6-linux-musl-x64.tar.gz failed, reason: connect ETIMEDOUT 140.82.113.3:443 
#7 140.1 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for sqlite3@5.1.2 and node@18.12.0 (node-v108 ABI, musl) (falling back to source compile with node-gyp) 
#7 140.1 npm ERR! node-pre-gyp WARN Hit error request to https://github.com/TryGhost/node-sqlite3/releases/download/v5.1.2/napi-v6-linux-musl-x64.tar.gz failed, reason: connect ETIMEDOUT 140.82.113.3:443 
#7 140.1 npm ERR! gyp info it worked if it ends with ok
#7 140.1 npm ERR! gyp info using node-gyp@9.1.0
#7 140.1 npm ERR! gyp info using node@18.12.0 | linux | x64
#7 140.1 npm ERR! gyp info ok 
#7 140.1 npm ERR! gyp info it worked if it ends with ok
#7 140.1 npm ERR! gyp info using node-gyp@9.1.0
#7 140.1 npm ERR! gyp info using node@18.12.0 | linux | x64
#7 140.1 npm ERR! gyp ERR! find Python 
#7 140.1 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
#7 140.1 npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
#7 140.1 npm ERR! gyp ERR! find Python checking if "python3" can be used
#7 140.1 npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
#7 140.1 npm ERR! gyp ERR! find Python checking if "python" can be used
#7 140.1 npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
#7 140.1 npm ERR! gyp ERR! find Python 
#7 140.1 npm ERR! gyp ERR! find Python **********************************************************
#7 140.1 npm ERR! gyp ERR! find Python You need to install the latest version of Python.
#7 140.1 npm ERR! gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#7 140.1 npm ERR! gyp ERR! find Python you can try one of the following options:
#7 140.1 npm ERR! gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
#7 140.1 npm ERR! gyp ERR! find Python   (accepted by both node-gyp and npm)
#7 140.1 npm ERR! gyp ERR! find Python - Set the environment variable PYTHON
#7 140.1 npm ERR! gyp ERR! find Python - Set the npm configuration variable python:
#7 140.1 npm ERR! gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
#7 140.1 npm ERR! gyp ERR! find Python For more information consult the documentation at:
#7 140.1 npm ERR! gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#7 140.1 npm ERR! gyp ERR! find Python **********************************************************
#7 140.1 npm ERR! gyp ERR! find Python 
#7 140.1 npm ERR! gyp ERR! configure error 
#7 140.1 npm ERR! gyp ERR! stack Error: Could not find any Python installation to use
#7 140.1 npm ERR! gyp ERR! stack     at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:330:47)
#7 140.1 npm ERR! gyp ERR! stack     at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:159:21)
#7 140.1 npm ERR! gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:202:16)
#7 140.1 npm ERR! gyp ERR! stack     at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:294:16)
#7 140.1 npm ERR! gyp ERR! stack     at exithandler (node:child_process:420:5)
#7 140.1 npm ERR! gyp ERR! stack     at ChildProcess.errorhandler (node:child_process:432:5)
#7 140.1 npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
#7 140.1 npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:289:12)
#7 140.1 npm ERR! gyp ERR! stack     at onErrorNT (node:internal/child_process:476:16)
#7 140.1 npm ERR! gyp ERR! stack     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
#7 140.1 npm ERR! gyp ERR! System Linux 5.12.2
#7 140.1 npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=6" "--node_napi_label=napi-v6"
#7 140.1 npm ERR! gyp ERR! cwd /app/node_modules/sqlite3
#7 140.1 npm ERR! gyp ERR! node -v v18.12.0
#7 140.1 npm ERR! gyp ERR! node-gyp -v v9.1.0
#7 140.1 npm ERR! gyp ERR! not ok 
#7 140.1 npm ERR! node-pre-gyp ERR! build error 
#7 140.1 npm ERR! node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/app/node_modules/sqlite3/lib/binding/napi-v6-linux-musl-x64 --napi_version=8 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
#7 140.1 npm ERR! node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/app/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
#7 140.1 npm ERR! node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
#7 140.1 npm ERR! node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1091:16)
#7 140.1 npm ERR! node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:302:5)
#7 140.1 npm ERR! node-pre-gyp ERR! System Linux 5.12.2
#7 140.1 npm ERR! node-pre-gyp ERR! command "/usr/local/bin/node" "/app/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
#7 140.1 npm ERR! node-pre-gyp ERR! cwd /app/node_modules/sqlite3
#7 140.1 npm ERR! node-pre-gyp ERR! node -v v18.12.0
#7 140.1 npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.10
#7 140.1 npm ERR! node-pre-gyp ERR! not ok
#7 140.1 
#7 140.1 npm ERR! A complete log of this run can be found in:
#7 140.1 npm ERR!     /root/.npm/_logs/2022-10-27T02_21_39_475Z-debug-0.log
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c npm install]: exit code: 1

Steps to reproduce the problem

• Create a new clean Shopify app (node)
• Deploy to Fly.io

Specifications

Build triggered with flyctl v0.0.421 on MacOS X 12.6 M1

[cquemin]

We have now put session storage interfaces and implementations in their own package which you can find in this monorepo.
You can either use one of the many implementations we put there - including sqlite, but also postgres, mysql, redis, in memory, KV , or you can implement the interface that is exposed there and have your own custom implementation.

For that you'll need to update to our new version 6.0.2 of shopify-api-js (you can find it here.