Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switched to prebuild + prebuild-install #1735

Merged
merged 1 commit into from
Dec 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
25 changes: 10 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,40 +82,35 @@ jobs:
echo "CFLAGS=${CFLAGS:-} -include ../src/gcc-preinclude.h" >> $GITHUB_ENV
echo "CXXFLAGS=${CXXFLAGS:-} -include ../src/gcc-preinclude.h" >> $GITHUB_ENV

- name: Configure build
run: yarn node-pre-gyp configure --target_arch=${{ env.TARGET }}

- name: Build binaries
run: yarn node-pre-gyp build --target_arch=${{ env.TARGET }}
run: yarn prebuild -a ${{ env.TARGET }}

- name: Print binary info
if: contains(matrix.os, 'ubuntu')
run: |
ldd lib/binding/*/node_sqlite3.node
ldd build/**/node_sqlite3.node
echo "---"
nm lib/binding/*/node_sqlite3.node | grep "GLIBC_" | c++filt || true
nm build/**/node_sqlite3.node | grep "GLIBC_" | c++filt || true
echo "---"
file lib/binding/napi-v*/*
file build/**/node_sqlite3.node

- name: Run tests
run: yarn test

- name: Package prebuilt binaries
run: yarn node-pre-gyp package --target_arch=${{ env.TARGET }}

- name: Upload binaries to commit artifacts
uses: actions/upload-artifact@v3
if: matrix.node == 18
with:
name: prebuilt-binaries
path: build/stage/*/*
path: prebuilds/*
retention-days: 7

- name: Upload binaries to GitHub Release
run: yarn node-pre-gyp-github publish
run: yarn upload
if: matrix.node == 18 && startsWith(github.ref, 'refs/tags/')
env:
NODE_PRE_GYP_GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ github.token }}

build-qemu:
runs-on: ubuntu-latest
if: github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/')
Expand Down Expand Up @@ -167,7 +162,7 @@ jobs:
retention-days: 7

- name: Upload binaries to GitHub Release
run: yarn install --ignore-scripts && yarn node-pre-gyp-github publish
run: yarn install --ignore-scripts && yarn upload
if: matrix.node == 18 && startsWith(github.ref, 'refs/tags/')
env:
NODE_PRE_GYP_GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ github.token }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,4 @@ setup.sh
*.tgz
package-lock.json
yarn.lock
prebuilds
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ yarn add sqlite3

`sqlite3` v5+ was rewritten to use [Node-API](https://nodejs.org/api/n-api.html) so prebuilt binaries do not need to be built for specific Node versions. `sqlite3` currently builds for both Node-API v3 and v6. Check the [Node-API version matrix](https://nodejs.org/api/n-api.html#node-api-version-matrix) to ensure your Node version supports one of these. The prebuilt binaries should be supported on Node v10+.

The module uses [node-pre-gyp](https://github.com/mapbox/node-pre-gyp) to download the prebuilt binary for your platform, if it exists. These binaries are hosted on GitHub Releases for `sqlite3` versions above 5.0.2, and they are hosted on S3 otherwise. The following targets are currently provided:
The module uses [`prebuild-install`](https://github.com/prebuild/prebuild-install) to download the prebuilt binary for your platform, if it exists. These binaries are hosted on GitHub Releases for `sqlite3` versions above 5.0.2, and they are hosted on S3 otherwise. The following targets are currently provided:

Format: `napi-v{napi_build_version}-{platform}-{libc}-{arch}`

Expand All @@ -56,11 +56,11 @@ Format: `napi-v{napi_build_version}-{platform}-{libc}-{arch}`
* `napi-v6-win32-unknown-ia32`
* `napi-v6-win32-unknown-x64`

Unfortunately, [node-pre-gyp](https://github.com/mapbox/node-pre-gyp) cannot differentiate between `armv6` and `armv7`, and instead uses `arm` as the `{arch}`. Until that is fixed, you will still need to install `sqlite3` from [source](#source-install).
Unfortunately, [prebuild](https://github.com/prebuild/prebuild/issues/174) cannot differentiate between `armv6` and `armv7`, and instead uses `arm` as the `{arch}`. Until that is fixed, you will still need to install `sqlite3` from [source](#source-install).

Support for other platforms and architectures may be added in the future if CI supports building on them.

If your environment isn't supported, it'll use `node-gyp` to build SQLite but you will need to install a C++ compiler and linker.
If your environment isn't supported, it'll use `node-gyp` to build SQLite, but you will need to install a C++ compiler and linker.

### Other ways to install

Expand Down Expand Up @@ -148,8 +148,6 @@ NODE_WEBKIT_VERSION="0.8.6" # see latest version at https://github.com/rogerwang
npm install sqlite3 --build-from-source --runtime=node-webkit --target_arch=ia32 --target=$(NODE_WEBKIT_VERSION)
```

This command internally calls out to [`node-pre-gyp`](https://github.com/mapbox/node-pre-gyp) which itself calls out to [`nw-gyp`](https://github.com/rogerwang/nw-gyp) when the `--runtime=node-webkit` option is passed.

You can also run this command from within a `sqlite3` checkout:

```bash
Expand Down
14 changes: 2 additions & 12 deletions binding.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
"includes": [ "deps/common-sqlite.gypi" ],
"variables": {
"sqlite%":"internal",
"sqlite_libname%":"sqlite3"
"sqlite_libname%":"sqlite3",
"module_name": "node_sqlite3",
},
"targets": [
{
Expand Down Expand Up @@ -52,17 +53,6 @@
"src/statement.cc"
],
"defines": [ "NAPI_VERSION=<(napi_build_version)", "NAPI_DISABLE_CPP_EXCEPTIONS=1" ]
},
{
"target_name": "action_after_build",
"type": "none",
"dependencies": [ "<(module_name)" ],
"copies": [
{
"files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
"destination": "<(module_path)"
}
]
}
]
}
6 changes: 1 addition & 5 deletions lib/sqlite3-binding.js
Original file line number Diff line number Diff line change
@@ -1,5 +1 @@
const binary = require('@mapbox/node-pre-gyp');
const path = require('path');
const binding_path = binary.find(path.resolve(path.join(__dirname,'../package.json')));
const binding = require(binding_path);
module.exports = exports = binding;
module.exports = require('bindings')('node_sqlite3.node');
17 changes: 7 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,6 @@
"url": "https://mapbox.com/"
},
"binary": {
"module_name": "node_sqlite3",
"module_path": "./lib/binding/napi-v{napi_build_version}-{platform}-{libc}-{arch}",
"host": "https://github.com/TryGhost/node-sqlite3/releases/download/",
"remote_path": "v{version}",
"package_name": "napi-v{napi_build_version}-{platform}-{libc}-{arch}.tar.gz",
"napi_versions": [
3,
6
Expand Down Expand Up @@ -49,14 +44,15 @@
"url": "https://github.com/TryGhost/node-sqlite3.git"
},
"dependencies": {
"@mapbox/node-pre-gyp": "^1.0.0",
"bindings": "^1.5.0",
"node-addon-api": "^4.2.0",
"prebuild-install": "^7.1.1",
"tar": "^6.1.11"
},
"devDependencies": {
"eslint": "8.56.0",
"mocha": "10.2.0",
"node-pre-gyp-github": "1.4.4"
"prebuild": "12.1.0"
},
"peerDependencies": {
"node-gyp": "8.x"
Expand All @@ -70,9 +66,10 @@
"node-gyp": "8.x"
},
"scripts": {
"build": "node-pre-gyp build",
"build:debug": "node-pre-gyp build --debug",
"install": "node-pre-gyp install --fallback-to-build",
"install": "prebuild-install -r napi || node-gyp rebuild",
"prebuild": "prebuild --runtime napi --all --verbose",
"rebuild": "node-gyp rebuild",
"upload": "prebuild --upload-all ${GITHUB_TOKEN}",
"test": "node test/support/createdb.js && mocha -R spec --timeout 480000"
},
"license": "BSD-3-Clause",
Expand Down
11 changes: 2 additions & 9 deletions tools/BinaryBuilder.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,12 @@ WORKDIR /usr/src/build
COPY . .
RUN npm install --ignore-scripts

# Workaround for https://github.com/mapbox/node-pre-gyp/issues/644
RUN cd node_modules/\@mapbox/node-pre-gyp \
&& npm install fs-extra@10.0.1 \
&& sed -i -e s/\'fs/\'fs-extra/ -e s/fs\.renameSync/fs.moveSync/ ./lib/util/napi.js

ENV CFLAGS="${CFLAGS:-} -include ../src/gcc-preinclude.h"
ENV CXXFLAGS="${CXXFLAGS:-} -include ../src/gcc-preinclude.h"
RUN npx node-pre-gyp configure
RUN npx node-pre-gyp build
RUN npm run prebuild

RUN if case $VARIANT in "alpine"*) false;; *) true;; esac; then ldd lib/binding/*/node_sqlite3.node; nm lib/binding/*/node_sqlite3.node | grep \"GLIBC_\" | c++filt || true ; fi
RUN if case $VARIANT in "alpine"*) false;; *) true;; esac; then ldd build/**/node_sqlite3.node; nm build/**/node_sqlite3.node | grep \"GLIBC_\" | c++filt || true ; fi

RUN npm run test
RUN npx node-pre-gyp package

CMD ["sh"]