older versions of sqlite

[jackdouglas]

Following the guide I can use a customized version of SQLite3 with better-sqlite3, but only for recent versions. Older versions like 3.16.2 fall over with errors like this:

npm ERR! ./src/objects/database.lzz: In static member function 'static void Database::JS_new(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/database.lzz:182:55: error: 'SQLITE_DBCONFIG_DEFENSIVE' was not declared in this scope; did you mean 'SQLITE_DBCONFIG_LOOKASIDE'?
npm ERR! ./src/objects/database.lzz: In static member function 'static void Database::JS_serialize(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/database.lzz:285:39: error: 'sqlite3_serialize' was not declared in this scope; did you mean 'sqlite3_strlike'?
npm ERR! ./src/objects/database.lzz: In static member function 'static void Database::JS_function(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/database.lzz:313:42: error: 'SQLITE_DIRECTONLY' was not declared in this scope; did you mean 'SQLITE_READONLY'?
npm ERR! ./src/objects/database.lzz: In static member function 'static void Database::JS_aggregate(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/database.lzz:342:42: error: 'SQLITE_DIRECTONLY' was not declared in this scope; did you mean 'SQLITE_READONLY'?
npm ERR! ./src/objects/database.lzz:345:21: error: 'sqlite3_create_window_function' was not declared in this scope; did you mean 'sqlite3_create_function'?
npm ERR! ./src/objects/database.lzz: In static member function 'static void Database::JS_unsafeMode(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/database.lzz:412:50: error: 'SQLITE_DBCONFIG_DEFENSIVE' was not declared in this scope; did you mean 'SQLITE_DBCONFIG_LOOKASIDE'?
npm ERR! ./src/objects/database.lzz: In static member function 'static bool Database::Deserialize(v8::Local<v8::Object>, Addon*, sqlite3*, bool)':
npm ERR! ./src/objects/database.lzz:427:38: error: 'SQLITE_DESERIALIZE_FREEONCLOSE' was not declared in this scope
npm ERR! ./src/objects/database.lzz:427:71: error: 'SQLITE_DESERIALIZE_RESIZEABLE' was not declared in this scope
npm ERR! ./src/objects/database.lzz:430:34: error: 'SQLITE_DESERIALIZE_READONLY' was not declared in this scope
npm ERR! ./src/objects/database.lzz:440:30: error: 'sqlite3_deserialize' was not declared in this scope; did you mean 'sqlite3_initialize'?
npm ERR! ./src/objects/statement.lzz: In static member function 'static void Statement::JS_new(const v8::FunctionCallbackInfo<v8::Value>&)':
npm ERR! ./src/objects/statement.lzz:98:29: error: 'SQLITE_PREPARE_PERSISTENT' was not declared in this scope
npm ERR! ./src/objects/statement.lzz:110:21: error: 'sqlite3_prepare_v3' was not declared in this scope; did you mean 'sqlite3_prepare_v2'?
npm ERR! ./src/util/custom-table.lzz: At global scope:
npm ERR! ./src/util/custom-table.lzz:45:9: error: too many initializers for 'sqlite3_module'
npm ERR! ./src/util/custom-table.lzz:72:9: error: too many initializers for 'sqlite3_module'
npm ERR! ./src/util/custom-table.lzz: In static member function 'static int CustomTable::xConnect(sqlite3*, void*, int, const char* const*, sqlite3_vtab**, char**)':
npm ERR! ./src/util/custom-table.lzz:200:67: error: 'SQLITE_VTAB_DIRECTONLY' was not declared in this scope; did you mean 'SQLITE_OPEN_READONLY'?

my dockerfile for the above is:

FROM alpine:3.16
RUN apk add --no-cache openrc util-linux haveged nodejs
RUN apk add --no-cache --virtual .build-deps unzip python3 make g++ npm \
  && wget https://www.sqlite.org/2017/sqlite-amalgamation-3160200.zip -O sqlite.zip \
  && unzip -j sqlite.zip \
  && rm sqlite.zip \
  && sed -i '1i#define SQLITE_DQS 0' sqlite3.c \
  && sed -i '1i#define SQLITE_THREADSAFE 0' sqlite3.c \
  && sed -i '1i#define SQLITE_DEFAULT_MEMSTATUS 0' sqlite3.c \
  && sed -i '1i#define SQLITE_DEFAULT_SYNCHRONOUS 0' sqlite3.c \
  && sed -i '1i#define SQLITE_LIKE_DOESNT_MATCH_BLOBS 1' sqlite3.c \
  && sed -i '1i#define SQLITE_OMIT_DEPRECATED 1' sqlite3.c \
  && sed -i '1i#define SQLITE_OMIT_SHARED_CACHE 1' sqlite3.c \
  && sed -i '1i#define SQLITE_USE_ALLOCA 1' sqlite3.c \
  && npm install -g better-sqlite3 --build-from-source --sqlite3=/ \
	&& apk del --no-network .build-deps
ENTRYPOINT ["sh"]

I've been able to compile SQLite 3.16.2 on the same alpine version:

FROM alpine:3.16
RUN apk add --no-cache openrc util-linux haveged sqlite-dev nodejs
RUN apk add --no-cache --virtual .build-deps alpine-sdk python3 npm
RUN wget http://www.sqlite.org/2017/sqlite-autoconf-3160200.tar.gz -O sqlite.tar.gz \
  && mkdir /sqlite \
  && tar -zxvf sqlite.tar.gz -C /sqlite --strip-components=1 \
  && rm sqlite.tar.gz
RUN sed -i '1i#define SQLITE_DQS 0' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_THREADSAFE 0' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_DEFAULT_MEMSTATUS 0' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_DEFAULT_SYNCHRONOUS 0' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_LIKE_DOESNT_MATCH_BLOBS 1' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_OMIT_DEPRECATED 1' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_OMIT_SHARED_CACHE 1' /sqlite/sqlite3.c \
  && sed -i '1i#define SQLITE_USE_ALLOCA 1' /sqlite/sqlite3.c
RUN /sqlite/configure --prefix=/usr --disable-static --enable-fts5 --enable-json1 \
    CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1" \ 
  && make \
  && make install
ENTRYPOINT ["sh"]

Is there anything I can do to get old versions compiling with better-sqlite3, or is there a way of linking to the version I'm able to compile independently?

[jackdouglas]

I've got 3.27 working :

  && sed -i '1i#define SQLITE_DIRECTONLY 0' sqlite3.h \
  && sed -i '1i#define SQLITE_VTAB_DIRECTONLY 1' sqlite3.h \
  && sed -i '1i#define SQLITE_ENABLE_DESERIALIZE 1' sqlite3.c \

SQLITE_ENABLE_DESERIALIZE is needed because of a change in defaults:

This option was formerly used to enable the sqlite3_serialize() and sqlite3_deserialize() interfaces. However, as of SQLite 3.36.0 (2021-06-18) those interfaces are enabled by default and a new compile-time option SQLITE_OMIT_DESERIALIZE is added to omit them.

The other two I assume are just new options.

[jackdouglas]

I don't think this approach is going to work for 3.16 as sqlite3_serialize() doesn't exist that far back. I can get 3.15 with v1.3.9 and 3.18 with v2.2.0 but am I correct that the option to compile from source didn't exist until recently so I won't be able to use either of those versions as a base for getting 3.16 working?

[JoshuaWise]

Since better-sqlite3 exposes features that only exist in modern versions of SQLite, such as db.serialize, older versions won't work. You would need to fork better-sqlite3 and remove all features that don't exist in the version of SQLite you're using.

[JoshuaWise]

Possibly related to #855

[jackdouglas]

Thanks @JoshuaWise that makes sense :)