BREAKING CHANGE: update: migrate to better-sqlite3

[UnschooledGamer]

Why

sqlite3 has received no updates in the past 2-3 years and has blockers on package updating, like tar/node-tar, which has vulnerabilities.

Better-sqlite3 is benchmarked to be faster, better, and more updated in comparison to sqlite3.

---

I've tested as much as I could; breaking changes could occur because of how the better-sqlite3 API is without callbacks.

To Do

The update schema script needs changes.

[greptile-apps]

Greptile Summary

This PR migrates the database layer from sqlite3 (callback-based) to better-sqlite3 (synchronous), resolving security vulnerabilities in outdated dependencies and improving performance.

Key Changes:

• Replaced callback-based db.all() and db.get() with synchronous prepared statements (db.prepare().all() and .get())
• Write operations (INSERT/UPDATE/DELETE) now return empty arrays instead of result objects for backward compatibility
• Added directory creation logic in server/lib/db.js to prevent startup failures
• Fixed server/apis/comment.js to fetch newly inserted comment after insert (since insert no longer returns the row)
• Added manual cascade deletion in Plugin.deletePermanently() to handle related records (comments, downloads, purchase orders)
• Fixed type conversions: OTP values stringified, boolean-to-integer for is_default, increment values stringified
• Improved error handling and null safety across OTP and user APIs
• Client-side improvements: fixed comment ID handling and added deletion success feedback

Migration Quality:

• The migration is well-executed with proper synchronous error handling
• The developer correctly identified and fixed all callback-to-sync conversions
• Type conversions have been handled appropriately for SQLite's type system
• Previous review comments about syntax errors and missing closing braces were incorrect - the code structure is valid

Confidence Score: 4/5

• This PR is generally safe to merge with careful testing of database operations
• The migration is well-implemented with proper error handling and backward compatibility. However, the breaking changes to the database layer require thorough testing of all CRUD operations, especially edge cases around concurrent inserts and type conversions. The manual cascade deletion logic needs verification that it covers all foreign key relationships.
• Pay close attention to server/entities/entity.js (core database abstraction), server/entities/plugin.js (manual cascade logic), and server/apis/comment.js (post-insert fetch pattern)

Important Files Changed

Filename	Overview
server/lib/db.js	Migrated from sqlite3 to better-sqlite3 with synchronous API, added directory creation logic and foreign key enforcement
server/entities/entity.js	Replaced callback-based DB calls with synchronous prepared statements, write operations now return empty arrays for compatibility
server/apis/comment.js	Fixed comment insertion to fetch the newly created comment ID after insert (previously relied on insert return value)
server/apis/plugin.js	Added await to Order.insert, stringified increment value to preserve integer type, added default for minVersionCode
server/apis/user.js	Fixed boolean-to-integer conversion for is_default, added OTP validation check, improved null safety for OTP row
server/entities/plugin.js	Added manual cascade deletion of related comments, downloads, and purchase orders before plugin deletion
updateSchema.js	Migrated to synchronous db.exec() and db.close() with proper error handling for duplicate columns

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[sqlite3 with callbacks] -->|Migration| B[better-sqlite3 synchronous]
    B --> C[Database Layer<br/>server/lib/db.js]
    C --> D{Foreign Keys Enabled}
    D -->|Yes| E[Entity Layer<br/>server/entities/entity.js]
    E --> F[execSql: Prepared Statements]
    F --> G{Statement Type}
    G -->|SELECT| H[stmt.all returns rows]
    G -->|INSERT/UPDATE/DELETE| I[stmt.run returns empty array]
    I --> J[API Layer Adaptations]
    H --> J
    J --> K[Comment API<br/>Fetch after insert]
    J --> L[Plugin API<br/>Manual cascades]
    J --> M[User/OTP APIs<br/>Type conversions]

Loading

_{Last reviewed commit: 6b9dcad}

[greptile-apps]

_{4 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[UnschooledGamer]

@greptileai Review it again.

[greptile-apps]

_{10 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

server/apis/plugin.js
Missing await on insert

In the paid-download flow, Order.insert(...) is called without await (and without handling the returned Promise). With the new DB layer being synchronous under the hood but still wrapped in Promises, this means the request can proceed (and later logic can observe no order) before the insert completes, and errors from the insert will be unhandled. Make this await Order.insert(...) (or otherwise handle/rethrow failures).

[UnschooledGamer]

@greptileai Review it again.

[greptile-apps]

_{5 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Write operations (INSERT/UPDATE/DELETE) now return empty array instead of the actual result info (changes, lastInsertRowid). If callers ever need the insert ID or affected row count, they won't have access to it. This could break code that relies on these values.

[UnschooledGamer]

This can be handled in future if needed, Currently it returns empty array for compatibility

[deadlyjack]

@greptileai Review changes.

[deadlyjack]

@greptileai review again

[greptile-apps]

_{15 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

server/entities/plugin.js
execSql called without required values and entity arguments

Entity.execSql(sql) is called with only one argument. Inside execSql, values is undefined and entity is undefined. When the query runs, stmt.all(...values) will throw TypeError: undefined is not iterable. Additionally, if an error is caught, entity.table will itself throw TypeError: Cannot read properties of undefined, masking the original error and crashing the server.

This will break the earnings calculation jobs in fixUserEarnings.js and updateEarnings.js, both of which call Plugin.getUsersWithPlugin().

    return Entity.execSql(sql, [], this);

[UnschooledGamer]

@greptileai Review Once Again.