Skip to content

Commit

Permalink
feat(libsql): add libSQL driver (#5417)
Browse files Browse the repository at this point in the history
Adds a new `@mikro-orm/libsql` package.

https://github.com/tursodatabase/libsql

Closes #5283
  • Loading branch information
B4nan committed Apr 4, 2024
1 parent 54a37d0 commit 6c63e4b
Show file tree
Hide file tree
Showing 43 changed files with 1,172 additions and 986 deletions.
26 changes: 15 additions & 11 deletions README.md
Expand Up @@ -2,7 +2,7 @@
<a href="https://mikro-orm.io"><img src="https://raw.githubusercontent.com/mikro-orm/mikro-orm/master/docs/static/img/logo-readme.svg?sanitize=true" alt="MikroORM" /></a>
</h1>

TypeScript ORM for Node.js based on Data Mapper, [Unit of Work](https://mikro-orm.io/docs/unit-of-work/) and [Identity Map](https://mikro-orm.io/docs/identity-map/) patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite databases.
TypeScript ORM for Node.js based on Data Mapper, [Unit of Work](https://mikro-orm.io/docs/unit-of-work/) and [Identity Map](https://mikro-orm.io/docs/identity-map/) patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite (including libSQL) databases.

> Heavily inspired by [Doctrine](https://www.doctrine-project.org/) and [Hibernate](https://hibernate.org/).
Expand Down Expand Up @@ -175,21 +175,25 @@ First install the module via `yarn` or `npm` and do not forget to install the da
> Since v4, you should install the driver package, but not the db connector itself, e.g. install `@mikro-orm/sqlite`, but not `sqlite3` as that is already included in the driver package.
```sh
yarn add @mikro-orm/core @mikro-orm/mongodb # for mongo
yarn add @mikro-orm/core @mikro-orm/mysql # for mysql/mariadb
yarn add @mikro-orm/core @mikro-orm/mariadb # for mysql/mariadb
yarn add @mikro-orm/core @mikro-orm/postgresql # for postgresql
yarn add @mikro-orm/core @mikro-orm/sqlite # for sqlite
yarn add @mikro-orm/core @mikro-orm/mongodb # for mongo
yarn add @mikro-orm/core @mikro-orm/mysql # for mysql/mariadb
yarn add @mikro-orm/core @mikro-orm/mariadb # for mysql/mariadb
yarn add @mikro-orm/core @mikro-orm/postgresql # for postgresql
yarn add @mikro-orm/core @mikro-orm/sqlite # for sqlite
yarn add @mikro-orm/core @mikro-orm/better-sqlite # for better-sqlite
yarn add @mikro-orm/core @mikro-orm/libsql # for libsql
```

or

```sh
npm i -s @mikro-orm/core @mikro-orm/mongodb # for mongo
npm i -s @mikro-orm/core @mikro-orm/mysql # for mysql/mariadb
npm i -s @mikro-orm/core @mikro-orm/mariadb # for mysql/mariadb
npm i -s @mikro-orm/core @mikro-orm/postgresql # for postgresql
npm i -s @mikro-orm/core @mikro-orm/sqlite # for sqlite
npm i -s @mikro-orm/core @mikro-orm/mongodb # for mongo
npm i -s @mikro-orm/core @mikro-orm/mysql # for mysql/mariadb
npm i -s @mikro-orm/core @mikro-orm/mariadb # for mysql/mariadb
npm i -s @mikro-orm/core @mikro-orm/postgresql # for postgresql
npm i -s @mikro-orm/core @mikro-orm/sqlite # for sqlite
npm i -s @mikro-orm/core @mikro-orm/better-sqlite # for better-sqlite
npm i -s @mikro-orm/core @mikro-orm/libsql # for libsql
```

Next, if you want to use decorators for your entity definition, you will need to enable support for [decorators](https://www.typescriptlang.org/docs/handbook/decorators.html) as well as `esModuleInterop` in `tsconfig.json` via:
Expand Down
1 change: 1 addition & 0 deletions docs/docs/configuration.md
Expand Up @@ -145,6 +145,7 @@ To select driver, you can either use `type` option, or provide the driver class
| `postgresql` | `PostgreSqlDriver` | `pg` | compatible with CockroachDB |
| `sqlite` | `SqliteDriver` | `sqlite3` | - |
| `better-sqlite` | `BetterSqliteDriver` | `better-sqlite3` | - |
| `libsql` | `LibSqlDriver` | `libsql` | - |

> Driver and connection implementations are not directly exported from `@mikro-orm/core` module. You can import them from the driver packages (e.g. `import { PostgreSqlDriver } from '@mikro-orm/postgresql'`).
Expand Down
3 changes: 3 additions & 0 deletions docs/docs/quick-start.md
Expand Up @@ -26,6 +26,9 @@ npm install @mikro-orm/core @mikro-orm/sqlite

# for better-sqlite
npm install @mikro-orm/core @mikro-orm/better-sqlite

# for libsql
npm install @mikro-orm/core @mikro-orm/libsql
```

Next you will need to enable support for [decorators](https://www.typescriptlang.org/docs/handbook/decorators.html) as well as `esModuleInterop` in `tsconfig.json` via:
Expand Down
2 changes: 1 addition & 1 deletion docs/src/pages/index.js
Expand Up @@ -37,7 +37,7 @@ const features = [
imageUrl: 'img/icons/creative-idea.svg',
description: (
<>
Supports <Link to="/docs/usage-with-mongo">MongoDB</Link>, <Link to="/docs/usage-with-sql">MySQL, MariaDB, PostgreSQL and SQLite</Link> databases, and more can be supported via custom drivers right now.
Supports <Link to="/docs/usage-with-mongo">MongoDB</Link>, <Link to="/docs/usage-with-sql">MySQL, MariaDB, PostgreSQL and SQLite (including libSQL)</Link> databases, and more can be supported via custom drivers right now.
</>
),
},
Expand Down
199 changes: 3 additions & 196 deletions packages/better-sqlite/src/BetterSqliteConnection.ts
@@ -1,203 +1,10 @@
import { ensureDir, readFile } from 'fs-extra';
import { dirname } from 'path';
import { AbstractSqlConnection, MonkeyPatchable, type Knex } from '@mikro-orm/knex';
import { Utils, type Dictionary } from '@mikro-orm/core';
import { BetterSqliteKnexDialect, BaseSqliteConnection } from '@mikro-orm/knex';

export class BetterSqliteConnection extends AbstractSqlConnection {
export class BetterSqliteConnection extends BaseSqliteConnection {

override createKnex() {
this.getPatchedDialect();
this.client = this.createKnexClient('better-sqlite3');
this.client = this.createKnexClient(BetterSqliteKnexDialect as any);
this.connected = true;
}

override async connect(): Promise<void> {
this.createKnex();
await ensureDir(dirname(this.config.get('dbName')!));
await this.client.raw('pragma foreign_keys = on');
}

getDefaultClientUrl(): string {
return '';
}

override getClientUrl(): string {
return '';
}

override async loadFile(path: string): Promise<void> {
const conn = await this.client.client.acquireConnection();
await conn.exec((await readFile(path)).toString());
await this.client.client.releaseConnection(conn);
}

protected override getKnexOptions(type: string): Knex.Config {
return Utils.mergeConfig({
client: type,
connection: {
filename: this.config.get('dbName'),
},
pool: this.config.get('pool'),
useNullAsDefault: true,
}, this.config.get('driverOptions'));
}

protected transformRawResult<T>(res: any, method: 'all' | 'get' | 'run'): T {
if (method === 'get') {
return res[0];
}

if (method === 'all') {
return res;
}

if (Array.isArray(res)) {
return {
insertId: res[res.length - 1]?.id ?? 0,
affectedRows: res.length,
row: res[0],
rows: res,
} as T;
}

return {
insertId: res.lastInsertRowid,
affectedRows: res.changes,
} as unknown as T;
}

/**
* monkey patch knex' BetterSqlite Dialect so it returns inserted id when doing raw insert query
*/
private getPatchedDialect() {
const { Sqlite3Dialect, Sqlite3DialectTableCompiler } = MonkeyPatchable;

if (Sqlite3Dialect.prototype.__patched) {
return Sqlite3Dialect;
}

const processResponse = Sqlite3Dialect.prototype.processResponse;
Sqlite3Dialect.prototype.__patched = true;
Sqlite3Dialect.prototype.processResponse = (obj: any, runner: any) => {
if (obj.method === 'raw' && this.isRunQuery(obj.sql)) {
return obj.response ?? obj.context;
}

return processResponse(obj, runner);
};

Sqlite3Dialect.prototype._query = (connection: any, obj: any) => {
const callMethod = this.getCallMethod(obj);

return new Promise((resolve: any, reject: any) => {
/* istanbul ignore if */
if (!connection?.[callMethod]) {
return reject(new Error(`Error calling ${callMethod} on connection.`));
}

connection[callMethod](obj.sql, obj.bindings, function (this: any, err: any, response: any) {
if (err) {
return reject(err);
}

obj.response = response;
obj.context = this;

return resolve(obj);
});
});
};

/* istanbul ignore next */
Sqlite3DialectTableCompiler.prototype.foreign = function (this: typeof Sqlite3DialectTableCompiler, foreignInfo: Dictionary) {
foreignInfo.column = this.formatter.columnize(foreignInfo.column);
foreignInfo.column = Array.isArray(foreignInfo.column)
? foreignInfo.column
: [foreignInfo.column];
foreignInfo.column = foreignInfo.column.map((column: unknown) =>
this.client.customWrapIdentifier(column, (a: unknown) => a),
);
foreignInfo.inTable = this.client.customWrapIdentifier(
foreignInfo.inTable,
(a: unknown) => a,
);
foreignInfo.references = Array.isArray(foreignInfo.references)
? foreignInfo.references
: [foreignInfo.references];
foreignInfo.references = foreignInfo.references.map((column: unknown) =>
this.client.customWrapIdentifier(column, (a: unknown) => a),
);
// quoted versions
const column = this.formatter.columnize(foreignInfo.column);
const inTable = this.formatter.columnize(foreignInfo.inTable);
const references = this.formatter.columnize(foreignInfo.references);
const keyName = this.formatter.columnize(foreignInfo.keyName);

const addColumnQuery = this.sequence.find((query: { sql: string }) => query.sql.includes(`add column ${column[0]}`));

// no need for temp tables if we just add a column
if (addColumnQuery) {
const onUpdate = foreignInfo.onUpdate ? ` on update ${foreignInfo.onUpdate}` : '';
const deleteRule = foreignInfo.deleteRule ? ` on delete ${foreignInfo.deleteRule}` : '';
addColumnQuery.sql += ` constraint ${keyName} references ${inTable} (${references})${onUpdate}${deleteRule}`;
return;
}

// eslint-disable-next-line @typescript-eslint/no-this-alias
const compiler = this;

if (this.method !== 'create' && this.method !== 'createIfNot') {
this.pushQuery({
sql: `PRAGMA table_info(${this.tableName()})`,
statementsProducer(pragma: any, connection: any) {
return compiler.client
.ddl(compiler, pragma, connection)
.foreign(foreignInfo);
},
});
}
};

return Sqlite3Dialect;
}

private isRunQuery(query: string): boolean {
query = query.trim().toLowerCase();

if ((query.startsWith('insert into') || query.startsWith('update ')) && query.includes(' returning ')) {
return false;
}

return query.startsWith('insert into') ||
query.startsWith('update') ||
query.startsWith('delete') ||
query.startsWith('truncate');
}

private getCallMethod(obj: any): string {
if (obj.method === 'raw') {
const query = obj.sql.trim().toLowerCase();

if ((query.startsWith('insert into') || query.startsWith('update ')) && query.includes(' returning ')) {
return 'all';
}

if (this.isRunQuery(query)) {
return 'run';
}
}

/* istanbul ignore next */
switch (obj.method) {
case 'insert':
case 'update':
return obj.returning ? 'all' : 'run';
case 'counter':
case 'del':
return 'run';
default:
return 'all';
}
}

}
90 changes: 2 additions & 88 deletions packages/better-sqlite/src/BetterSqlitePlatform.ts
@@ -1,85 +1,15 @@
// @ts-ignore
import { escape } from 'sqlstring-sqlite';
import { JsonProperty, Utils, type EntityProperty } from '@mikro-orm/core';
import { AbstractSqlPlatform } from '@mikro-orm/knex';
import { BaseSqlitePlatform } from '@mikro-orm/knex';
import { BetterSqliteSchemaHelper } from './BetterSqliteSchemaHelper';
import { BetterSqliteExceptionConverter } from './BetterSqliteExceptionConverter';

export class BetterSqlitePlatform extends AbstractSqlPlatform {
export class BetterSqlitePlatform extends BaseSqlitePlatform {

protected override readonly schemaHelper: BetterSqliteSchemaHelper = new BetterSqliteSchemaHelper(this);
protected override readonly exceptionConverter = new BetterSqliteExceptionConverter();

override usesDefaultKeyword(): boolean {
return false;
}

override usesReturningStatement(): boolean {
return true;
}

override getCurrentTimestampSQL(length: number): string {
return super.getCurrentTimestampSQL(0);
}

override getDateTimeTypeDeclarationSQL(column: { length: number }): string {
return 'datetime';
}

override getEnumTypeDeclarationSQL(column: { items?: unknown[]; fieldNames: string[]; length?: number; unsigned?: boolean; autoincrement?: boolean }): string {
if (column.items?.every(item => Utils.isString(item))) {
return 'text';
}

return this.getTinyIntTypeDeclarationSQL(column);
}

override getTinyIntTypeDeclarationSQL(column: { length?: number; unsigned?: boolean; autoincrement?: boolean }): string {
return this.getIntegerTypeDeclarationSQL(column);
}

override getSmallIntTypeDeclarationSQL(column: { length?: number; unsigned?: boolean; autoincrement?: boolean }): string {
return this.getIntegerTypeDeclarationSQL(column);
}

override getIntegerTypeDeclarationSQL(column: { length?: number; unsigned?: boolean; autoincrement?: boolean }): string {
return 'integer';
}

override getFloatDeclarationSQL(): string {
return 'real';
}

override getBooleanTypeDeclarationSQL(): string {
return 'integer';
}

override getVarcharTypeDeclarationSQL(column: { length?: number }): string {
return 'text';
}

override convertsJsonAutomatically(): boolean {
return false;
}

override allowsComparingTuples() {
return false;
}

/**
* This is used to narrow the value of Date properties as they will be stored as timestamps in sqlite.
* We use this method to convert Dates to timestamps when computing the changeset, so we have the right
* data type in the payload as well as in original entity data. Without that, we would end up with diffs
* including all Date properties, as we would be comparing Date object with timestamp.
*/
override processDateProperty(value: unknown): string | number | Date {
if (value instanceof Date) {
return +value;
}

return value as number;
}

override quoteVersionValue(value: Date | number, prop: EntityProperty): Date | string | number {
if (prop.runtimeType === 'Date') {
return escape(value, true, this.timezone).replace(/^'|\.\d{3}'$/g, '');
Expand All @@ -101,20 +31,4 @@ export class BetterSqlitePlatform extends AbstractSqlPlatform {
return escape(value, true, this.timezone);
}

override getIndexName(tableName: string, columns: string[], type: 'index' | 'unique' | 'foreign' | 'primary' | 'sequence'): string {
if (type === 'primary') {
return this.getDefaultPrimaryName(tableName, columns);
}

return super.getIndexName(tableName, columns, type);
}

override getDefaultPrimaryName(tableName: string, columns: string[]): string {
return 'primary';
}

override supportsDownMigrations(): boolean {
return false;
}

}

0 comments on commit 6c63e4b

Please sign in to comment.