Skip to content

v0.6.0

Choose a tag to compare

View all tags
@chtitux chtitux released this 20 Apr 18:18
· 14 commits to main since this release
v0.6.0
4df5c6d

Breaking changes — pluggable database adapter

The library now talks to a small async GtfsDatabase interface. sql.js becomes one adapter among others (better-sqlite3 ships in the box; op-sqlite / expo-sqlite / … pluggable by the user). Three things change at every call site: (1) query methods return Promise<T>, (2) an adapter is required, (3) sql.js is an optional peer dependency — you install it yourself.

See the full migration write-up in README and the Usage Guide.

New features

  • New src/adapters/types.ts public surface: GtfsDatabase, GtfsStatement, GtfsDatabaseAdapter, SqlValue, Row, ExportNotSupportedError.
  • sql.js adapter at subpath gtfs-sqljs/adapters/sql-js (exports createSqlJsAdapter, wrapSqlJsDatabase). The core module no longer imports sql.js.
  • better-sqlite3 adapter at subpath gtfs-sqljs/adapters/better-sqlite3 (exports wrapBetterSqlite3, createBetterSqlite3Adapter). First-class Node / file-backed path; the adapter is the only file in the repo that imports better-sqlite3, so projects that do not reference this subpath never pull in the native module. Exercised by tests/e2e-better-sqlite3.test.ts on every CI run.
  • Cache layer now catches ExportNotSupportedError from adapters that cannot serialize in-memory and logs a warning instead of failing the load; file-backed drivers persist their own DB on disk.

Performance

  • Ingestion is ~35-45% faster on medium-to-large feeds: ASTUCE (Rouen, ~430k stop_times rows) drops from ~2650 ms to ~1670 ms; Car Jaune from ~312 ms to ~188 ms. Wins come from parsing each CSV only once (progress totals now use a fast newline-based row-count estimate), loading rows as positional arrays instead of per-row objects, and reusing a single prepared INSERT per table instead of re-preparing a multi-row statement per 1000-row batch.
  • Dropped the bulk-load PRAGMA block (synchronous, journal_mode, temp_store, cache_size, locking_mode) from ingestion. Benchmarked aggregate effect on sql.js is within noise (≤1%); removing them simplifies the code and unblocks the pluggable adapter.

Behaviour changes

  • ProgressInfo.totalRows is now an estimate based on CSV line count — typically exact, but may differ by a few rows per file in edge cases (e.g. trailing blank lines). For a precise post-ingest row count, query the database directly with COUNT(*).
Assets 2
Loading