Skip to content

feat: remove cjs support, move to esm#24

Merged
JackDevAU merged 13 commits into
mainfrom
feat/remove-cjs-fix-bugs
May 12, 2026
Merged

feat: remove cjs support, move to esm#24
JackDevAU merged 13 commits into
mainfrom
feat/remove-cjs-fix-bugs

Conversation

@JackDevAU
Copy link
Copy Markdown
Member

@JackDevAU JackDevAU commented Dec 4, 2025

This is a major version change

Description:

Migrate from cjs to esm, replace Jest with Vitest, fix SQL injection vulnerabilities, improve type safety, and add comprehensive test coverage.

What was Added/Changed:

Package.json

  • Added "type": "module" and exports field for ESM support
  • Replaced Jest with Vitest (vitest: ^2.1.0)
  • Removed @tinacms/scripts, ts-jest, jest and related dependencies
  • Removed sucrase peer dependency
  • Added typedoc: ^0.28.15 for documentation generation
  • Updated scripts: build now uses tsc directly, test uses vitest run

src/index.ts

  • Fixed SQL injection vulnerability by using parameterized queries instead of string interpolation
  • Changed _batch to use SQLite transactions with prepared statements
  • Changed _put to use INSERT ... ON CONFLICT (upsert) instead of plain INSERT
  • Added _close methods to all iterators to properly release resources
  • Fixed LIMIT clause to use parameterized binding
  • Added UNIQUE constraint on the key column in table creation
  • Removed unused client property from iterator classes
  • Improved type safety: NextCallback defined locally, params typed as unknown[]

Tests

  • Migrated from Jest to Vitest
    • Iterator options (gt, gte, lt, lte, reverse, limit)
    • Read-only mode
    • Clear with options
    • Edge cases (empty strings, special characters, unicode, long values)
    • Upsert behavior verification
    • File-based persistence
    • SQL injection prevention
    • Iterator closing behavior

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Dec 4, 2025

🦋 Changeset detected

Latest commit: 108883f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
sqlite-level Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Dec 4, 2025

CLA assistant check
All committers have signed the CLA.

@JackDevAU JackDevAU force-pushed the feat/remove-cjs-fix-bugs branch from 13b0849 to 1af81e7 Compare December 4, 2025 05:38
@JackDevAU JackDevAU force-pushed the feat/remove-cjs-fix-bugs branch from 1af81e7 to b47374c Compare December 4, 2025 08:03
JackDevAU and others added 3 commits December 4, 2025 18:08
*  _clear was broken — only handled gte via a LIKE prefix match. Rewrote to support all range options (gt, gte, lt, lte) and clear-all with no options, matching the abstract-level contract.
* limit: 0 was falsy — if (options.limit) skipped limit: 0, returning everything instead of nothing. Fixed to options.limit != null && options.limit >= 0.
* gt/gte/lt/lte falsy checks — empty string or 0 values were silently skipped. Changed to != null checks. Also refactored condition building to use a conditions[] array.
* stmt.iterate(params) — changed to stmt.iterate(...params) for explicit spread rather than relying on better-sqlite3 implicit array handling.
* async _close — removed misleading async keyword from iterator _close methods since they're synchronous callback-based.
* _clear missing readOnly guard — added the same LEVEL_READ_ONLY check that _put, _del, and _batch have.
* Dead main/types fields in package.json — removed; redundant with the exports map now that CJS is dropped.

  Added test coverage

  Went from 55 → 66 tests. Added:
  - 7 _clear tests (clear-all, gt, gte, lt, lte, two range combos)
  - _clear readOnly test
  - limit: 0 test
  - Key iterator _close via break
  - Value iterator _close via break
  - Updated SQL injection clear tests to match new range semantics

  Accepted / left as-is

  UNIQUE constraint migration — major version bump covers it.
@github-actions
Copy link
Copy Markdown
Contributor

📦 Tagged Release Published

Your tagged PR has been published! You can install the packages using their version tags:

pnpm add sqlite-level@0.0.0-20260511052325

Published Packages:

  • sqlite-level@0.0.0-20260511052325

Commit: 3c86961403dbeafdf67291712def0a864b643762

@github-actions
Copy link
Copy Markdown
Contributor

📦 Tagged Release Published

Your tagged PR has been published! You can install the packages using their version tags:

pnpm add sqlite-level@0.0.0-20260511053032

Published Packages:

  • sqlite-level@0.0.0-20260511053032

View on npm:

Commit: d540def6749e7ac0b18d5fea4c4c47e2161daf32

@JackDevAU JackDevAU requested a review from wicksipedia May 11, 2026 07:00
JackDevAU added a commit to tinacms/tinacms that referenced this pull request May 11, 2026
The prerelease lands the named `SqliteLevel` export from
tinacms/sqlite-level#24, so the CJS namespace-import workaround
in @tinacms/search can go. The new package is pure ESM (only an
`import` export condition), so jest-resolve needs to be in ESM
mode to find it — added `extensionsToTreatAsEsm: ['.ts']` to the
search package's jest config to flip that switch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kulesy added a commit to tinacms/tinacms that referenced this pull request May 14, 2026
Closes #6675
Part of #6750

## Problem

TinaCMS v3's CLI bundles `tina/database.ts` with esbuild and dynamically
imports the result. Since the December 2025 ESM migration, this is
wedged between two failure modes:

1. **Bundling** native modules (`better-sqlite3` via `sqlite-level`)
crashes — `bindings` reads `__filename`/`__dirname`, which don't exist
in ESM scope. See #6675.
2. **Externalizing** native modules from `os.tmpdir()` doesn't work
either — Node's resolver walks up from the importing file's directory
and never reaches the project's `node_modules`.

## Fix

Two architectural changes plus the API + ergonomics work that comes out
of them:

### Build cache moved into the project tree

- `loadDatabaseFile` and `loadConfigFile` now write to
`<project>/tina/__generated__/.cache/<timestamp>/` instead of
`os.tmpdir()`, so Node can resolve `node_modules` from the bundle's
location at runtime.
- Each load removes its own subdir after the dynamic-import resolves,
and `ConfigManager.processConfig()` sweeps `.cache/` on startup to clean
up residue from crashed prior runs (Ctrl+C mid-build, OOM kills).
- The empty `<timestamp>/` parent is also reaped after both loads
complete (using `rmdirSync`, so concurrent in-flight loads don't
conflict).
- Read-only project mounts (Docker `:ro`, AWS Lambda `/var/task`,
sandboxed CI runners) now fail with an actionable error explaining the
cause and how to resolve it, instead of a cryptic mid-build `EACCES`
from esbuild.

### Externalize curated baseline + user-extensible

- `loadDatabaseFile` externalizes `better-sqlite3` so Node loads it as
CJS where `__filename` exists. The `sqlite-level` wrapper stays bundled
so esbuild handles its CJS named-export interop.
- New `defineConfig` field: `build.externalDependencies?: string[]`.
Users with custom native adapters outside the baseline can extend the
externalize list from their config — discoverable, type-safe,
version-controlled. Supersedes the env-var approach proposed in #6683.
- The merge logic and baseline live in `external-resolver.ts` with 9
unit tests.

### `tina init` template

- New projects (and existing projects without it) get
`tina/__generated__` added to `.gitignore`. Belt + suspenders for the
cache cleanup.

## Verification

Manually verified end-to-end against
`examples/next/tina-self-hosted-demo` temporarily reconfigured to use
`SqliteLevel`:

- Bundle lands in
`<project>/tina/__generated__/.cache/<timestamp>/database/database.build.mjs`
(not `/tmp/`)
- Node loads `bindings.js`'s `__filename`-based path successfully
- `better-sqlite3` native binary loads
- SqliteLevel instantiates, opens the DB file, and gets through to
content indexing
- Cache directory is empty after the build

The remaining indexing crash is a known SQL-injection bug in
sqlite-level **v1**, fixed by
[tinacms/sqlite-level#24](tinacms/sqlite-level#24)
— tracked as a follow-up in #6848. Not in scope for this PR.

The pre-existing mongo demo is unchanged so its CI coverage is
preserved. A standalone SQLite example with full CI matrix is filed as
#6786.

## Tests

- 9 new unit tests on the externalize merge logic
(`external-resolver.test.ts`)
- All 180 CLI tests pass (was 171 pre-PR — 9 new)
- Build clean, type-check clean (via `pnpm types`), format clean

## Follow-ups (filed)

- #6786 — Add self-hosted SQLite example + PR-blocking CI matrix
- #6787 — Add unit tests for `config-manager.ts` (output location, cache
cleanup, watch list, etc.)
- #6788 — Adapter compatibility test matrix
- #6789 — Promote starter-template smoke subset to PR-blocking
- #6847 — Migrate `mongodb-level` to ESM
- #6848 — Bump `sqlite-level` to v2.x in `@tinacms/search`
- #6849 — Expand the externalize baseline as adapters migrate

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JackDevAU <57518417+JackDevAU@users.noreply.github.com>
Co-authored-by: kulesy <kulewidak@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Eli Kent [SSW] <69125238+kulesy@users.noreply.github.com>
Co-authored-by: JackDevAU <jpr178@live.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants