Tabularis Decoupling RFC

[NewtTheWolf]

Tabularis Decoupling RFC

Note

Category: Idea / RFC. A survey of what we could decouple, how, and why — plus what's probably not worth it. This is a starting point to spark discussion, not a finished plan. Pushback and alternatives very welcome.

Why decouple at all

The core keeps absorbing knowledge that really belongs to a specific driver, feature, or transport. Symptoms we hit recently:

• The same dialect knowledge lives in core in several places — e.g. three overlapping SQL keyword lists (SQL_RESERVED, SQL_KEYWORDS, sqlSplitter/classify.ts) that already drift (fix(autocomplete): suggest clause keywords and correct columns after … #295), and per-dialect pagination branches threaded through core (PaginationDialect::{MySql,Postgres,Sqlite}, MySQL LIMIT off,cnt, FETCH FIRST…) in Derive pagination LIMIT/OFFSET from a SQL parser #286.
• src-tauri/src/commands.rs is ~152 KB in one file — a coupling magnet every feature has to touch.
• Big optional subsystems (AI ~80 KB across ai*.rs, tunnels, export/dump) are wired straight into the core even though not every user needs them.

Important

Goal: a thin, dialect-agnostic core with clear seams — so adding a driver, transport, or feature is a self-contained change instead of edits sprinkled across shared files.

How to think about it (principles)

Important

Find the seam first. Most of this is "define a capability contract" before moving code. Decoupling without a clean interface just relocates the mess.

1. Three tiers of decoupling — pick the lightest that fits:
   • Module boundary (same binary, clear interface) — cheapest, for internal cohesion.
   • Feature gate (compile-time / runtime toggle) — for optional subsystems (AI, tunnels).
   • Plugin (out-of-process, versioned API) — only where 3rd parties add real value (drivers, exporters).
2. Incremental, one concern at a time. Extract keyword lists → pagination → quoting → catalog, not a big-bang rewrite.
3. Don't decouple for its own sake. If there's one impl and no 3rd-party value, a module boundary beats a plugin.

---

The list

Group 1 — Driver / dialect knowledge → a driver capability contract

Note

The foundational seam. Define "what every driver must provide", then move core's dialect special-cases behind it.

#	What	Why	How
1	SQL keyword / reserved-word sets	3 drifting copies in core; every dialect's keywords bloat shared code	Each driver exposes its keyword/reserved set; autocomplete + alias parsing read it from the active driver
2	Pagination / LIMIT-OFFSET rewriting	build_paginated_query special-cases each dialect in core (#286)	Driver provides paginate(query, page, size) (or a dialect descriptor); core just calls it
3	Identifier quoting + schema/database model	backticks vs ", MySQL "database == schema" vs PG schemas leak into core (the multiDb mess in #295)	Driver capability: quote(), schema model, qualifier rules
4	Catalog / metadata queries	information_schema vs native catalog hardcoded per path	Driver owns its catalog queries; core asks for tables/columns/types abstractly
5	Type formatting / blob / safe-int handling	drivers/common/{blob,safe_int}.rs mix generic + dialect bits	Generic stays in core; dialect-specific rendering moves to driver

Group 2 — Backend monolith decomposition

Note

Mostly module boundaries + feature gates here, not plugins.

#	What	Why	How
6	Split commands.rs (~152 KB)	one file every feature edits → merge pain, unclear ownership	Split by domain: connection / query / schema / dump / export / ai command modules
7	AI subsystem (ai*.rs, ~80 KB)	large, optional, not all users want it; couples core to an AI provider	Feature-gate it; put it behind a capability/module so a build can ship without it
8	Connectivity / tunnels (ssh_tunnel.rs, k8s_tunnel.rs)	each transport is bespoke; adding one touches core	A TunnelProvider interface; SSH/k8s become implementations; new transports plug in
9	Export / dump / explain-import	feature subsystems entangled with the driver core	Behind a capability; decouple "how to read data" from "how to write a file/format"
10	Pool / connection management (pool_manager.rs)	sits between drivers and commands	Keep core generic; per-driver pool quirks move to the driver capability

Group 3 — Frontend

#	What	Why	How
11	Dialect-aware editor logic (Editor.tsx, autocomplete, sqlAnalysis.ts)	hardcodes keywords/quoting; should follow the active driver	Consume Group-1 capabilities via a hook instead of static lists
12	Component clusters (modals/ ~592 KB, ui/ ~420 KB)	large bundles, eager-loaded	Split per feature + lazy-load; keep DataGrid generic (no dialect logic)
13	Provider/context sprawl (~12 context pairs)	many fine-grained contexts	Mostly OK already; group only where they always change together

Group 4 — The enabling seam

#	What	Why	How
14	Driver capability contract	the interface that makes Group 1 + 11 possible	Define the trait/manifest: keywords, quoting, pagination, catalog, types, capabilities flags. Build this first.
15	Extend the plugin host beyond UI slots	plugin-api already has hooks/slots/host — extend to driver-level capabilities	Reuse the existing plugin runtime; drivers become first-class plugins (bundled or wizard-installed)

---

What makes sense first

Tip

Suggested order — small, high-signal steps before the big restructuring.

1. Feature Request: ER Diagram for Single Table with Related Tables #14 capability contract — nothing else is clean without it.
2. New Feature: Edit Cells - Commit/Rollback #1 keyword sets — smallest, highest signal, kills the 3-list drift.
3. Not loading on macos. #2 pagination + build: Add openssl dependency with vendored feature #3 quoting/schema — the dialect special-cases that bit us in Derive pagination LIMIT/OFFSET from a SQL parser #286/fix(autocomplete): suggest clause keywords and correct columns after … #295.
4. Does not list databases, have to manually enter database name in connection window #6 split commands.rs — independent, pure maintainability win, can run in parallel.
5. feat(custom-openai): add support for custom OpenAI-compatible API endpoints #7 AI feature-gate + Support for OpenAI-compatible Endpoints #8 tunnel interface — optionality wins.
6. Then Database explorer only shows public scheme on Postgres #15 (drivers-as-plugins) once the contract has proven itself internally.

What's probably not worth decoupling

Caution

Decoupling has a cost. These are where it likely isn't worth it — flagging them so the RFC doesn't slide into over-engineering.

• i18n — already cleanly separated.
• Plugin transport (JSON-RPC over stdio) — works; leave it.
• The grid/render layer — keep it generic; don't push dialect logic into it, but don't pluginize the renderer either.
• Turning every internal module into an out-of-process plugin — module boundaries + feature gates cover most of this at a fraction of the cost. Reserve plugins for drivers/exporters where 3rd parties genuinely extend the product.

Open questions

Warning

Unresolved — these need answers before any of the above becomes a concrete proposal.

1. Where exactly is the core/driver line — does parsing/autocomplete stay in core and read dialect info, or move wholesale into drivers?
2. How do plugins expose dialect info to the UI hot path (autocomplete) without an IPC round-trip — static manifest vs. capability descriptors?
3. Bundled vs. wizard-installed drivers: which ship by default, and what's the first-run/offline UX?
4. Plugin API versioning so core and drivers evolve independently.
5. Backwards compatibility for existing connections and bundles.

---

Starting point. If the direction lands, each row above can become its own small, concrete proposal.

[debba]

Thanks for writing this up @NewtTheWolf — it's a good map of the tension, and the symptoms you list are real (the keyword-list drift in #295 and the pagination branches in #286 are exactly the kind of thing that makes me wince every time I open those files).

Before I dig in: none of what follows is a decision. It's my read as a maintainer, and it's one input into the discussion — not a verdict. I'd rather we converge on these in the open than have me close them, so push back freely where you see it differently.

(Note: the N) below refer to the numbered items in your RFC, not to GitHub issues.)

One framing correction before the specifics, because I think it changes how big this looks: the capability contract (14) isn't greenfield. We already have DatabaseDriver + a declarative PluginManifest/DriverCapabilities that carries a lot of the dialect knowledge this RFC wants to move — identifier_quote, sql_dialect, auto_increment_keyword, serial_type, inline_pk, schemas, readonly, file_based, and so on. And plugins are already first-class drivers (Mongo, ClickHouse, DuckDB, Db2, CSV all live as out-of-tree repos).

The cleanest proof it works is happening right now: the SQL Server driver on feat/sql-server (cc @saurabh500) is a fourth dialect being added almost entirely through the manifest — it declares its splitter dialect and quoting instead of editing core branches. So I'd reframe the work as "move the special-cases that still live in core behind the contract we already have" rather than "build the contract." That makes the scope smaller and the order safer.

On priorities I'm basically with you:

• 1) keyword sets and 6) splitting commands.rs are the two I'd start tomorrow. They're independent, low-risk, and pure wins — 1) kills the drift, 6) is just maintainability and can run in parallel with everything else.
• 2) pagination + 3) quoting/schema next, since they're half-done in the manifest already.
• 7. (feature-gate AI) and 8) (TunnelProvider) are worth it but cost more.
• Agreed on the "don't bother" list. I don't want dialect logic in the grid and I don't want to pluginize the renderer or the transport.

On the open questions — again, these are positions to argue with, not calls I'm making:

1. Where's the core/driver line for parsing/autocomplete?
Hybrid, but with a heavy bias toward declarative. The parsing/autocomplete engine stays in core (one implementation, synchronous, fast); drivers contribute dialect data, not dialect code. The trap I want us to avoid: the moment a capability becomes a behavioral call (paginate(query, …), classify(stmt)) instead of data, an out-of-process plugin pays an IPC round-trip — and for pagination that's once per query. So the rule I'd write down is: if the behavior has a small enumerable shape, express it as a descriptor in the manifest and let core do the rewrite; reserve actual behavioral calls for things that genuinely can't be enumerated. Pagination and quoting fit the descriptor model; I don't want a paginate() IPC hop. The other thing to watch is drift in the opposite direction — if we start letting drivers ship parser logic, we'll grow N half-correct parsers. Engine stays in one place.

2. How does a plugin expose dialect info to the autocomplete hot path without IPC?
Static manifest, read once when the driver activates and cached on the frontend. We already serialize DriverCapabilities to the UI; this is the same channel. Nothing should touch IPC on a keystroke.

3. Bundled vs wizard-installed drivers.
I'm fine with "everything is conceptually a plugin" as the end state — but conforming to the contract is not the same as being out-of-process. The native drivers stay compiled in and pre-installed in the bundle — and note they already run on a mix of crates (sqlx for MySQL/SQLite, tokio-postgres for Postgres, tiberius/mssql-tiberius-bridge for SQL Server), which is itself the argument for keeping the contract implementation-agnostic. They're the hot path, first-run/offline has to just work with zero downloads, and they simply sit behind the same manifest/trait as everyone else; third-party drivers stay out-of-process over JSON-RPC.

One thing I'd explicitly want layered on top: separation of code and enablement are different axes. I'd like a first-run "which databases do you use?" picker that enables the bundled natives (instant toggle, still fully offline) and offers to install third-party ones from the registry (the download path). It declutters the connection modal and gives the registry a natural home. The one line I'd hold firm: enabling a bundled driver must never need a network round-trip — only third-party installs do.

Bonus: dogfooding the contract on the drivers we use most — across three different client crates — is the best forcing function for keeping it honest.

4. Plugin API versioning.
SemVer on the manifest plus a minimum host version — extend the assertHostCompat / MIN_HOST_VERSION check that's already in plugin-api. Each plugin declares the API version it targets, core warns or refuses below the floor. No runtime feature negotiation until we actually need it.

5. Backwards compatibility.
We already have the pattern, it just needs to become explicit policy:

• Driver IDs stay stable (mysql/postgres/sqlite), so saved connections keep resolving even after natives become "plugins."
• New fields are additive with defaults, never breaking — sql_dialect already falls back to postgres for older plugins, and DatabaseSelection uses untagged serde for the same reason. That's the template.
• The minimum host version from 4) is the safety net for the day we do need a hard break.

That's my two cents, not the final word — if the direction broadly lands I'm happy to turn 1) and 6) into concrete issues so we have something specific to chew on, but I'd like to hear where people disagree first.