refactor: extract CatalogDB interface for WASM portability#277
refactor: extract CatalogDB interface for WASM portability#277ako merged 3 commits intomendixlabs:mainfrom
Conversation
AI Code ReviewCritical IssuesNone found. Moderate IssuesNone found. Minor Issues
What Looks Good
RecommendationThe PR is ready to be merged. It successfully extracts the database interfaces needed for WASM portability while maintaining all existing functionality through careful, consistent refactoring. The minor concern about the type assertion in Approve the PR. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
There was a problem hiding this comment.
Pull request overview
This PR refactors catalog and linter database access behind new CatalogDB / CatalogTx interfaces (to enable non-database/sql-backed implementations later), and continues the executor cleanup by removing ExecContext’s Executor back-pointer in favor of explicit context fields + sync-back.
Changes:
- Introduce
CatalogDB/CatalogTxinterfaces and a native SQLite implementation (SqliteCatalogDB) plus test helpers/contract tests. - Update catalog + linter code (and tests) to consume
CatalogDBrather than*sql.DB. - Update executor handlers/tests to rely on
ExecContextstate +syncBackrather thanctx.executorback-references.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
mdl/catalog/catalogdb.go |
Adds CatalogDB / CatalogTx interfaces. |
mdl/catalog/catalogdb_sqlite.go |
Adds SqliteCatalogDB implementation (!js) + RawDB() + WrapSqlDB(). |
mdl/catalog/catalog.go |
Switches catalog to CatalogDB, adds NewFromDB, updates SaveToFile to use SQLite-specific escape hatch. |
mdl/catalog/builder.go |
Switches builder transaction type from *sql.Tx to CatalogTx. |
mdl/catalog/catalogdb_test.go |
Adds contract/interface satisfaction tests. |
mdl/linter/context.go |
Switches linter context DB field/accessors to catalog.CatalogDB. |
mdl/linter/rules/* |
Updates rule + tests to use CatalogDB and WrapSqlDB. |
mdl/executor/* |
Removes back-pointer usage, adds sync-back and explicit ExecContext callbacks/fields. |
Comments suppressed due to low confidence (1)
mdl/executor/cmd_catalog.go:466
- In
REFRESH CATALOG ... BACKGROUND, the goroutine builds the catalog using a shallow-copiedbgCtx. That avoids races, but it also meansbgCtx.Catalogis never propagated back to the live session/Executor, while the handler already clearedctx.Catalog. Net effect: after a background build finishes (and even prints "Catalog ready"), subsequent commands still seectx.Catalog == niluntil the user manually reloads from cache. If the intent is to keep background refresh usable in-session, consider returning the built catalog via a channel/callback to the main goroutine (or avoiding clearing the existing catalog until the replacement is ready) and then atomically swappingctx.Catalog/syncing state back.
// Close existing catalog if any
if ctx.Catalog != nil {
ctx.Catalog.Close()
ctx.Catalog = nil
}
// Handle background mode — clone ctx so the goroutine doesn't race
// with the main dispatch loop (which may syncBack and mutate fields).
// NOTE: bgCtx.Output still shares the underlying writer with the main
// goroutine. This is a pre-existing limitation — the original code also
// wrote to ctx.Output from the goroutine. A synchronized writer would
// fix this but is out of scope for the executor cleanup.
if stmt.Background {
bgCtx := *ctx // shallow copy — isolates scalar fields
bgCtx.Cache = nil // detach shared cache so preWarmCache writes stay local
go func() {
if err := buildCatalog(&bgCtx, stmt.Full, stmt.Source); err != nil {
fmt.Fprintf(bgCtx.Output, "Background catalog build failed: %v\n", err)
}
}()
fmt.Fprintln(ctx.Output, "Catalog build started in background...")
return nil
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
mdl/executor/cmd_catalog.go:466
REFRESH CATALOG ... BACKGROUNDnow builds the catalog on a shallow-copiedbgCtx, but the resultingbgCtx.Catalogis never propagated back to the live session/executor state. Because the handler also clearsctx.Catalogbefore spawning the goroutine, the session can end up with no in-memory catalog even after the background build completes (only the cache file is written). Consider adding an explicit sync-back mechanism for background work (e.g., aSetCatalog/SyncBackcallback onExecContext, or sending the built catalog back to the main goroutine via a channel) so the built catalog becomes available without requiring an extra reload command.
// Handle background mode — clone ctx so the goroutine doesn't race
// with the main dispatch loop (which may syncBack and mutate fields).
// NOTE: bgCtx.Output still shares the underlying writer with the main
// goroutine. This is a pre-existing limitation — the original code also
// wrote to ctx.Output from the goroutine. A synchronized writer would
// fix this but is out of scope for the executor cleanup.
if stmt.Background {
bgCtx := *ctx // shallow copy — isolates scalar fields
bgCtx.Cache = nil // detach shared cache so preWarmCache writes stay local
go func() {
if err := buildCatalog(&bgCtx, stmt.Full, stmt.Source); err != nil {
fmt.Fprintf(bgCtx.Output, "Background catalog build failed: %v\n", err)
}
}()
fmt.Fprintln(ctx.Output, "Catalog build started in background...")
return nil
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8a95512 to
7dc3cc6
Compare
ako
left a comment
There was a problem hiding this comment.
Code Review — PR #277 (retrospective, already merged)
Overview: Extracts a CatalogDB/CatalogTx interface pair so the catalog can be swapped from *sql.DB (SQLite) to a WASM-compatible backend without touching callers. Good incremental approach.
What works well
//go:build !jsoncatalogdb_sqlite.gocleanly isolates the SQLite dependency- Compile-time interface check
var _ CatalogDB = (*SqliteCatalogDB)(nil)is good practice WrapSqlDB()gives tests a realCatalogDBwithout mockingNewFromDB(CatalogDB)constructor is cleanSaveToFiletype-assertion is documented and safe for now
Design gap: CatalogTx is not WASM-portable
CatalogTx.Prepare() returns *sql.Stmt, a concrete database/sql type:
type CatalogTx interface {
Prepare(query string) (*sql.Stmt, error)
...
}A future JsCatalogTx cannot satisfy this interface without wrapping a real *sql.Stmt. That defeats the stated WASM portability goal — the interface will need revision before a JS backend can implement it. Consider either:
- Removing
Preparefrom the interface (inline the query instead), or - Introducing a
CatalogStmtinterface and returning that instead
Same issue applies to Query returning *sql.Rows and QueryRow returning *sql.Row — those are also concrete types. For now this is fine (incremental step), but worth tracking before the JS backend is written.
Minor points
WrapSqlDB()is exported but only used in tests (_test.gofiles). Consider making itwrapSqlDB(unexported) or moving it to a_test.gofile to avoid polluting the package surface.- Error message wrapping was removed from
NewSqliteCatalogDB— the caller now gets a bare SQLite error with no context. The old"failed to open in-memory database: %w"wrapper was useful.
Overall: Solid first step. The leaky abstraction is documented and acceptable for now, but CatalogTx.Prepare() returning *sql.Stmt should be addressed before implementing a real WASM backend.
Why
mdl/catalog/andmdl/linter/depend directly on*sql.DBandmodernc.org/sqlite. To compile mxcli as a WASM module (for embedding in Studio Pro's Maia client), the catalog layer needs to delegate SQL operations to the host process instead. This PR introduces an interface boundary so a futureJsCatalogDBimplementation can swap in without touching any consumer code.Changes
catalogdb.go—CatalogDBinterface (Query, QueryRow, Exec, Begin, Close) +CatalogTxinterface (Prepare, Exec, QueryRow, Commit, Rollback)catalogdb_sqlite.go—SqliteCatalogDBwrapping*sql.DB(//go:build !js), withRawDB()escape hatch andWrapSqlDB()test helpercatalog.go—db *sql.DB→CatalogDB,DB()→CatalogDB(),New()delegates toNewSqliteCatalogDB(),NewFromDB()constructor added,SaveToFileuses type assertionbuilder.go—tx *sql.Tx→CatalogTxlinter/context.go—db *sql.DB→catalog.CatalogDB,DB()→CatalogDB()rules/missing_translations.go—ctx.DB()→ctx.CatalogDB()*sql.DBviacatalog.WrapSqlDB(), contract tests for both interfaces, compile-time interface satisfaction checksDesign decisions
*sql.Rows/*sql.Rowto avoid rewriting 18 linter scan sitesCatalogTx.Prepare()returns*sql.Stmt— keeps Builder's bulk insert pattern unchangedSaveToFilestays SQLite-specific via type assertion (usesVACUUM INTO)JsCatalogDBstub deferred to follow-up PR — needs WASM bridge design