feat(codegen): pluggable embedding system with --auto-embed flag [CI]#924
Merged
pyramation merged 12 commits intomainfrom Mar 28, 2026
Merged
feat(codegen): pluggable embedding system with --auto-embed flag [CI]#924pyramation merged 12 commits intomainfrom
pyramation merged 12 commits intomainfrom
Conversation
Tests generated CLI commands (codegen → transpile → execute) against a running PostgreSQL database with a real GraphQL server. - Uses ts.transpileModule to strip types without resolving imports - Uses async spawn (not execFileSync) to keep event loop unblocked - Sets up appstash context pointing at test server endpoint - Resolves NODE_PATH for pnpm's strict module isolation 5 focused corner-case tests: 1. Paginated list with --where (dot-notation) + --fields 2. Cursor-based forward pagination (--after) 3. find-first with --where.name.equalTo 4. Combined --where + --orderBy + --fields 5. Empty result set handling
…SE_DIR, bump appstash 0.7.0 - Type buildAnimalsTable() with proper Table interface from codegen - Replace internal path.join require hacks with proper package exports - Use APPSTASH_BASE_DIR env var instead of overriding HOME - Bump appstash to ^0.7.0 for APPSTASH_BASE_DIR support - Fix cliEntryPoint -> entryPoint config property
Suite 2 — Search CLI (6 tests): - tsvector search via --where.tsvTsv (dot-notation passthrough) - trgm fuzzy matching via --where.trgmTitle - composite fullTextSearch filter - search + pagination (--limit) - pgvector similarity (conditional, skip if unavailable) - _meta query from live server (MetaSchemaPlugin verification) Uses search-seed fixture (5 articles with tsvector, pg_trgm, optional pgvector). All search tests use list --where dot-notation to pass filter field names directly to the server, testing the full pipeline: codegen -> transpile -> spawn child process -> ORM findMany -> GraphQL -> real PostgreSQL.
- pgvector test: vector arrays can't be passed via CLI dot-notation (they become strings, not JSON arrays). Changed test to verify the CLI reports a clear GraphQL type error rather than crashing silently. - _meta test: replaced with schema introspection test since the search-seed server doesn't load MetaSchemaPlugin (enableServicesApi is false). New test verifies Article type exposes expected search fields (tsvRank, titleTrgmSimilarity, bodyTrgmSimilarity, searchScore, and conditionally pgvector fields).
The CLI exits with code 0 even on GraphQL errors, returning
{ ok: false, errors: [...] }. Updated the test to check the
response content instead of expecting the promise to reject.
…eral for runner script - Moved runCli() from inline per-suite to a shared module-level function that takes (distDir, tmpHome, ...args) — eliminates 60 lines of duplication - Replaced string[] array joined with newlines with a readable template literal (RUNNER_SCRIPT constant) - Fixed duplicate JSDoc comment on setupAppstashContext - Removed stale Suite 3 reference from file header
When a table has search-capable fields (tsvector, trgm, BM25, pgvector), the generated README and skill references now include concrete CLI examples showing the exact dot-notation flags for each search type: - tsvector: --where.<field> "query" - trgm: --where.trgm<Base>.value "query" --where.trgm<Base>.threshold 0.3 - BM25: --where.bm25<Base>.query "query" - pgvector: --where.<field>.vector '[...]' --where.<field>.distance 1.0 - composite: --where.fullTextSearch "query" Also adds a combined search + pagination example. Field-name derivation mirrors buildSearchHandler so examples always match the generated code. Integrated into all four generators: single-target README, single-target skills, multi-target README, and multi-target skills.
Adds a pluggable text-to-vector embedding system for CLI search commands: - New embedder.ts template using @agentic-kit/ollama for Ollama provider - resolveEmbedder() resolves from env vars or appstash config - autoEmbedWhere() converts text values to vector embeddings in where clauses - --auto-embed flag on list and search commands for tables with vector fields - Embedder module conditionally generated when tables have embedding fields - Updated docs-generator with --auto-embed examples for vector fields - CLI e2e tests for embedder: error without provider, real Ollama integration
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
devin-ai-integration Bot
pushed a commit
that referenced
this pull request
Apr 20, 2026
… config
Replace old grant_roles/grant_privileges/policy_* fields with unified:
- grants[]: array of { roles: string[], privileges: unknown[] } objects
- policies[]: array of { $type, data, privileges, policy_role, permissive } objects
Updated files:
- generate-types.ts: BlueprintEntityTableProvision + BlueprintTable
- relation-many-to-many.ts: parameter schema for junction table
- export-utils.ts: secure_table_provision column types
- export.test.ts: test data + snapshots
Companion to constructive-db PR #929 (grants[]) and PR #924 (policies[]).
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a pluggable text-to-vector embedding system for generated CLI commands. When a table has pgvector embedding fields (
Vectoror[Float]gqlType), the codegen now:Generates an
embedder.tsruntime module using@agentic-kit/ollama— providesresolveEmbedder()(env vars → appstash config → null) andautoEmbedWhere()to convert text values in vector where-clause fields to real embedding vectors.Injects
--auto-embedflag intolistandsearchhandlers via a newbuildAutoEmbedBlock()AST builder. When passed, the CLI resolves an embedder and converts text queries to vectors before sending to the GraphQL server.Extends
--auto-embedtocreateandupdatemutations viaautoEmbedInput()andbuildAutoEmbedInputBlock(). When--auto-embedis passed on a create/update command, text strings in vector fields are converted to embeddings before the ORM call. This is a CLI-only convenience until server-side triggers/job queues handle it.Generates field-specific search examples in docs (README, skills) showing
--auto-embedusage for vector fields (read and write paths), replacing the previous generic search examples.Adds CLI e2e tests — verifies error without
EMBEDDER_PROVIDER, plus conditional real Ollama/nomic-embed-text integration tests.Renames
--fieldsto--selectacross all CLI codegen, templates, docs, and e2e tests for consistency with the TS SDK'sselectparameter naming.Embedding is opt-in via
--auto-embed— no implicit behavior or field name clobbering. Configuration viaEMBEDDER_PROVIDER,EMBEDDER_MODEL,EMBEDDER_BASE_URLenv vars (or appstash config equivalents).Updates since last revision
--fields→--selectrename: All CLI usage text, doc examples, runtimeparseSelectFlag()(now readsargv.select), generated snapshots, and e2e tests updated for TS SDK consistency. The internal variable names (select,parseSelectFlag) were already correct — only the user-facing flag name changed.Review & Testing Checklist for Human
@agentic-kit/ollamaAPI:embedder.tsimportsOllamaClientas default export and callsclient.generateEmbedding(text, model). If this doesn't match the actual@agentic-kit/ollama@1.0.3API, every generated CLI embedder will break at runtime. This is the highest-risk item since it was written without runtime verification.{ vector: query }key for embedding fields:buildSearchHandlerchanged the where-clause key fromvaluetovectorfor embedding category fields (~line 693 of table-command-generator.ts). Confirm the graphile pgvector plugin expectsvectornotvalueas the input key for similarity search.autoEmbedInputmutation integration: The auto-embed block is injected into the mutation handler'stryBodyaftercleanedDatais created but before the ORM call.autoEmbedInputmutatesdatain-place (converts string values in vector fields to embedding arrays). Confirm the generated code positions the block correctly and that mutatingcleanedDatabefore passing it to the ORM is safe with the TypeScript type assertions (as CreateXxxInput[...]/as XxxPatch).if (argv['auto-embed']) { ... }block appears inhandleList,handleSearch,handleCreate, andhandleUpdate— and thathandleDeletedoes not include it.--selectrename: Verify that--select id,nameworks in a generated CLI command (replaces the old--fieldsflag). The runtime readsargv.selectnow — confirm minimist parses--select foointo{ select: 'foo' }without conflicts.Notes
@agentic-kit/ollamaresolution cascading through pnpm deduplication — not a code concern but worth a glance.autoEmbedInputis intentionally a temporary CLI-only convenience. The long-term plan is server-side triggers/job queues for embedding generation.buildSearchExamples/buildSearchExamplesMarkdownindocs-utils.tsnow include create/update examples alongside the existing search examples.--fields→--selectrename was done for consistency with the TS SDK'sselectparameter. All 7 affected snapshots were regenerated and pass.Link to Devin session: https://app.devin.ai/sessions/c92c3a11450342f8875625a60fa1be28
Requested by: @pyramation