feat: consolidate search into graphile-search + rename filter arg to where#814
feat: consolidate search into graphile-search + rename filter arg to where#814pyramation merged 63 commits intomainfrom
Conversation
Implements a from-scratch PostGraphile v5 native connection filter plugin, replacing the upstream postgraphile-plugin-connection-filter dependency. New package: graphile/graphile-connection-filter/ Plugin architecture (7 plugins): - ConnectionFilterInflectionPlugin: filter type naming conventions - ConnectionFilterTypesPlugin: registers per-table and per-scalar filter types - ConnectionFilterArgPlugin: injects filter arg on connections via applyPlan - ConnectionFilterAttributesPlugin: adds per-column filter fields - ConnectionFilterOperatorsPlugin: standard/sort/pattern/jsonb/inet/array/range operators - ConnectionFilterCustomOperatorsPlugin: addConnectionFilterOperator API for satellite plugins - ConnectionFilterLogicalOperatorsPlugin: and/or/not logical composition Key features: - Full v5 native: uses Grafast planning, PgCondition, codec system, behavior registry - EXPORTABLE pattern for schema caching - Preserves addConnectionFilterOperator API for PostGIS, search, pgvector, textsearch plugins - No relation filter plugins (simplifies configuration vs upstream) - Preset factory: ConnectionFilterPreset(options) Also updates graphile-settings to use the new workspace package.
…Operator filterType is for table-level filter types (UserFilter), while filterFieldType is for scalar operator types (StringFilter). Satellite plugins pass scalar type names, so the lookup must use filterFieldType to match the registration in ConnectionFilterTypesPlugin. Previously worked by coincidence since both inflections produce the same output, but would silently fail if a consumer overrode one inflection but not the other.
Adds computed column filter support — allows filtering on PostgreSQL functions that take a table row as their first argument and return a scalar. Controlled by connectionFilterComputedColumns schema option. The preset factory includes the plugin only when the option is truthy (default in preset: true, but constructive-preset sets it to false).
- Remove phantom postgraphile-plugin-connection-filter dep from graphile-pgvector-plugin (never used) - Remove phantom postgraphile-plugin-connection-filter dep from graphile-pg-textsearch-plugin (never used) - Update graphile-plugin-connection-filter-postgis to use graphile-connection-filter workspace dep with typed imports - Update graphile-search-plugin to use graphile-connection-filter workspace dep with typed imports - Replace (build as any).addConnectionFilterOperator casts with properly typed build.addConnectionFilterOperator
…on-filter - Update search plugin, pgvector, and postgis test files to import from graphile-connection-filter instead of postgraphile-plugin-connection-filter - Use ConnectionFilterPreset() factory instead of PostGraphileConnectionFilterPreset - Import ConnectionFilterOperatorSpec type from graphile-connection-filter - Fix smart quote characters in filter descriptions to match existing snapshots
…ion filter tests - Add graphile-connection-filter as devDependency in graphile-pgvector-plugin (test file imports ConnectionFilterPreset but package had no dependency) - Skip connectionFilterRelations tests in search plugin (relation filters are intentionally not included in the v5-native plugin; they were disabled in production via disablePlugins with the old plugin)
…toggle - ConnectionFilterForwardRelationsPlugin: filter by FK parent relations - ConnectionFilterBackwardRelationsPlugin: filter by backward relations (one-to-one + one-to-many with some/every/none) - connectionFilterRelations toggle in preset (default: false) - Un-skip relation filter tests in search plugin - Updated augmentations, types, and exports
… at runtime The preset factory now always includes relation plugins in the plugin list. Each plugin checks build.options.connectionFilterRelations at runtime and early-returns if disabled. This allows the toggle to be set by any preset in the chain, not just the ConnectionFilterPreset() call.
Enables relation filter fields in the production schema: - Forward: filter by FK parent (e.g. clientByClientId on OrderFilter) - Backward: filter by children with some/every/none - Codegen will pick up the new filter fields automatically
- Search plugin: isPgCondition → isPgConnectionFilter scope
- BM25 plugin: isPgCondition → isPgConnectionFilter scope
- Disable PgConditionArgumentPlugin and PgConditionCustomFieldsPlugin in preset
- Update all tests from condition: {...} to filter: {...}
- Add graphile-connection-filter devDependency to BM25 plugin
- Update search plugin graceful degradation tests to use filter
BREAKING CHANGE: The condition argument has been removed entirely.
All filtering now uses the filter argument exclusively.
- Search plugin plugin.test.ts: condition → filter syntax, add ConnectionFilterPreset - Server-test: condition → filter in query with equalTo operator - Clear stale snapshots (schema-snapshot, introspection) for regeneration
- Search plugin: update snapshot keys to match renamed filter-based tests - Schema snapshot: remove all condition arguments and XxxCondition input types - Introspection snapshot: remove condition arg and UserCondition type - Kept conditionType in _meta schema (unrelated to deprecated condition arg)
… behavior for pgCodecRelation, update schema snapshot with relation filter types
… types to schema snapshot
…, and proper type ordering
…y filter at applyPlan level
Top-level empty filter {} is now treated as 'no filter' (skipped) instead of
throwing an error. Nested empty objects in and/or/not and relation filters are
still rejected. This removes the need for the connectionFilterAllowEmptyObjectInput
workaround in pgvector tests.
- Extract shared getQueryBuilder utility into graphile-connection-filter/src/utils.ts - Remove duplicate getQueryBuilder from search, BM25, and pgvector plugins - Replace (build as any).dataplanPg with build.dataplanPg (already typed on Build) - Replace (build as any).behavior with build.behavior (already typed on Build) - Replace (build as any).input.pgRegistry with build.input.pgRegistry (already typed) - Remove scope destructuring as any casts (pgCodec already typed on ScopeInputObject) - Add pgCodec comment to augmentations.ts noting it's already declared by graphile-build-pg - Export getQueryBuilder from graphile-connection-filter for satellite plugin use
Adds index safety check for relation filter fields. When enabled (default: true), relation filter fields are only created for FKs with supporting indexes. This prevents generating EXISTS subqueries that would cause sequential scans on large tables. Uses PgIndexBehaviorsPlugin's existing relation.extensions.isIndexed metadata which is set at gather time. The check runs at schema build time with zero runtime cost. Applied to both forward and backward relation filter plugins.
Comprehensive test coverage using graphile-test infrastructure: - Scalar operators: equalTo, notEqualTo, distinctFrom, isNull, in/notIn, lessThan, greaterThan, like, iLike, includes, startsWith, endsWith - Logical operators: and, or, not, nested combinations - Relation filters: forward (child->parent), backward one-to-one, backward one-to-many (some/every/none), exists fields - Computed column filters - Schema introspection: filter types, operator fields, relation fields - Options toggles: connectionFilterRelations, connectionFilterComputedColumns, connectionFilterLogicalOperators, connectionFilterAllowedOperators, connectionFilterOperatorNames Also adds graphile/graphile-connection-filter to CI matrix (41 jobs).
Exercises multiple plugins working together in a single test database: - Connection filter (scalar operators, logical operators, relation filters) - PostGIS spatial filters (geometry column) - pgvector (vector column, search function, distance ordering) - tsvector search plugin (fullText matches, rank, orderBy) - BM25 search (pg_textsearch body index, score, orderBy) - Kitchen sink queries combining multiple plugins 34 test cases across 8 describe blocks, all passing locally. Added postgres-plus CI job for tests requiring PostGIS/pgvector/pg_textsearch.
…tion filter + scalar in one query
… test The mega query now exercises all SIX plugin types in a single filter: - tsvector (fullTextTsv) - BM25 (bm25Body) - relation filter (category name) - scalar filter (isActive) - pgvector (vectorEmbedding nearby) - PostGIS (geom intersects polygon bbox) Also validates returned coordinates fall within the bounding box.
New package: graphile-pg-trgm-plugin — a PostGraphile v5 plugin for pg_trgm trigram-based fuzzy text matching. Zero config, works on any text column. Features: - similarTo / wordSimilarTo filter operators on StringFilter - trgm<Column> direct filter fields on connection filter types - <column>Similarity computed score fields (0-1, null when inactive) - SIMILARITY_<COLUMN>_ASC/DESC orderBy enum values - TrgmSearchPreset for easy composition into presets - connectionFilterTrgmRequireIndex option (default: false) - 14 dedicated tests + integrated into mega query as 7th plugin type Mega query now exercises ALL 7 plugin types in one GraphQL query: tsvector + BM25 + pgvector + PostGIS + pg_trgm + relation filter + scalar
Updated introspection and SDL snapshots to include new fields from TrgmSearchPlugin: similarTo/wordSimilarTo operators on StringFilter, *Similarity computed fields, trgm* filter fields, and SIMILARITY_* orderBy enum values.
- orderBy: [BM25_BODY_SCORE_ASC, SIMILARITY_NAME_DESC] demonstrates multi-signal relevance ranking in a single query - Added comprehensive JSDoc explaining all 7 plugin types, the 2-phase meta system, and ORDER BY priority semantics - Inline GraphQL comments explain each filter and score field - Assertion verifies BM25 ASC ordering (primary sort) - Documents important subtlety: ORDER BY priority follows schema field processing order, not the orderBy array order
New standalone package: graphile-search
Architecture:
- Single plugin iterates over SearchAdapter implementations
- 4 adapters: tsvector (ts_rank), bm25 (pg_textsearch), trgm (similarity), pgvector (distance)
- Composite searchScore field (normalized 0..1) aggregating all active search signals
- Per-algorithm score fields: {column}{Algorithm}{Metric}
- OrderBy enums: {COLUMN}_{ALGORITHM}_{METRIC}_ASC/DESC + SEARCH_SCORE
- Filter fields on connection filter input types
NOT added to ConstructivePreset yet — standalone package for testing/evaluation.
Old plugins remain completely untouched.
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…1773437586-graphile-search
…o prevent CI hang
- Fix 'Must not call build.getTypeByName before init phase is complete' error by wrapping type registrations in try/catch instead of using getTypeByName guard - Add TsvectorCodecPlugin to test preset (was missing, causing 'Could not build PgCodec for pg_catalog.tsvector' warning and tsvector adapter failing to detect columns) - Add graphile-tsvector as devDependency for test imports
…site filter - Rename tsvector adapter default filterPrefix from 'fullText' to 'tsv' for consistency - Add supportsTextSearch property and buildTextSearchInput() method to SearchAdapter interface - Set supportsTextSearch: true on tsvector, bm25, trgm adapters; false on pgvector - Add enableFullTextSearch option (default: true) to UnifiedSearchOptions - Implement fullTextSearch composite filter field in plugin.ts that fans out plain text queries to all text-compatible adapters with OR logic - Update all tests: rename fullTextTsv -> tsvTsv, add fullTextSearch tests
…ons, ConnectionFilterPreset)
…ed test to separate suite
…ores unknown input fields)
| const fragment = resolve(sqlIdentifier, sqlValue, value, $where, { | ||
| fieldName: parentFieldName ?? null, | ||
| operatorName: fieldName, | ||
| }); | ||
|
|
||
| $where.where(fragment); |
There was a problem hiding this comment.
🔴 trgm operator resolve returns null which is passed unchecked to $where.where()
The similarTo and wordSimilarTo operators in graphile-trgm/src/preset.ts:38-41 and graphile-trgm/src/preset.ts:63-66 can return null when the input value is an empty string (e.g., { value: "" }). The TrgmSearchInput.value field is String! (non-null), but an empty string passes GraphQL validation. When resolve returns null, graphile-connection-filter/src/plugins/operatorApply.ts:110 calls $where.where(null) without checking for null, which will pass null to PgCondition.where() — a method that expects a SQL fragment. This can cause a runtime error or produce invalid SQL.
Trace through the code path
- User sends
filter: { name: { similarTo: { value: "" } } } operatorApply.ts:45-89validates the outervalueobject (non-null ✓)operatorApply.ts:105callsresolve(sqlIdentifier, sqlValue, {value: "", threshold: undefined}, ...)preset.ts:40:!valueevaluates!""→true→ returnsnulloperatorApply.ts:110:$where.where(null)is called
| const fragment = resolve(sqlIdentifier, sqlValue, value, $where, { | |
| fieldName: parentFieldName ?? null, | |
| operatorName: fieldName, | |
| }); | |
| $where.where(fragment); | |
| // Generate the WHERE clause fragment and apply it | |
| const fragment = resolve(sqlIdentifier, sqlValue, value, $where, { | |
| fieldName: parentFieldName ?? null, | |
| operatorName: fieldName, | |
| }); | |
| if (fragment != null) { | |
| $where.where(fragment); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
graphile/graphile-trgm/src/preset.ts
Outdated
| schema: { | ||
| connectionFilterOperatorFactories: [ | ||
| createTrgmOperatorFactories(), | ||
| ], |
There was a problem hiding this comment.
🟡 Duplicate connectionFilterOperatorFactories from sub-presets are silently lost when used standalone
Each sub-preset (PgSearchPreset, TrgmSearchPreset, GraphilePostgisPreset) defines its own connectionFilterOperatorFactories array. When these presets are composed via extends, graphile-config replaces arrays (not concatenates). In ConstructivePreset this is handled by explicitly collecting all factories at graphile/graphile-settings/src/presets/constructive-preset.ts:159-163. However, if a user composes the sub-presets directly (e.g., extends: [PgSearchPreset(), TrgmSearchPreset(), GraphilePostgisPreset]), only the last preset's factories survive — the earlier presets' matches and similarTo/wordSimilarTo operators would be silently dropped. The TrgmSearchPreset() preset at graphile/graphile-trgm/src/preset.ts:96-99 and PgSearchPreset() at graphile/graphile-tsvector/src/preset.ts:72-75 each declare their factories independently with no guidance that they'll be overwritten when combined.
Was this helpful? React with 👍 or 👎 to provide feedback.
- Move codec plugins into graphile-search/src/codecs/ (tsvector, bm25, pgvector) - Move operator factories (createMatchesOperatorFactory, createTrgmOperatorFactories) - Update all imports to use internal codecs (no cross-package deps) - Add tree-shakable exports from graphile-search index - Update UnifiedSearchPreset to include codec plugins + operator factories - Update ConstructivePreset to import from graphile-search - Update graphile-settings plugins/index.ts re-exports - Delete graphile-tsvector, graphile-bm25, graphile-trgm, graphile-pgvector - Remove deleted packages from CI config - Add mega query v2 (fullTextSearch + searchScore with composite ordering)
- Rename fullTextTsv -> tsvTsv in preset-integration tests (filter prefix changed) - Delete stale snapshots for server-test and graphql/test (will auto-regenerate)
- Update trgm score descriptions to concise format - Update trgm filter descriptions to concise format - Update threshold description (remove verbose example text) - Add searchScore field to Post, Tag, User, Comment types - Add fullTextSearch filter to PostFilter, TagFilter, UserFilter, CommentFilter - Add new orderBy enums (TITLE_TRGM, CONTENT_TRGM, EXCERPT_TRGM, etc.) - Add SEARCH_SCORE_ASC/DESC orderBy enums
…terArgumentName) - Add connectionFilterArgumentName option to ConnectionFilterPreset (default: 'where') - Update ConnectionFilterArgPlugin to use configurable argument name - Update codegen to remove argName:'filter' translation layer (SDK where maps directly) - Update graphql/query package (ast.ts, select.ts) to use where argument - Update all test GraphQL queries from filter: to where: - Delete stale snapshots for CI regeneration
feat: consolidate search into graphile-search + rename filter arg to where
Summary
Two major changes in this PR:
1. Unified Search Plugin — Package Consolidation
Consolidates 4 individual search packages (
graphile-tsvector,graphile-bm25,graphile-trgm,graphile-pgvector) into the singlegraphile-searchpackage. All 4 packages are deleted with no deprecation — codec plugins, operator factories, and adapters now live insidegraphile-searchwith tree-shakable exports.2. Configurable
connectionFilterArgumentName(default:'where')Renames the GraphQL connection filter argument from
filtertowhereby default. The argument name is configurable via theconnectionFilterArgumentNameschema option. Since the GraphQL argument now defaults towhere(matching the SDK'swhereparameter), the codegen translation layer (argName: 'filter') is removed — no more SDK↔GraphQL mismatch.What changed for the
whererename:graphile-connection-filter/src/types.ts— AddedconnectionFilterArgumentName?: stringtoConnectionFilterOptionsgraphile-connection-filter/src/augmentations.ts— Added option toSchemaOptionsinterfacegraphile-connection-filter/src/plugins/ConnectionFilterArgPlugin.ts— Reads option with'where'default, uses dynamic arg namegraphile-connection-filter/src/preset.ts— Default set toconnectionFilterArgumentName: 'where'graphql/codegen/.../query-builder.ts— RemovedargName: 'filter'from 2addVariablecalls (lines ~248, ~364)graphql/query/src/ast.ts— Changed variable name fromfiltertowheregraphql/query/src/generators/select.ts— Changed argument from'filter'to'where'functions/send-email-link/src/index.ts— Updated 3 hardcoded GraphQL queries fromcondition:/filter:towhere:with proper filter syntax (e.g.{ id: { equalTo: $id } })jobs/knative-job-service/__tests__/jobs.e2e.test.ts— UpdatedjobByIdQueryfromfilter:towhere:filter:→where:in GraphQL query strings (5 test files)whereargument namegraphile-settings/__tests__/preset-integration.test.ts— Fixed assertion:toContain('filter')→toContain('where')What was deleted (search consolidation)
graphile/graphile-tsvector/— entire packagegraphile/graphile-bm25/— entire packagegraphile/graphile-trgm/— entire packagegraphile/graphile-pgvector/— entire packagegraphile-search-pluginandgraphile-pgvector-pluginWhat was moved into
graphile-search/src/codecs/tsvector-codec.ts— tsvector/tsquery type registration (from graphile-tsvector)src/codecs/bm25-codec.ts— BM25 index discovery + codec (from graphile-bm25)src/codecs/vector-codec.ts— pgvector type registration (from graphile-pgvector)src/codecs/operator-factories.ts—createMatchesOperatorFactory+createTrgmOperatorFactories(from graphile-tsvector/graphile-trgm)src/codecs/index.ts— barrel re-exportsArchitecture (unchanged from v1)
Tree-shakable imports
Generated GraphQL fields (unchanged from v1)
{algorithm}{Column}(e.g.bm25Body,trgmTitle,vectorEmbedding)fullTextSearch: String— fans out to all text-compatible adapters{column}{Algorithm}{Metric}(e.g.bodyBm25Score,titleTrgmSimilarity){COLUMN}_{ALGORITHM}_{METRIC}_ASC/DESC+SEARCH_SCORE_ASC/DESCsearchScore: Normalized 0..1 aggregating all active search signalsUpdates since last revision
Fixed knative-job-service CI failures — The
filter→whererename broke 2 locations that had hardcoded GraphQL queries:functions/send-email-link/src/index.ts— 3 queries (GetUser,GetDatabaseInfo,siteModules) were usingcondition:orfilter:and needed to be updated towhere:with proper filter syntax (e.g.condition: { id: $userId }→where: { id: { equalTo: $userId } })jobs/knative-job-service/__tests__/jobs.e2e.test.ts— ThejobByIdQueryusedfilter:and needed to be updated towhere:The knative test was failing because the
send-email-linkfunction would make GraphQL queries withfilter:(which returned HTTP 400), causing the job to fail before it could call thesimple-emailfunction. This cascaded into the "Missing required field 'subject'" error because the email subject was never generated.CI is now fully green — 42/42 passing (previously reported 41/41 with knative failing).
Review & Testing Checklist for Human
whereargument rename is acceptable — This is a breaking change for existing GraphQL clients. Clients usingfilter:will need to update towhere:or configureconnectionFilterArgumentName: 'filter'to maintain backward compatibilitysend-email-linkfunction had hardcoded queries that broke. Are there other deployed services or scripts with similar hardcodedfilter:queries that aren't covered by CI?filter:→where:). CI is passing, but review the snapshot diffs to ensure no unintended schema changesgraphile-tsvector,graphile-bm25,graphile-trgm,graphile-pgvector)filtertowhereTest Plan
pnpm buildcompleted successfully with no type errorswhereargumentfilter:queries in send-email-link function)where:argument in GraphQL queries works correctlyconnectionFilterArgumentNameoption with custom valuesconnectionFilterArgumentName: 'filter'Notes
filtertowhere. Configurable viaconnectionFilterArgumentNameschema option for backward compatibilitysend-email-linkfunction had hardcoded GraphQL queries usingcondition:/filter:arguments that caused HTTP 400 errors. These are now fixed and usewhere:with proper filter syntaxfilter:→where:argument references. CI passing validates the edits are correctwhere→ GraphQLfilterviaargName: 'filter'. Since GraphQL now useswhereby default, the translation is removedUserFilter,StringFilter, etc. — only the argument name changed towhereDevin Session: https://app.devin.ai/sessions/cf88f3fd383b4421a5169ed01612899d
Requested by: @pyramation