Skip to content

Register missing SQL V2 relevance functions in Calcite function table#5440

Merged
dai-chen merged 2 commits into
opensearch-project:mainfrom
dai-chen:feature/add-missing-relevance-functions-calcite
May 14, 2026
Merged

Register missing SQL V2 relevance functions in Calcite function table#5440
dai-chen merged 2 commits into
opensearch-project:mainfrom
dai-chen:feature/add-missing-relevance-functions-calcite

Conversation

@dai-chen
Copy link
Copy Markdown
Collaborator

@dai-chen dai-chen commented May 14, 2026

Description

Register missing SQL V2 relevance functions in PPLFuncImpTable so the Calcite engine in UnifiedQueryPlanner can resolve them when processing RelNodes from the SQL V2 AST.

Function Support Matrix

Function SQL V2 (Legacy) PPL V3 (Calcite) Unified SQL (Calcite) — this PR
match
match_phrase
match_bool_prefix
match_phrase_prefix
multi_match
simple_query_string
query_string
query ✅ new
wildcard_query ✅ new
wildcardquery ✅ new (alias → wildcard_query)
match_query ✅ new (alias → match)
matchquery ✅ new (alias → match)
matchphrase ✅ new (alias → match_phrase)
matchphrasequery ✅ new (alias → match_phrase)
multimatch ✅ new (alias → multi_match)
multimatchquery ✅ new (alias → multi_match)
score
scorequery
score_query
  • Note: score/scorequery/score_query produce a ScoreFunction AST node (wrapper around inner relevance function) requiring dedicated visitor handling — not standard function table resolution.
  • Known limitation: PredicateAnalyzer pushdown handlers for the new functions are not yet implemented. These functions are only reachable from SQL (PPL grammar does not expose them).

Planned PRs

  • PR 1: Route unified SQL path through V2 ANTLR parser #5438
  • PR 2 (current): Add missing relevance functions to Calcite function table
  • PR 3: Extend V2 grammar/parser for missing statements, JOIN/UNION/MINUS
  • PR 4: Fix remaining CalciteRelNodeVisitor issues identified during V2 AST integration

Related Issues

Part of #5248; Related to #5279

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…on table

Add SQL V2 relevance functions (query, wildcard_query, score,
scorequery) and aliases (match_query, matchquery, matchphrase,
matchphrasequery, multimatch, multimatchquery) to PPLFuncImpTable
so CalciteRelNodeVisitor can resolve them when processing RelNodes
generated by SQL V2 AstBuilder.

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen self-assigned this May 14, 2026
@dai-chen dai-chen added enhancement New feature or request SQL labels May 14, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 14, 2026

PR Reviewer Guide 🔍

(Review updated until commit 064092d)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 14, 2026

PR Code Suggestions ✨

Latest suggestions up to 48f2911
Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Separate query function from single-field set

The query function is being added to SINGLE_FIELD_RELEVANCE_FUNCTION_SET, but based
on the test case it doesn't take a field parameter like other single-field
functions. The test shows query('name:John') with only one argument, not
query(field, 'value'). This function should likely be in a separate category or
handled differently to avoid incorrect validation or processing.

core/src/main/java/org/opensearch/sql/calcite/utils/UserDefinedFunctionUtils.java [78-92]

 public static Set<String> SINGLE_FIELD_RELEVANCE_FUNCTION_SET =
     ImmutableSet.of(
         "match",
         "match_phrase",
         "match_bool_prefix",
         "match_phrase_prefix",
         // TODO: Functions below are registered for SQL V2 AstBuilder compatibility but lack
         //  PredicateAnalyzer pushdown handlers. Only reachable from SQL (not PPL grammar).
-        "query",
         "wildcard_query",
         "wildcardquery",
         "match_query",
         "matchquery",
         "matchphrase",
         "matchphrasequery");
+public static Set<String> QUERY_STRING_RELEVANCE_FUNCTION_SET =
+    ImmutableSet.of("query");
Suggestion importance[1-10]: 8

__

Why: The query function takes a single query string parameter (query('name:John')) unlike other single-field functions that take separate field and value parameters (match(name, 'John')). Including it in SINGLE_FIELD_RELEVANCE_FUNCTION_SET could lead to incorrect validation or processing logic that expects the standard two-parameter pattern.

Medium

Previous suggestions

Suggestions up to commit f01a6cb
CategorySuggestion                                                                                                                                    Impact
Possible issue
Remove misclassified no-field function

The function query is a no-field relevance function (takes only query string, no
field parameter), not a single-field function. It should be removed from
SINGLE_FIELD_RELEVANCE_FUNCTION_SET to avoid incorrect validation or processing
logic that expects a field parameter.

core/src/main/java/org/opensearch/sql/calcite/utils/UserDefinedFunctionUtils.java [86]

 public static Set<String> SINGLE_FIELD_RELEVANCE_FUNCTION_SET =
     ImmutableSet.of(
         "match",
         "match_phrase",
         "match_bool_prefix",
         "match_phrase_prefix",
         // TODO: Functions below are registered for SQL V2 AstBuilder compatibility but lack
         //  PredicateAnalyzer pushdown handlers. Only reachable from SQL (not PPL grammar).
-        "query",
         "wildcard_query",
         "wildcardquery",
         "match_query",
         "matchquery",
         "matchphrase",
         "matchphrasequery");
Suggestion importance[1-10]: 8

__

Why: The query function is documented in the test file as a no-field relevance function (line 199-209), taking only a query string without a field parameter. Including it in SINGLE_FIELD_RELEVANCE_FUNCTION_SET is incorrect and could lead to validation or processing errors where field parameters are expected.

Medium

@dai-chen dai-chen force-pushed the feature/add-missing-relevance-functions-calcite branch from f01a6cb to 48f2911 Compare May 14, 2026 17:56
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 48f2911

… function table

Add SQL V2 relevance functions and aliases to PPLFuncImpTable so
CalciteRelNodeVisitor can resolve them from SQL V2 AstBuilder.
Aliases normalize to canonical operators (e.g. matchquery -> match).

New functions: query, wildcard_query
Aliases: wildcardquery, match_query, matchquery, matchphrase,
matchphrasequery, multimatch, multimatchquery

Not included: score/scorequery/score_query (needs dedicated visitor).

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen force-pushed the feature/add-missing-relevance-functions-calcite branch from 48f2911 to 064092d Compare May 14, 2026 18:23
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 064092d

@dai-chen dai-chen enabled auto-merge (squash) May 14, 2026 19:19
@dai-chen dai-chen merged commit c5ae56f into opensearch-project:main May 14, 2026
38 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request SQL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants