Skip to content

[FIX] SPARQL Parsing of sub-SELECT as a Set-Operation Operand#1436

Merged
aaj3f merged 2 commits into
mainfrom
fix/sparql-subselect-setop-projection
Jul 6, 2026
Merged

[FIX] SPARQL Parsing of sub-SELECT as a Set-Operation Operand#1436
aaj3f merged 2 commits into
mainfrom
fix/sparql-subselect-setop-projection

Conversation

@aaj3f

@aaj3f aaj3f commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #1435. Repairs the root cause behind fluree/azure-chat#42 (sub-SELECT in UNION) and fluree/azure-chat#43 (sub-SELECT in MINUS).

The bug

A SPARQL sub-SELECT ({ SELECT ... }) used as an operand of a set/graph operation was mis-parsed, silently returning wrong results:

Query shape Expected Before
{ SELECT (…AS ?n) } UNION { SELECT (…AS ?n) } both arms one arm only (or 0 rows)
{ SELECT (…AS ?n) } MINUS { SELECT (…AS ?n) } difference full left set (no subtraction)
{ plain } UNION { SELECT (…AS ?n) } both arms sub-SELECT arm dropped
OPTIONAL { SELECT (COUNT(…) AS ?c) … } ?c bound ?c lost, rows duplicated

Silent (HTTP 200, no error), which is worse than a hard failure — NL→SPARQL / SQL→SPARQL generators emit { SELECT } UNION { SELECT } and { SELECT } MINUS { SELECT } routinely for "either/or" and "without" questions.

Root cause

Parser::parse_group_graph_pattern (fluree-db-sparql/src/parse/query/pattern.rs) only recognised a { SELECT ... } sub-SELECT inside its own nested-{ branch. Every other position that opens a group — OPTIONAL, GRAPH, SERVICE, the right arm of MINUS, each arm of UNION (parse_union_continuation), EXISTS/NOT EXISTS (the free parse_group_graph_pattern), and a bare WHERE { SELECT … } — consumes the { itself and then calls parse_group_graph_pattern positioned at the SELECT, where the leading SELECT was treated as an unexpected token.

  • UNION: the left sub-SELECT was pushed with no trailing-UNION check (the plain-group branch had one), so UNION hit "UNION must follow a pattern" and was discarded — the query collapsed into two independent subqueries lowered as a conjunction. At execution that becomes a correlated join on the shared projected variable; the arms' values are disjoint, so one arm survives (the smaller-cardinality one; ties → left) or 0 rows. (So it is not "only the last arm," as azure-chat#42 guessed.)
  • MINUS / OPTIONAL / GRAPH / SERVICE: the sub-SELECT was parsed as a bare BGP with the (expr AS ?v) projection dropped. MINUS then shares no bound variable with the left → subtracts nothing; OPTIONAL loses the projected/aggregate variable.

Lowered IR confirms it — before: {SELECT} UNION {SELECT}[Subquery, Subquery] (no Union); {SELECT} MINUS {SELECT}[Subquery, Minus(Triple×4)] (projection gone).

The fix (parser-only)

fluree-db-sparql/src/parse/query/pattern.rs:

  1. Detect the SubSelect alternative at the top of parse_group_graph_pattern (the opening { is already consumed by the caller), so every caller implements the grammar rule GroupGraphPattern ::= '{' ( SubSelect | GroupGraphPatternSub ) '}'. This fixes MINUS/OPTIONAL/GRAPH/SERVICE/EXISTS/UNION-right and bare WHERE { SELECT } in one place.
  2. Route the nested-{ branch through parse_group_graph_pattern (which now handles both a group and a sub-SELECT) and run the trailing-UNION check for a sub-SELECT left arm too.

Error recovery for a malformed sub-SELECT is preserved (skip to the matching }).

Performance

No query-engine change. Fixed queries now compile to the same UnionOperator the equivalent flat UNION always used, and to a proper sub-SELECT MINUS — i.e. the already-optimised plans. The buggy path was actually a more expensive per-row correlated SubqueryOperator join chain. So this is neutral-to-positive on speed, with no memory impact.

Testing

  • Parser AST tests (fluree-db-sparql): sub-SELECT in UNION (both arms + mixed group/sub-SELECT), MINUS right arm, OPTIONAL body. assert_parses also guards that the spurious "UNION must follow a pattern" diagnostic is gone.
  • Integration tests (fluree-db-api/tests/it_query_sparql_setop_subselect.rs): the verbatim azure-chat 42 & 43 queries and near-neighbors, asserting end-to-end results (UNION == flat UNION; MINUS == FILTER NOT EXISTS; OPTIONAL binds the aggregate).
  • W3C SPARQL suite (before → after): syntax-query 81 → 83 (syntax-SELECTscope1, a positive {SELECT} UNION {SELECT} syntax test), subquery 11 → 12 (sq12, bare WHERE { SELECT } in CONSTRUCT), negation 12 → 12 (no W3C test covers sub-SELECT-as-MINUS-operand — the new integration tests fill that gap). Zero regressions.
  • Full grp_query_sparql (265) and grp_query JSON-LD (303) integration groups pass; cargo clippy -p fluree-db-sparql --all-features -- -D warnings clean.
  • JSON-LD query surface is unaffected (separate parser; {query}-in-union already worked and still passes).

Files

  • fluree-db-sparql/src/parse/query/pattern.rs — the fix.
  • fluree-db-sparql/src/parse/query/tests.rs — parser AST regression tests.
  • fluree-db-api/tests/it_query_sparql_setop_subselect.rs (+ grp_query_sparql.rs) — integration regression tests.

A `{ SELECT ... }` sub-SELECT used as an operand of UNION, MINUS, OPTIONAL,
GRAPH, SERVICE, or EXISTS/NOT EXISTS was mis-parsed, silently producing wrong
results (fluree/azure-chat#42: UNION returned only one arm or 0 rows;
fluree/azure-chat#43: MINUS subtracted nothing and returned the full left set).

Root cause: `parse_group_graph_pattern` only detected a `{ SELECT ... }`
sub-SELECT inside its own nested-`{` branch. Every other caller (OPTIONAL,
GRAPH, SERVICE, the MINUS right arm, each UNION arm, and the free
`parse_group_graph_pattern` used by EXISTS/NOT EXISTS) consumes the `{` and then
calls `parse_group_graph_pattern` positioned at the `SELECT`, where the leading
`SELECT` was treated as an unexpected token. So the sub-SELECT was parsed as a
bare BGP with its `(expr AS ?v)` projection dropped (MINUS/OPTIONAL lose the
projected var), or — for UNION — the sub-SELECT left arm was pushed without a
trailing-UNION check, the `UNION` token was discarded as "UNION must follow a
pattern", and the query collapsed into a conjunction of two subqueries (a
correlated join on the shared projected var → one arm survives or 0 rows).

Fix (parser-only):
- Detect the `SubSelect` alternative at the top of `parse_group_graph_pattern`,
  so every position that admits a group implements the grammar rule
  `GroupGraphPattern ::= '{' ( SubSelect | GroupGraphPatternSub ) '}'`.
- Route the nested-`{` branch through `parse_group_graph_pattern` and run the
  trailing-UNION check for a sub-SELECT left arm too.

Fixed queries compile to the already-optimised `UnionOperator` / sub-SELECT
`MINUS` plans, so there is no query-engine change and no hot-path cost.

Tests:
- AST parser tests (fluree-db-sparql): sub-SELECT in UNION (both arms + mixed),
  MINUS right arm, OPTIONAL body.
- Integration tests (fluree-db-api): end-to-end UNION (both arms, mixed,
  order-independent, == flat UNION), MINUS (== FILTER NOT EXISTS), OPTIONAL.
- W3C SPARQL suite: syntax-query 81→83 (syntax-SELECTscope1), subquery 11→12
  (sq12); negation 12→12; zero regressions.

Fixes #1435
@aaj3f aaj3f requested review from bplatz and zonotope July 6, 2026 17:16

@bplatz bplatz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Follow-up to the review of #1436 (comment-only, no behavior change):

- parse_group_graph_pattern: warn contributors not to re-add a per-caller
  KwSelect check that bypasses the centralized sub-SELECT detection — doing so
  would silently reintroduce the dropped-UNION / dropped-projection bug (#1435).
- it_query_sparql_setop_subselect: note that the OPTIONAL sub-SELECT test relies
  on an intentional uncorrelated COUNT broadcast, so a future reader does not
  "correct" it into a per-row correlated count.
@aaj3f aaj3f merged commit f0cb5ea into main Jul 6, 2026
13 checks passed
@aaj3f aaj3f deleted the fix/sparql-subselect-setop-projection branch July 6, 2026 18:50
aaj3f added a commit that referenced this pull request Jul 7, 2026
aaj3f added a commit that referenced this pull request Jul 7, 2026
…1436)

CI runs the PR merge commit, and #1436 landed on main after this branch's
baseline: test_21/test_23/test_64 (sub-SELECT placement positives) and
subquery12 now pass — removed per the both-way stale-entry policing that
flagged them. test_65 (syntax-SELECTscope2) now parses instead of failing
for the wrong reason, so it moves to the accepts-invalid class awaiting the
SELECT-scope validation pass (burn-down PR-2).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SPARQL: sub-SELECT mis-parsed as a set-operation operand (UNION dropped; MINUS/OPTIONAL projection lost)

2 participants