Skip to content

Add experimental openCypher graph query frontend#17938

Open
xiangfu0 wants to merge 1 commit intoapache:masterfrom
xiangfu0:experimental-graph-query-frontend
Open

Add experimental openCypher graph query frontend#17938
xiangfu0 wants to merge 1 commit intoapache:masterfrom
xiangfu0:experimental-graph-query-frontend

Conversation

@xiangfu0
Copy link
Copy Markdown
Contributor

Summary

Adds an experimental graph query feature that translates a subset of openCypher into SQL JOINs and executes via Pinot's multi-stage engine. This enables graph-style traversal queries over existing Pinot tables without native graph runtime operators.

  • New pinot-graph module (SPI + planner) with hand-written Cypher parser → IR → SQL transpiler
  • POST /query/graph broker endpoint behind pinot.graph.enabled feature flag (default: false)
  • Supports 1-hop and 2-hop traversals, WHERE (comparisons, AND/OR/NOT, STARTS WITH, ENDS WITH, CONTAINS, IN), RETURN (DISTINCT, AS aliases, COUNT/SUM/AVG/MIN/MAX with GROUP BY), ORDER BY, SKIP, LIMIT
  • Graph schema passed inline in request JSON mapping vertex/edge labels to Pinot tables
  • Zero impact on existing SQL query path — purely additive changes

Design doc posted on #17935: #17935 (comment)

Closes #17935

Test plan

  • 211 unit tests (parser: 94, SQL generator: 49, translator: 52, golden: 16) — all pass
  • 12 broker handler tests — all pass
  • 10 E2E integration tests with 2 servers and distributed data (50 users, 200 edges, 4 segments/table) — all pass
  • spotless, checkstyle, license checks — all clean
  • Existing SQL path unaffected (no modifications to query execution)
  • Feature disabled by default; returns clear 400 error when disabled
  • Unsupported Cypher syntax rejected with clear error messages

🤖 Generated with Claude Code

@xiangfu0 xiangfu0 force-pushed the experimental-graph-query-frontend branch 3 times, most recently from c8d0f7a to 7902d0f Compare March 23, 2026 06:49
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 23, 2026

Codecov Report

❌ Patch coverage is 83.73860% with 214 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.43%. Comparing base (1329e1f) to head (0263674).
⚠️ Report is 238 commits behind head on master.

Files with missing lines Patch % Lines
...a/org/apache/pinot/graph/planner/CypherParser.java 88.41% 30 Missing and 27 partials ⚠️
...che/pinot/graph/planner/CypherToSqlTranslator.java 72.32% 27 Missing and 17 partials ⚠️
...pinot/broker/api/resources/PinotClientRequest.java 0.00% 43 Missing ⚠️
...a/org/apache/pinot/graph/planner/SqlGenerator.java 85.50% 27 Missing and 12 partials ⚠️
...e/pinot/graph/planner/CypherSemanticValidator.java 79.77% 9 Missing and 9 partials ⚠️
.../java/org/apache/pinot/graph/planner/CypherIR.java 97.00% 3 Missing and 2 partials ⚠️
...not/broker/requesthandler/GraphRequestHandler.java 93.75% 0 Missing and 4 partials ⚠️
...ache/pinot/graph/planner/CypherParseException.java 50.00% 2 Missing ⚠️
.../org/apache/pinot/graph/spi/GraphSchemaConfig.java 96.42% 1 Missing ⚠️
...va/org/apache/pinot/spi/utils/CommonConstants.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17938      +/-   ##
============================================
+ Coverage     63.25%   63.43%   +0.17%     
- Complexity     1490     1543      +53     
============================================
  Files          3191     3208      +17     
  Lines        192609   195484    +2875     
  Branches      29542    30111     +569     
============================================
+ Hits         121844   124002    +2158     
- Misses        61224    61761     +537     
- Partials       9541     9721     +180     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.40% <83.73%> (+0.18%) ⬆️
java-21 63.40% <83.73%> (+0.24%) ⬆️
temurin 63.43% <83.73%> (+0.17%) ⬆️
unittests 63.43% <83.73%> (+0.17%) ⬆️
unittests1 55.54% <0.00%> (+0.02%) ⬆️
unittests2 34.52% <83.73%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Implements an experimental graph query feature that translates a subset of
openCypher into SQL JOINs and executes via Pinot's multi-stage engine (MSE).
This enables graph-style traversal queries over existing Pinot tables without
introducing native graph runtime operators.

Supported Cypher syntax:
- MATCH with 1-hop and 2-hop traversals (outgoing, incoming, undirected)
- WHERE with comparison operators, AND/OR/NOT, string predicates
  (STARTS WITH, ENDS WITH, CONTAINS), IN lists
- RETURN with DISTINCT, column aliases (AS), aggregations
  (COUNT, SUM, AVG, MIN, MAX) with auto GROUP BY
- ORDER BY (ASC/DESC), SKIP, LIMIT

Architecture:
- New pinot-graph module (pinot-graph-spi + pinot-graph-planner)
- Hand-written recursive descent Cypher parser → IR → SQL generator
- POST /query/graph broker endpoint (behind pinot.graph.enabled flag)
- Graph schema passed inline in request JSON (vertex/edge → table mapping)

Tests: 233 total (211 unit + 12 handler + 10 E2E integration with 2 servers)

Closes: apache#17935

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@xiangfu0 xiangfu0 force-pushed the experimental-graph-query-frontend branch from 7902d0f to 0263674 Compare March 30, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[discuss] Native Graph Query Language Support in Pinot

2 participants