[AURON #1856] Introduce Flink expression-level converter framework#2146
[AURON #1856] Introduce Flink expression-level converter framework#2146Tartarus0zm merged 2 commits intoapache:masterfrom
Conversation
github-actions
bot
added
documentation
|
Hi @Tartarus0zm, the implementation PR for #1856 is ready for review. It implements the five classes from the approved Rev 3 design: FlinkNodeConverter, FlinkRexNodeConverter, FlinkAggCallConverter, FlinkNodeConverterFactory, and ConverterContext. This PR only includes the framework layer and does not add any concrete converters yet. To help with the review, I also included:
These markdown files are only for review support and will be removed after the review is complete. Could you please take a look when you have a chance? Thanks! |
weiqingy
force-pushed
the
AURON-1856-impl
branch
4 times, most recently
from
f732361 to
d6ae3b4
Compare
Tartarus0zm
left a comment
Tartarus0zm
left a comment
There was a problem hiding this comment.
thanks for your contributions!
I left a few comments, PTAL
.../src/main/java/org/apache/auron/flink/table/planner/converter/FlinkNodeConverterFactory.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/apache/auron/flink/table/planner/converter/FlinkNodeConverterFactory.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/apache/auron/flink/table/planner/converter/FlinkNodeConverterFactory.java
Outdated
Show resolved
Hide resolved
Add five foundational classes in auron-flink-planner that form the expression-level converter framework for Flink's native query acceleration: - FlinkNodeConverter<T>: generic base interface (getNodeClass, isSupported, convert) - FlinkRexNodeConverter: sub-interface for Calcite RexNode expressions - FlinkAggCallConverter: sub-interface for Calcite AggregateCall aggregates - FlinkNodeConverterFactory: singleton registry with separate rexConverterMap and aggConverterMap, typed registration and fail-safe dispatch - ConverterContext: immutable holder for input schema (RowType), Flink config, Auron config, and classloader This is framework-only — no concrete converter implementations are included. Follow-up issues will add RexLiteral, RexInputRef, RexCall, and aggregate converters that plug into this framework. Closes apache#1856
weiqingy
force-pushed
the
AURON-1856-impl
branch
from
d6ae3b4 to
2b1acef
Compare
@Tartarus0zm Thanks for the review! Switched to org.slf4j.Logger / LoggerFactory in both the import and the two logging call sites. Updated and pushed. |
|
@weiqingy LGTM, You can delete these Markdown files. |
Per reviewer request, remove design doc and review helper files from the PR. These were included for review purposes only.
github-actions
bot
removed
the
documentation
Thanks @Tartarus0zm. The MD files have been removed. |
Tartarus0zm
left a comment
Tartarus0zm
left a comment
There was a problem hiding this comment.
thanks for your contribution again!
LGTM
2 participants
Which issue does this PR close?
Closes #1856
Rationale for this change
Auron's Flink integration has the data exchange layer in place (Arrow Writer #1850, Arrow Reader #1851) but lacks the conversion infrastructure — the machinery that decides which Flink expressions and aggregates can be converted to native execution and how to translate them into Auron's protobuf representation.
What changes are included in this PR?
Five foundational Java classes in
auron-flink-planner:FlinkNodeConverter<T>getNodeClass(),isSupported(),convert()→PhysicalExprNodeFlinkRexNodeConverterRexNodeexpressionsFlinkAggCallConverterAggregateCallaggregatesFlinkNodeConverterFactoryrexConverterMap+aggConverterMap, typed registration and fail-safe dispatchConverterContextRowType), Flink config, Auron config, and classloaderpom.xml:
flink-corescope changed fromtesttoprovided(required forReadableConfig).Framework-only — no concrete converter implementations. Follow-up issues will add RexLiteral, RexInputRef, RexCall, and aggregate converters.
Design doc:
docs/PR-AURON-1856/AURON-1856-DESIGN.mdReview helper:
docs/reviewhelper/AURON-1856/01-converter-framework.mdAre there any user-facing changes?
No.
How was this patch tested?
8 unit tests. Checkstyle: 0 violations.