Skip to content

[SYSTEMDS-3921] New rewrites for relational operators #2424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
migraine-user wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

[SYSTEMDS-3921] New rewrites for relational operators #2424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
113d5e1
fix small typo
migraine-user Dec 6, 2025
aa6401c
initial implementation of DPScan based rewrite rule for raJoin
migraine-user Jan 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion scripts/builtin/raJoin.dml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
# A Matrix of left input data [shape: N x M]
# colA Integer indicating the column index of matrix A to execute inner join command
# B Matrix of right left data [shape: N x M]
# colA Integer indicating the column index of matrix B to execute inner join command
# colB Integer indicating the column index of matrix B to execute inner join command
# method Join implementation method (nested-loop, sort-merge, hash, hash2)
# ------------------------------------------------------------------------------
#
Expand Down
1 change: 1 addition & 0 deletions src/main/java/org/apache/sysds/hops/OptimizerUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,7 @@ public enum MemoryManager {
*/
public static boolean ALLOW_TRANSITIVE_SPARK_EXEC_TYPE = true;

public static boolean ALLOW_JOIN_REORDERING_REWRITE = true;
/**
* Enable prefetch and broadcast. Prefetch asynchronously calls acquireReadAndRelease() to trigger remote
* operations, which would otherwise make the next instruction wait till completion. Broadcast allows
Expand Down
5 changes: 4 additions & 1 deletion src/main/java/org/apache/sysds/hops/rewrite/ProgramRewriter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ public ProgramRewriter(boolean staticRewrites, boolean dynamicRewrites)
_dagRuleSet.add( new RewriteRemoveReadAfterWrite() ); //dependency: before blocksize
_dagRuleSet.add( new RewriteBlockSizeAndReblock() );
_dagRuleSet.add( new RewriteInjectOOCTee() );

if( OptimizerUtils.ALLOW_ALGEBRAIC_SIMPLIFICATION )
_dagRuleSet.add( new RewriteRemoveUnnecessaryCasts() );
if( OptimizerUtils.ALLOW_COMMON_SUBEXPRESSION_ELIMINATION )
Expand All @@ -93,7 +94,6 @@ public ProgramRewriter(boolean staticRewrites, boolean dynamicRewrites)
_dagRuleSet.add( new RewriteInjectSparkPReadCheckpointing() ); //dependency: reblock
if( OptimizerUtils.ALLOW_QUANTIZE_COMPRESS_REWRITE )
_dagRuleSet.add( new RewriteQuantizationFusedCompression() );

//add statement block rewrite rules
if( OptimizerUtils.ALLOW_BRANCH_REMOVAL )
_sbRuleSet.add( new RewriteRemoveUnnecessaryBranches() ); //dependency: constant folding
Expand All @@ -119,6 +119,9 @@ public ProgramRewriter(boolean staticRewrites, boolean dynamicRewrites)
_sbRuleSet.add( new MarkForLineageReuse() );
_sbRuleSet.add( new RewriteRemoveTransformEncodeMeta() );
_dagRuleSet.add( new RewriteNonScalarPrint() );
if( OptimizerUtils.ALLOW_JOIN_REORDERING_REWRITE )
_sbRuleSet.add( new RewriteJoinReordering() );

}

// DYNAMIC REWRITES (which do require size information)
Expand Down
Loading