Skip to content

[SPARK-57064][SQL] Widen bucketing rule pattern matches to use FileSourceScanLike trait#56105

Open
rdtr wants to merge 1 commit into
apache:masterfrom
rdtr:widen-scan-trait-match
Open

[SPARK-57064][SQL] Widen bucketing rule pattern matches to use FileSourceScanLike trait#56105
rdtr wants to merge 1 commit into
apache:masterfrom
rdtr:widen-scan-trait-match

Conversation

@rdtr
Copy link
Copy Markdown

@rdtr rdtr commented May 26, 2026

Description:

Summary

  • Widen 3 read-only pattern match sites in DisableUnnecessaryBucketedScan and CoalesceBucketsInJoin from concrete FileSourceScanExec to the FileSourceScanLike trait
  • Two match sites that call .copy() are intentionally left on FileSourceScanExec
  • Enables third-party columnar plugins (Gluten, Comet, RAPIDS) to be recognized by bucketing optimization rules

Test plan

  • Existing bucketing test suites pass (no behavioral change for vanilla Spark since FileSourceScanExec already extends FileSourceScanLike)
  • Verified all accessed fields (bucketedScan, relation, optionalNumCoalescedBuckets) are declared on the FileSourceScanLike trait

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant