Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Remove substrait-spark module #4609

Merged
merged 1 commit into from
Feb 2, 2024

Conversation

ulysses-you
Copy link
Contributor

What changes were proposed in this pull request?

  • Remove $GLUTEN_PROJECT/substrait, as it is not used in any place
  • Remove $GLUTEN_PROJECT/gluten-core/src/main/resources/substrait/extensions, as we are mapping expression manually
  • Remove Offset, as it is created for substrait-spark since Spark3.4

How was this patch tested?

Pass CI

Copy link

github-actions bot commented Feb 1, 2024

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Feb 1, 2024

Run Gluten Clickhouse CI

@ulysses-you
Copy link
Contributor Author

cc @baibaichen @zzcclp @rui-mo @FelixYBW thank you

Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ulysses-you ulysses-you merged commit 24acd4b into apache:main Feb 2, 2024
19 checks passed
@ulysses-you ulysses-you deleted the substrait-spark branch February 2, 2024 03:21
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4609_time.csv log/native_master_01_31_2024_f1ed68005_time.csv difference percentage
q1 33.90 32.75 -1.143 96.63%
q2 24.16 24.25 0.089 100.37%
q3 38.66 37.43 -1.227 96.83%
q4 37.42 37.40 -0.023 99.94%
q5 70.60 70.15 -0.451 99.36%
q6 7.05 7.00 -0.053 99.25%
q7 82.74 83.66 0.927 101.12%
q8 83.22 85.87 2.654 103.19%
q9 119.42 124.41 4.985 104.17%
q10 43.98 43.68 -0.299 99.32%
q11 20.34 20.32 -0.018 99.91%
q12 26.64 26.52 -0.115 99.57%
q13 44.23 45.62 1.389 103.14%
q14 21.55 19.16 -2.396 88.88%
q15 28.77 28.85 0.085 100.30%
q16 13.74 14.00 0.257 101.87%
q17 100.64 102.02 1.377 101.37%
q18 147.80 149.84 2.037 101.38%
q19 13.24 12.62 -0.620 95.32%
q20 27.21 26.35 -0.862 96.83%
q21 221.51 228.32 6.816 103.08%
q22 13.54 13.50 -0.036 99.73%
total 1220.35 1233.72 13.373 101.10%

vbarua pushed a commit to substrait-io/substrait-java that referenced this pull request Jun 26, 2024
Provides code to assist w/ translating Substrait to and from Spark

This module was originally part of the Gluten project, but it was removed from there.

For more details see:
apache/incubator-gluten#4609
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants