-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Support integrationing into gluten #940
Comments
@kaijchen Can you help publish the jar package to the public maven repository. |
If we want to publish the jar package, we need to finish binary release. And we only finish source releasenow. We can discuss this issue on Uniffle regular meeting in the next week. |
Doing so might require releasing process, let's do it in release 0.7.1? |
cc @summaryzb |
We can do it in next release version, after we complete this feature. |
I‘d like to involve this feature, shall you open a discussion on it |
SGTM. |
gluten side change |
@summaryzb Could you write a simple document like https://docs.google.com/document/d/1G0cOFVJbYLf2oX1fiadh7zi2M6DlEcjTQTh4kSkb0LA/edit#heading=h.kl0vgpq0nxcy |
Sure I'll post it here today later |
### What changes were proposed in this pull request? support read and write serialized columnar data ### Why are the changes needed? Fix: #940 ### Does this PR introduce any user-facing change? No. ### How was this patch tested? UnitTest, it's covered by the existing test
https://docs.google.com/document/d/1oi5QqrZejPRWBUIBv84eHbUhn2mTcgxP-s1ybh4zB1k/edit |
### What changes were proposed in this pull request? Make it possible to extend uniffle in spark3. Optimize the shuffleMetric when use columnar shuffle ### Why are the changes needed? #940 ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Unit test
Code of Conduct
Search before asking
Describe the feature
gluten
is a spark native engine. When we integrate uniffle into gluten, we need to adjust some codes of uniffle.In addition, we also need to publish the jar package to the public maven warehouse, like
celeborn
done.https://github.com/oap-project/gluten/tree/main/gluten-celeborn
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: