Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] feat: support partial merge phase in aggregation #1330

Merged
merged 2 commits into from
Apr 14, 2023

Conversation

rui-mo
Copy link
Contributor

@rui-mo rui-mo commented Apr 11, 2023

What changes were proposed in this pull request?

Support partial merge phase on Velox backend. This PR mainly focuses on supporting the partial merge step brings by count distinct, and a follow-up PR will be proposed to support other distinct functions.
oap-project/velox#193

How was this patch tested?

Verified on Jenkins.

@github-actions
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

structTypeNodes.add(ConverterUtils.getTypeNode(DoubleType, nullable = true))
structTypeNodes.add(ConverterUtils.getTypeNode(LongType, nullable = true))
structTypeNodes.add(ConverterUtils.getTypeNode(DoubleType, nullable = true))
structTypeNodes.add(ConverterUtils.getTypeNode(DoubleType, nullable = true))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @liujiayi771, could you help review this change? Eight intermediate types were used here, but looks to me only four is needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only need four, It should be that a rebase has retained the wrong code.

Copy link
Contributor Author

@rui-mo rui-mo Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Thanks for checking.

@rui-mo rui-mo force-pushed the wip_part_merge branch 3 times, most recently from 8a41884 to c0d5893 Compare April 14, 2023 01:36
@rui-mo rui-mo requested review from zhouyuan and JkSelf April 14, 2023 03:20
zhouyuan
zhouyuan previously approved these changes Apr 14, 2023
Copy link
Contributor

@zhouyuan zhouyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 verified locally, Q95 can off-load hashagg operators with this patch

@rui-mo rui-mo merged commit e0ab9b3 into apache:main Apr 14, 2023
9 checks passed
@rui-mo
Copy link
Contributor Author

rui-mo commented Apr 14, 2023

Thanks for your review!

@rui-mo rui-mo deleted the wip_part_merge branch October 25, 2023 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants