Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compute skew when concurrent queries #1743

Closed
imay opened this issue Sep 4, 2019 · 1 comment
Closed

compute skew when concurrent queries #1743

imay opened this issue Sep 4, 2019 · 1 comment

Comments

@imay
Copy link
Contributor

imay commented Sep 4, 2019

When we do high concurrency test to Doris, we found that one of all backends are complete consumed, but other backends have 60% CPU idle.

image

The test query likes "select day, sum(pv) from tbl where day = 20190904 group by day". After looking into backend's log, I found that all queries' second phase aggregation was processed in the exhausted backend.

This is because Doris will assign the same backend for aggregation/join shuffle slot, this will cause compute skew when the query has smaller group by cardinality than number of backends.

@imay
Copy link
Contributor Author

imay commented Sep 10, 2019

related to #1726

@imay imay closed this as completed Sep 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant