Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Enable AO/AOCO insert to multiple files even enable_parallel is off #39

Closed
2 tasks done
avamingli opened this issue Jul 24, 2023 · 0 comments · Fixed by #83
Closed
2 tasks done

[Bug] Enable AO/AOCO insert to multiple files even enable_parallel is off #39

avamingli opened this issue Jul 24, 2023 · 0 comments · Fixed by #83
Assignees
Labels
help wanted Extra attention is needed type: Bug Something isn't working

Comments

@avamingli
Copy link
Collaborator

Cloudberry Database version

No response

What happened

When enable_parallel is off, we will insert into only one AO segfile even gp_appendonly_insert_files is > 1.

Think about the case: user set enable_parallel to on, have some data inserted, query and reset it to false.

That will make data skew after user set enable_parallel to off, and there are a lot of data inserted later or an online-steaming ETL(all data would be inserted into only one segfile).

And that make our parallel plan has a bottleneck.

We should take it back, insert into multiple files according to gp_appendonly_insert_files whatever enable_parallel is.

In general, we should try to make AO segfiles as much as gp_appendonly_insert_files and avoid data skew for users, no matter users use parallel or not.

And only keep gp_appendonly_insert_files default value to 4 is enough.

What you think should happen instead

No response

How to reproduce

Need to create cases.

Operating System

Ubuntu

Anything else

By fixing this, to make regression pass , we need to set GUC gp_appendonly_insert_files = 0 when deploying CBDB at CI pipeline. Need help from @sandiandian .

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

@avamingli avamingli added type: Bug Something isn't working help wanted Extra attention is needed labels Jul 24, 2023
@avamingli avamingli self-assigned this Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed type: Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant