[Improvement]: Full-Optimizing occupies all optimizer resources making other tables' Self-Optimizing hang up

[wangtaohz]

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

Full-Optimizing takes a long time to optimize and occupies all resources, for example, it takes 1h 17min to rewrite ~40G files, making other tables' Self-Optimizing hang up

How should we improve?

1. We should limit the resource occupation for each Self-Optimizing, like limiting the total file size for each plan, a moderate default value may be 5~10G
2. The big file( file size >= 0.95 target size?) not related to any delete-files should not be rewritten when executing Full-Optimizing

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• I agree to follow this project's Code of Conduct