Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][DISCUSS] operator support customize some feature configurations,like parallel #2925

Closed
3 tasks done
liugddx opened this issue Sep 28, 2022 · 5 comments · Fixed by #2941
Closed
3 tasks done

Comments

@liugddx
Copy link
Member

liugddx commented Sep 28, 2022

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

Sometimes transform need more parallel, so i think operator support customize some feature configurations.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@EricJoy2048
Copy link
Member

+1
Yes, It a good idea.

@liugddx
Copy link
Member Author

liugddx commented Oct 3, 2022

Do we set each operator individually or set the parallelism uniformly? @ashulin

@ashulin
Copy link
Member

ashulin commented Oct 3, 2022

@liugddx My opinion is the same as the [spark scenario] #2965, the user can set the parallelism of a single source and sink, and can also set the global parallelism. When the parallelism of the plugin does not exist, the global parallelism is used.

@liugddx
Copy link
Member Author

liugddx commented Oct 3, 2022

@liugddx My opinion is the same as the [spark scenario] #2965, the user can set the parallelism of a single source and sink, and can also set the global parallelism. When the parallelism of the plugin does not exist, the global parallelism is used.

I found that some operators are specified in parallelism itself.like jdbcSource and jdbcSink.Should we have uniform Settings? like this pr.

@ashulin
Copy link
Member

ashulin commented Oct 3, 2022

@liugddx I would like to have uniform the parallelism option Settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants