Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] StreamPark support submit spark task to yarn #3569

Closed
3 tasks done
ChengJie1053 opened this issue Feb 23, 2024 · 6 comments
Closed
3 tasks done

[Feature] StreamPark support submit spark task to yarn #3569

ChengJie1053 opened this issue Feb 23, 2024 · 6 comments
Assignees
@ChengJie1053
Labels
feature/accepted This feature request is accepted feature/wip This feature is in WIP SPARK

Comments

@ChengJie1053
Copy link
Member

ChengJie1053 commented Feb 23, 2024 •
edited

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

Currently, StreamPark only supports flink, but many users are using spark, so StreamPark was extended to support Spark

The features to be implemented include：
1.Submit the spark jar task to yarn
2.Submit the spark sql task to yarn
3.Spark yarn status tracing
4.Spark yarn task stop

Design documentation: https://www.yuque.com/chengjie-lhwqa/kb/sqok17szgtggndno?singleDoc# 《Streampark支持Spark任务提交至Yarn》

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@caicancai
Copy link
Member

Maybe you should show your plan

@ChengJie1053
Copy link
Member Author

Maybe you should show your plan

Ok, I'm writing the technical plan

@ChengJie1053 ChengJie1053 mentioned this issue Feb 23, 2024
Merged
@caicancai caicancai added feature/wip This feature is in WIP SPARK labels Feb 23, 2024
@wolfboys wolfboys added the feature/accepted This feature request is accepted label Feb 23, 2024
@caicancai
Copy link
Member

@wolfboys @ChengJie1053 We should find time to evaluate the plan. What do you think?

@caicancai
Copy link
Member

@ChengJie1053 Can this large function be split into small tasks?

@ChengJie1053
Copy link
Member Author

Ok, thanks for the advice

@caicancai
Copy link
Member

This issue seems to be completed, closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ChengJie1053 ChengJie1053

Labels
feature/accepted This feature request is accepted feature/wip This feature is in WIP SPARK
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wolfboys @caicancai @ChengJie1053