Skip to content

Conversation

@fsk119
Copy link
Member

@fsk119 fsk119 commented Nov 22, 2024

What is the purpose of the change

Relax the limitation that application mode requires a main jar.

Brief change log

  • Remove the check that pipeline.jar must have one jar

Verifying this change

This change added tests and can be verified as follows:

  • Add tests to get packaged program if pipeline.jar is empty
  • Add tests to deploy a cluster if pipeline.jar is empty

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Nov 22, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@davidradl
Copy link
Contributor

@fsk119 I am unsure what problem this is fixing and what the motivation is behind this change.

@fsk119
Copy link
Member Author

fsk119 commented Nov 22, 2024

@davidradl Hi. Please read the FLIP about the change[1] or the PoC code[2]. This PR tries to relax the limit that user must upload a main jar to run the job in application mode. But it's not true for sql jobs because the main class is located at the job master.

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-480%3A+Support+to+deploy+SQL+script+in+application+mode
[2] master...fsk119:flink:application-mode

Copy link
Contributor

@JunRuiLee JunRuiLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @fsk119 for contributing. I've left a comment, PTAL.

Copy link
Contributor

@JunRuiLee JunRuiLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fsk119 fsk119 merged commit 44da52d into apache:master Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants