Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-34103][examples] Include flink-connector-datagen for AsyncIO example and bundle it into the dist by default #24207

Merged
merged 1 commit into from
Jan 30, 2024

Conversation

X-czh
Copy link
Contributor

@X-czh X-czh commented Jan 27, 2024

What is the purpose of the change

Include flink-connector-datagen for AsyncIO example to fix dep missing problem and bundle it into the dist by default.

Brief change log

  • Use maven shade to package AsyncIO example instead, including flink-connector-datagen deps.

Note that it was not bundled with the dist previously because we will only bundle those example jars with shortened names using the rewrite rule:

<!--simplify the name of example JARs for build-target/examples -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-antrun-plugin</artifactId>
				<executions>
					<execution>
						<id>rename</id>
						<configuration>
							<target>
								<copy file="${project.basedir}/target/flink-examples-streaming-${project.version}-WindowJoin.jar" tofile="${project.basedir}/target/WindowJoin.jar" />
								<copy file="${project.basedir}/target/flink-examples-streaming-${project.version}-WordCount.jar" tofile="${project.basedir}/target/WordCount.jar" />
								<copy file="${project.basedir}/target/flink-examples-streaming-${project.version}-SocketWindowWordCount.jar" tofile="${project.basedir}/target/SocketWindowWordCount.jar" />
							</target>
						</configuration>
					</execution>
				</executions>
			</plugin>

This change also integrates the name rewriting rule in the shade plugin config so that it is bundled by default.

Verifying this change

Manually verified it by submitting it to a standalone cluster. Note that since the AsyncIO example itself mocks instable connection randomly, the job will soon fail on its own.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no) no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) no
  • The serializers: (yes / no / don't know) no
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know) no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) no
  • The S3 file system connector: (yes / no / don't know) no

Documentation

  • Does this pull request introduce a new feature? (yes / no) no
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) not applicable

@flinkbot
Copy link
Collaborator

flinkbot commented Jan 27, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@X-czh X-czh force-pushed the FLINK-34103 branch 2 times, most recently from 36184f2 to f612c65 Compare January 28, 2024 11:01
Copy link
Contributor

@JunRuiLee JunRuiLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@X-czh Thanks for contribution, nice work. LGTM

Copy link
Member

@reswqa reswqa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. Could you please resolve the conflict also?

…xample and bundle it into the dist by default
@X-czh
Copy link
Contributor Author

X-czh commented Jan 29, 2024

@reswqa Rebased, could you help merge it?

@reswqa reswqa merged commit 3b4fa6c into apache:master Jan 30, 2024
@X-czh X-czh deleted the FLINK-34103 branch January 30, 2024 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants