Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-32670] Cascade deprecation to classes that implement SourceFunction #23079

Closed
wants to merge 3 commits into from
Closed

[FLINK-32670] Cascade deprecation to classes that implement SourceFunction #23079

wants to merge 3 commits into from

Conversation

afedulov
Copy link
Contributor

@afedulov afedulov commented Jul 26, 2023 •
edited
Loading

This is a trivial change that marks sub-classes/interfaces of SourceFunction as @Deprecated and adjusts some examples to fix the strict deprecation complier check failure.

@flinkbot
Copy link
Collaborator

flinkbot commented Jul 26, 2023 •
edited
Loading

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@afedulov afedulov force-pushed the 32670-deprecate branch 5 times, most recently from cc4b4ed to de09e34 Compare July 28, 2023 11:07
@afedulov afedulov changed the title [FLINK-32670] Annotate interfaces that inherit from SourceFunction as deprecated [FLINK-32670] Cascade deprecation to classes that implement SourceFunction Jul 28, 2023
@leonardBang
Copy link
Contributor

@flinkbot run azure

@afedulov afedulov force-pushed the 32670-deprecate branch 5 times, most recently from fd464bc to 85cead2 Compare August 3, 2023 21:02
@leonardBang leonardBang self-requested a review August 7, 2023 14:01
leonardBang
leonardBang reviewed Aug 7, 2023
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @afedulov for the contribution, the changes generally looks good to me, I only left minor comments to polish the PR.

...xamples-streaming/src/main/java/org/apache/flink/streaming/examples/gpu/MatrixVectorMul.java Outdated
DataStreamSource<List<Float>> result =
env.fromSource(generatorSource, WatermarkStrategy.noWatermarks(), "Vectors Source");

result.print();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redundant call

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noticing. I cannot test with CUDA and had to resort to "print"-testing. Fixed.

.../src/main/java/org/apache/flink/streaming/examples/statemachine/KafkaEventsGeneratorJob.java Outdated
String kafkaTopic = params.get("kafka-topic");
String brokers = params.get("brokers", "localhost:9092");
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any limitation we need to change from default 4 to 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an accidental leftover from manual testing the rps calculation, thanks for spotting.

...ming/src/main/java/org/apache/flink/streaming/examples/statemachine/StateMachineExample.java
@@ -62,7 +66,7 @@ public static void main(String[] args) throws Exception {
// ---- print some usage help ----

System.out.println(
"Usage with built-in data generator: StateMachineExample [--error-rate <probability-of-invalid-transition>] [--sleep <sleep-per-record-in-ms>]");
"Usage with built-in data generator: StateMachineExample [--error-rate <probability-of-invalid-transition>] [--sleep <sleep-per-record-in-ms> | --rps <records-per-second>]");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about remove legacy --sleep <sleep-per-record-in-ms> ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably better to keep backwards compatibility for users' sake, it does not actually "cost" us anything.

...n/java/org/apache/flink/streaming/examples/windowing/GroupedProcessingTimeWindowExample.java

@Override
public void run(SourceContext<Tuple2<Long, Long>> ctx) throws Exception {
private static class DataGeneratorFunction
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a javadoc for class?

flink-examples/flink-examples-streaming/pom.xml Outdated
Comment on lines 75 to 82
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-shaded-jackson</artifactId>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-shaded-jackson</artifactId>
</dependency>

<!-- test dependencies -->
<!-- test dependencies -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert tUnnecessary changes ?

flink-examples/flink-examples-streaming/pom.xml Outdated
Comment on lines 73 to 75
<!-- required by the shade plugin -->
<optional>${flink.markBundledAsOptional}</optional>
</dependency>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: please keep align.

...ming/src/main/java/org/apache/flink/streaming/examples/statemachine/StateMachineExample.java Outdated
Types.POJO(Event.class));
TypeInformation.of(Event.class));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: we can remove this change to above commit and thus let this commit clean

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Since we are OK with shading the datagen, I was planning to roll this commit up into one of the above anyway (marked as [tmp])

flink-examples/flink-examples-streaming/pom.xml
@@ -341,6 +343,19 @@ under the License.
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
Copy link
Contributor

@leonardBang leonardBang Aug 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shading data-gen connector to flink-examples-streaming is necessary and it makes sense to me. minor: We can add a meaningful <id> to describe the shade purpose

@afedulov
Copy link
Contributor Author

afedulov commented Aug 7, 2023

Thanks @leonardBang , I addressed your commends.
I rolled up the last [tmp] commit into the previous one, as proposed.

@afedulov afedulov force-pushed the 32670-deprecate branch 3 times, most recently from 2c5fe0b to 95c6690 Compare August 8, 2023 06:27
@leonardBang
Copy link
Contributor

leonardBang commented Aug 8, 2023 •
edited
Loading

Thanks @afedulov for the update, LGTM, but the compile failed, please check it

@afedulov
Copy link
Contributor Author

afedulov commented Aug 8, 2023

@flinkbot run azure

@afedulov
Copy link
Contributor Author

afedulov commented Aug 8, 2023 •
edited
Loading

CI is green again.

leonardBang
leonardBang approved these changes Aug 8, 2023
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @afedulov for the contribution, LGTM

@leonardBang leonardBang mentioned this pull request Aug 8, 2023
Merged
Stonewuu pushed a commit to Stonewuu/flink that referenced this pull request Aug 9, 2023
@afedulov @Stonewuu
[FLINK-32670][examples] Migrate examples with strict deprecation comp…
d991eab 
…iler checks to Source API

This closes apache#23079.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leonardBang leonardBang leonardBang approved these changes

Assignees
No one assigned
Labels
component=<none>
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@afedulov @flinkbot @leonardBang