Skip to content

Conversation

Myasuka
Copy link
Member

@Myasuka Myasuka commented Mar 4, 2023

What is the purpose of the change

FLINK-28853 changed the default constructor of SingleThreadFetcherManager. Though the SingleThreadFetcherManager is annotated as Internal, it actually acts as some-degree public API, which is widely used in many connector projects:
flink-cdc-connector, flink-connector-mongodb and so on.

Once flink-1.17 is released, all these existing connectors are broken and cannot be used in new release version, and will throw exceptions like:

java.lang.NoSuchMethodError: org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager.<init>(Lorg/apache/flink/connector/base/source/reader/synchronization/FutureCompletingBlockingQueue;Ljava/util/function/Supplier;)V
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlSourceReader.<init>(MySqlSourceReader.java:91) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
	at com.ververica.cdc.connectors.mysql.source.MySqlSource.createReader(MySqlSource.java:159) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
	at org.apache.flink.streaming.api.operators.SourceOperator.initReader(SourceOperator.java:312) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask.init(SourceOperatorStreamTask.java:94) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:699) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:675) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:921) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]

Thus, I suggest to make the original SingleThreadFetcherManager constructor as depreacted instead of removing it.

Brief change log

Introduce the previous constructor again and make it as deprecated.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 4, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Member

@wuchong wuchong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@Myasuka Myasuka closed this in d5a6606 Mar 6, 2023
Myasuka added a commit that referenced this pull request Mar 6, 2023
…ger constructor and mark it as deprecated

This closes #22094.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants