Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-35089][runtime] Initialize lastRecordAttributes in AbstractStreamOperator during setup #24655

Closed
wants to merge 1 commit into from

Conversation

Sxnan
Copy link
Contributor

@Sxnan Sxnan commented Apr 12, 2024

What is the purpose of the change

Currently the lastRecordAttributes1 and lastRecordAttributes2 in the AbstractStreamOperator are transient. The two fields will be null when it is deserialized in TaskManager, which may cause an NPE.

To fix it, we will initialize the two fields in the setup method.

Brief change log

  • Initialize the two fields in the setup method.

Verifying this change

This change added tests and can be verified as follows:

  • Integration test is added org.apache.flink.test.streaming.runtime.RecordAttributesPropagationITCase

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializer: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

@flinkbot
Copy link
Collaborator

flinkbot commented Apr 12, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@Sxnan
Copy link
Contributor Author

Sxnan commented Apr 12, 2024

@xintongsong Can you help review this PR?

@Sxnan Sxnan changed the title [FLINK-35089][runtime] Serialize the lastRecordAttributes in AbstractStreamOperator [FLINK-35089][runtime] Initialize lastRecordAttributes in AbstractStreamOperator during setup Apr 16, 2024
xintongsong pushed a commit that referenced this pull request Apr 17, 2024
hanyuzheng7 pushed a commit to hanyuzheng7/flink that referenced this pull request May 6, 2024
@elvirasru
Copy link

elvirasru commented May 10, 2024

Hello,

Could you tell me when the new version with this fix will be released?

Thank you in advance!

PS: I have been facing this issue in version 1.19.0 when trying to connect a HybridSource with another source and process them. This is the error:
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.flink.streaming.runtime.streamrecord.RecordAttributes.isBacklog()" because "lastAttributes" is null
I am not sure if this BUG was introduced in version 1.19.0, as apparently version 1.18.1 works for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants