Skip to content

[FLINK-29707][cli] Fix possible comparator violation for "flink list"#21127

Merged
mbalassi merged 1 commit intoapache:masterfrom
ferenc-csaky:FLINK-29707
Oct 26, 2022
Merged

[FLINK-29707][cli] Fix possible comparator violation for "flink list"#21127
mbalassi merged 1 commit intoapache:masterfrom
ferenc-csaky:FLINK-29707

Conversation

@ferenc-csaky
Copy link
Contributor

What is the purpose of the change

Fixes a comparator related bug for flink list, which can occur in JDK8. Probably fixed with JDK9, but I did not validated that.

Brief change log

Removed the custom comparator for JobStatusMessage and replaced it with a safe Comparator.comparing(...).

Verifying this change

If we put back the removed custom comparator, the updated test with the tricky data will fail.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no

@zentol
Copy link
Contributor

zentol commented Oct 20, 2022

I don't get how this can be JDK-specific. Probably related to overflows, no?

Is the test replicating the exact case you ran into?

@zentol zentol self-assigned this Oct 20, 2022
@zentol
Copy link
Contributor

zentol commented Oct 20, 2022

The fix itself is fine though.

@flinkbot
Copy link
Collaborator

flinkbot commented Oct 20, 2022

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@ferenc-csaky
Copy link
Contributor Author

ferenc-csaky commented Oct 20, 2022

I don't get how this can be JDK-specific. Probably related to overflows, no?

TBH I'm not sure anymore :) I did not debug it thoroughly, because of the easy fix. I went through this thread. It is not the same thing that happened here, there were mixed objects in the list in those cases and it was explicitly mentioned it works with the same object types.

Is the test replicating the exact case you ran into?

Yes.

@MartijnVisser
Copy link
Contributor

@flinkbot run azure

@ferenc-csaky
Copy link
Contributor Author

@flinkbot run azure

@mbalassi
Copy link
Contributor

Since @ferenc-csaky made the test more concise (which was the most important concern) and @zentol already gave a 👍 on the logic I am merging this.

@mbalassi mbalassi merged commit 5176fb2 into apache:master Oct 26, 2022
@ferenc-csaky ferenc-csaky deleted the FLINK-29707 branch October 27, 2022 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants