Skip to content

[1.20][FLINK-35699][k8s] Fix shading Jackson through fabric8#24980

Merged
gyfora merged 1 commit intoapache:release-1.20from
ferenc-csaky:FLINK-35699-1.20
Jun 28, 2024
Merged

[1.20][FLINK-35699][k8s] Fix shading Jackson through fabric8#24980
gyfora merged 1 commit intoapache:release-1.20from
ferenc-csaky:FLINK-35699-1.20

Conversation

@ferenc-csaky
Copy link
Contributor

@ferenc-csaky ferenc-csaky commented Jun 25, 2024

What is the purpose of the change

The flink-kubernetes artifact shades Jackson classes coming through fabric8, but since Jackson 2.15, Jackson is a multi-release JAR, which requires some additional relocations for correct shading.

What Jackson does itself can be checked here.

Brief change log

  • Added exclusions for Jackson related meta files, which otherwise overwrites the Flink ones.
  • Added relocations for JDK-specific code.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

To verify, I pasted a comparison of the content of the JAR before and after the change below.

JAR content before change:
% jar tf flink-kubernetes-*.jar | grep 'META-INF/*/versions'
META-INF/versions/
META-INF/versions/9/
META-INF/versions/9/module-info.class
META-INF/versions/11/
META-INF/versions/11/com/
META-INF/versions/11/com/fasterxml/
META-INF/versions/11/com/fasterxml/jackson/
META-INF/versions/11/com/fasterxml/jackson/core/
META-INF/versions/11/com/fasterxml/jackson/core/io/
META-INF/versions/11/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/11/com/fasterxml/jackson/core/io/doubleparser/BigSignificand.class
META-INF/versions/11/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/11/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/11/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class
META-INF/versions/17/
META-INF/versions/17/com/
META-INF/versions/17/com/fasterxml/
META-INF/versions/17/com/fasterxml/jackson/
META-INF/versions/17/com/fasterxml/jackson/core/
META-INF/versions/17/com/fasterxml/jackson/core/io/
META-INF/versions/17/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/17/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/17/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/17/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class
META-INF/versions/19/
META-INF/versions/19/com/
META-INF/versions/19/com/fasterxml/
META-INF/versions/19/com/fasterxml/jackson/
META-INF/versions/19/com/fasterxml/jackson/core/
META-INF/versions/19/com/fasterxml/jackson/core/io/
META-INF/versions/19/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/19/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/19/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/19/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class
JAR content after change:
% jar tf flink-kubernetes-1.20-SNAPSHOT.jar | grep 'META-INF/*/versions'
META-INF/versions/
META-INF/versions/11/
META-INF/versions/11/org/
META-INF/versions/11/org/apache/
META-INF/versions/11/org/apache/flink/
META-INF/versions/11/org/apache/flink/kubernetes/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/BigSignificand.class
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/11/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class
META-INF/versions/17/
META-INF/versions/17/org/
META-INF/versions/17/org/apache/
META-INF/versions/17/org/apache/flink/
META-INF/versions/17/org/apache/flink/kubernetes/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/17/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class
META-INF/versions/19/
META-INF/versions/19/org/
META-INF/versions/19/org/apache/
META-INF/versions/19/org/apache/flink/
META-INF/versions/19/org/apache/flink/kubernetes/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath$UInt128.class
META-INF/versions/19/org/apache/flink/kubernetes/shaded/com/fasterxml/jackson/core/io/doubleparser/FastIntegerMath.class

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no

@flinkbot
Copy link
Collaborator

flinkbot commented Jun 25, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@ferenc-csaky ferenc-csaky changed the title [FLINK-35699][k8s] Fix shading fabric8 Jackson [FLINK-35699][k8s] Fix shading Jackson through fabric8 Jun 26, 2024
@ferenc-csaky ferenc-csaky changed the title [FLINK-35699][k8s] Fix shading Jackson through fabric8 [1.20][FLINK-35699][k8s] Fix shading Jackson through fabric8 Jun 27, 2024
@ferenc-csaky
Copy link
Contributor Author

@flinkbot run azure

@gyfora gyfora merged commit 7f40c35 into apache:release-1.20 Jun 28, 2024
@ferenc-csaky ferenc-csaky deleted the FLINK-35699-1.20 branch June 28, 2024 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants