[Bug] Shaded distributions include Log4j dependencies from Plan

[rosemarYuan]

Search before asking

• I searched in the issues and found nothing similar.

Description

The current logging dependency setup is inconsistent across the Plan, Runtime, Dist, and E2E modules. This issue was discovered while working on #821.

In the current dependency setup, the Plan module declares slf4j-api, log4j-core, and log4j-slf4j-impl with compile scope.

Because the Dist modules depend on Plan and package their dependencies with Maven Shade, these logging dependencies are copied into the generated Dist JARs.

However, the default logging setup of the supported Flink distributions already provides Log4j2 and the SLF4J binding for the JobManager and TaskManager processes. After a Flink Agents Dist JAR is installed, the same process may therefore contain:

• the Log4j API and implementation provided by Flink; and
• another Log4j implementation bundled inside Flink Agents.

If the two sets use different versions, the JVM may load incompatible Log4j API and Core classes together. This can cause runtime linkage errors such as:

NoSuchMethodError

Expected behavior:

• Dist JARs do not include Log4j or SLF4J classes inherited from Plan.
• Flink Agents uses the logging implementation provided by the selected Flink distribution.
• META-INF/NOTICE matches the actual contents of the generated JARs.

How to reproduce

1. Build the Dist artifacts:

mvn -DskipTests \
  -pl dist/common,dist/flink-1.20,dist/flink-2.0,dist/flink-2.1,dist/flink-2.2,dist/flink-2.3 \
  -am clean package

2. Inspect the generated JARs:

for artifact in \
  dist/common/target/flink-agents-dist-common-*.jar \
  dist/flink-*/target/flink-agents-dist-flink-*.jar; do
    echo "===== ${artifact} ====="
    jar tf "${artifact}" | grep -E \
      '(^|/)org/apache/logging/log4j/|(^|/)org/slf4j/|log4j2'
done

Expected:

No Log4j or SLF4J classes are found.

Actual:

Some Dist artifacts contain Log4j and SLF4J classes inherited from Plan.

Version and environment

• Flink Agents version: 0.3-SNAPSHOT

Are you willing to submit a PR?

• I'm willing to submit a PR!