[FLINK-26163] Refactor FlinkUtils#getEffectiveConfig into smaller pieces

[bgeng777]

1. Refactor FlinkUtils#getEffectiveConfig into smaller pieces.
2. Refactor some codes in TestUtils and add unit test for FlinkUtils#getEffectiveConfig.

[bgeng777]

cc @wangyang0918 @gyfora @tweise

[wangyang0918]

1. Refactor FlinkUtils#getEffectiveConfig into smaller pieces.
2. Fix the problem that the field of replicas in JobManagerSpec will not take effect.
3. Fix the problem that the basic podTemplate in FlinkDeploymentSpec will not take effect if JM or TM spec does not specify podTemplate on their own.
4. Refactor some codes in TestUtils and add unit test for FlinkUtils#getEffectiveConfig.

• I am suggesting to factor out No.2 into a separate ticket.
• Why we have the No.3 issue? IIUC, currently the pod template should work for both JobManager and TaskManager.
• For No.1, what do you think about introducing a FlinkConfigBuilder just like following. It will make things easier to test.

/** Build effective flink config from {@link FlinkDeployment} */
public class FlinkConfigBuilder {
    private final FlinkDeployment deploy;
    private final Configuration effectiveConfig;

    public FlinkConfigBuilder(FlinkDeployment deploy) {
        this.deploy = deploy;
        this.effectiveConfig = FlinkUtils.loadConfiguration();
    }

    public FlinkConfigBuilder applyImageToConfig() {
        effectiveConfig.set(KubernetesConfigOptions.CONTAINER_IMAGE, deploy.getSpec().getImage());
        return this;
    }

    // Apply other fields to the config
    ... ...

    public Configuration build() {
        // Set some basic configuration
        final String namespace = deploy.getMetadata().getNamespace();
        final String clusterId = deploy.getMetadata().getName();
        effectiveConfig.setString(KubernetesConfigOptions.NAMESPACE, namespace);
        effectiveConfig.setString(KubernetesConfigOptions.CLUSTER_ID, clusterId);
        return effectiveConfig;
    }
}

final Configuration effectiveConfig = new FlinkConfigBuilder(flinkApp).applyImageToConfig().apply<...>.build();

[bgeng777]

Thanks a lot for the detailed review!

1. Sure. It seems that the fix of JM replicas is just a piece of codes so I did it here. I will move it out and leave the work to the dedicated jira for better tracking.
2. My bad. Check the KUBERNETES_POD_TEMPLATE field again and I understood it wrongly before.
3. Yes, I also think about using Builder pattern when refactoring. Considering not to change the code style too much, I did not use it initially. But I believe it could make codes more neat and flexible. Updated.

[gyfora]

Added some minor cosmetic comments, otherwise looks good :)

[tweise]

Thanks, great to get some test coverage for this area.

[tweise]

Also see #10 for related field changes

[gyfora]

Looks good! Thank you :)

Will merge this after breakfast