Improve coordinator construction to support language runtime configs

[jason810496]

Why

Question from the community:

• How can I use different JDK version?
• How can I use different JVM arguments?

We hardcoded the subprocess cmd as the following, so user have to subclass another Coordinator to override the Java config to solve the above questions.

           return [
                "java",
                "-classpath",
                classpath,
                BundleScanner.resolve_jar(jar_path),
                f"--comm={comm_addr}",
                f"--logs={logs_addr}",
            ]

We should make the config more customizable and composable like AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST.

What

Current format before refactor:

[sdk]
queue_to_sdk = {"java-11": "java", "java-12": "java"}

After:

# Rename the `[sdk] queue_to_sdk` as `[sdk] queue_to_coordinator`
[sdk]
queue_to_coordinator = {"legacy-java-queue": "jdk-11", "modern-java-queue": "java-17"}

# but we introduce a new `[sdk] coordinators` config
[sdk]
coordinators = [
    {
        "name": "jdk-11",
        "classpath": "airflow.providers.sdk.java.coordinator.JavaCoordinator",
        "kwargs": {
            "java_executable": "/usr/lib/jvm/java-11-openjdk-amd64/bin/java",
            "jvm_args": ["-Xmx512m"],
            "jdk_home": "/usr/lib/jvm/java-11-openjdk-amd64"
        }
    },
    {
        "name": "jdk-17",
        "classpath": "airflow.providers.sdk.java.coordinator.JavaCoordinator",
        "kwargs": {
            "java_executable": "/usr/lib/jvm/java-17-openjdk-amd64/bin/java",
            "jvm_args": ["-Xmx1024m", "-Xms256m"],
            "jdk_home": "/usr/lib/jvm/java-17-openjdk-amd64"
        }
    }
]

Additionally, we should update the ADR to reflect the trace-off list out in this issue.