[Spark] Migrate streampark-spark from Scala to Java

[shangeyao]

Summary

Parent epic: #4408
Spark migration (post Stage 2)

Migrate streampark-spark from Scala to Java with no behavior changes, and keep Console Spark submit/cancel/SQL validation working against the new Java APIs.

Module role

Remaining Spark runtime modules after #4432 removed user-facing dev traits (SparkStreaming, SparkBatch) and connector wrappers:

Submodule	Scala files	Role
streampark-spark-core	4	SQL utilities — SparkSqlExecutor, SqlCommandParser, SparkSqlValidator, SQLContextUtil
streampark-spark-client-api	7	Public client API — SparkClient, request/response beans, SparkConfiguration, SparkShimsProxy
streampark-spark-client-core	3	Deploy implementations — SparkClientTrait, YarnClient, SparkClientEndpoint
streampark-spark-sqlclient	1	Spark SQL CLI entry — SqlClient

Total: 15 .scala files under src/main (no test Scala today).

Downstream compatibility (must not break)

Console (streampark-console-service) depends on these packages today:

• org.apache.streampark.spark.client.SparkClient + submit/cancel beans — SparkApplicationActionServiceImpl
• org.apache.streampark.spark.client.proxy.SparkShimsProxy — SparkSqlServiceImpl
• org.apache.streampark.spark.core.util.SparkSqlValidationResult / SparkSqlValidator (reflection) — SparkSqlController, SparkSqlServiceImpl

Keep package paths and public method signatures stable where possible; update Console imports only when unavoidable.

Note: Spark K8s pod template models (SparkK8sPodTemplates) already live in streampark-flink-kubernetes (Java) — out of scope for this issue.

Dependencies

• Depends on: [Flink] Migrate streampark-flink-client from Scala to Java #4452 (streampark-flink-client migration must merge first — completes Flink Stage 2)
• Blocks: Epic [Proposal] Remove Scala from StreamPark — design and migration plan #4408 acceptance (no .scala under src/main / src/test)

Do not start until #4452 is merged. Follow the same sequential PR workflow as Flink Stage 2.

Constraints

• Follow Java architecture and idioms; keep package paths where possible
• Each new Java class should have tests (or document why not) — PMC workflow on [Proposal] Remove Scala from StreamPark — design and migration plan #4408
• Preserve Console Spark submit/cancel/deploy and SQL validation functional parity
• Mirror patterns from completed Flink migrations ([Flink] Migrate streampark-flink-client from Scala to Java #4452 client, [Migrate] Migrate streampark-flink-shims from Scala to Java #4461 shims): singleton facades → static methods, trait → interface + impl, json4s → Jackson if present, ScalaTest → JUnit 5
• Spark module is activated via -Pspark Maven profile — verify build with and without profile where applicable

Acceptance

• No .scala under streampark-spark (src/main and src/test)
• -Pspark module builds and tests pass
• streampark-console-service compiles and Spark integration paths work (submit/cancel, SQL validate)
• Remove Scala-only build deps from Spark POMs where no longer needed (enumeratum, scala-maven-plugin config, etc.)

References

• Epic: [Proposal] Remove Scala from StreamPark — design and migration plan #4408
• Roadmap: [Roadmap] StreamPark 3.0 #4410
• JDK 11: [Proposal] Migrate StreamPark Console baseline from JDK 8 to JDK 11 #4409 / [Build] Raise StreamPark compile and CI baseline to JDK 11 #4417 ✅
• Dev framework removal: [Flink] Remove dev framework and connector wrappers #4432 ✅
• Flink client migration (pattern reference): [Flink] Migrate streampark-flink-client from Scala to Java #4452

[shangeyao]

Pausing the Spark Scala-to-Java migration for now. PR #4469 has been closed; the work can be reopened when build/CI issues are resolved and the migration is ready to continue.