HIVE-30011: MR jobs submitted outside ExecDriver miss the JDK 17 --add-opens flags

[shmilygkd]

What changes were proposed in this pull request?

Move the --add-opens injection for MR jobs from a private method in ExecDriver into JavaVersionUtils.addOpensFlags(Configuration) and call it from every code path that submits an MR job through JobClient:

• ExecDriver.execute (unchanged behaviour, now delegates to the helper)
• MergeFileTask.execute (ALTER TABLE ... CONCATENATE on the MR engine)
• ColumnTruncateTask.execute (TRUNCATE TABLE ... COLUMNS on RCFile tables)
• MRCompactor.launchCompactionJob (MR-based ACID compaction launched by the metastore Worker)

The helper appends the flags to mapreduce.map.java.opts, mapreduce.reduce.java.opts and yarn.app.mapreduce.am.command-opts, keeping whatever is already configured. TestJavaVersionUtils is added to cover the helper.

Why are the changes needed?

HIVE-28869 only injects the flags in ExecDriver. The three paths above submit MR jobs directly, so on a JDK 17 cluster whose mapred-site.xml does not carry the flags their task JVMs fail on the first reflective access into java.base, for example:

Error: java.lang.reflect.InaccessibleObjectException: Unable to make field private volatile java.lang.String java.net.URI.string accessible:
  module java.base does not "opens java.net" to unnamed module
    at org.apache.hadoop.hive.common.StringInternUtils.<clinit>
    at org.apache.hadoop.hive.ql.io.HiveInputFormat.init

MRCompactor is still the default compactor for full-CRUD tables (hive.compactor.crud.query.based=false), so this affects regular ACID maintenance, not only the MR engine.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

• New unit test TestJavaVersionUtils#testAddOpensFlagsAppendedToMRJobOpts (existing -Xmx value preserved, all three keys end with the flag set).
• The qtest MiniMR drivers run tasks in the local job runner, where *.java.opts is ignored, so the container command line cannot be exercised in CI. An equivalent patch on a 3.1.x-based build was verified on a 3-node YARN cluster (Hadoop 3.4.3, JDK 17 on submitter and containers, no cluster-side --add-opens): before the patch ALTER TABLE ... CONCATENATE failed as above; after it the AM and task launch_container.sh of the merge job, the RCFile column-truncate job and a metastore-launched major compaction all carry the flags and the jobs succeed.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud