HIVE-26628: Iceberg table is created when running explain ctas command

[kasakrisz]

What changes were proposed in this pull request?

Drop Iceberg table after compiling an explain create table as a select command if the table did not exist before executing the statement.

Why are the changes needed?

Explain command should not execute any part of the command.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

mvn test -Dtest=TestIcebergCliDriver -Dqfile=ctas_iceberg_partitioned_orc.q -pl itests/qtest-iceberg -Piceberg -Pitests -Drat.skip

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[deniskuzZ]

hi, are we actually creating and dropping the table in case of Iceberg EXPLAIN CTAS?

[kasakrisz]

Yes, that is the case.
Unfortunately the table is created without this patch too which is a side affect of executing an explain statement.
The create is called from HiveIcebergSerDe init and this happening at compile time during FileSinkDesc generation.

hive/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java

Line 129 in 63b6134

createTableForCTAS(configuration, serDeProperties);

Later the Iceberg Table object is also required when it's properties are added to the job configs when augmenting the FileSinkOperator plan

hive/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java

Line 1611 in 63b6134

public void augmentPlan() {

hive/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java

Line 796 in 63b6134

Table table = IcebergTableUtil.getTable(configuration, props);

The goal of this patch is to clean up this table after executing the statement.

[deniskuzZ]

question

[kasakrisz]

not merging, issue is fixed by #3745