Hive: Push Iceberg table property values to HMS table properties

[marton-bod]

As agreed with the community:

• Iceberg table properties are the canonical source of truth
• HMS table properties should be maintained as much as possible to be in sync with the Iceberg table, but it can only happen on a best effort basis

This PR makes the following changes:

• Ensures that all Iceberg table properties are propagated to the HMS table during HiveTableOperations commit
• All HMS table properties are pushed down to Iceberg as well during table creation (except for metadata location and spec props)
• Refactors the various property check assertions scattered throughout various test cases into a single property-focused unit test case

What is left out and should be done in the future:

• Push property changes occurring via Hive DDL (ALTER TABLE SET TBLPROPERTIES) down to Iceberg as well. Currently this can't be done reliably because the HiveMetaHook interface only contains a preAlterTable method, but no commitAlterTable method. We'll need to extend this interface and include the change in an upcoming Hive upstream release.

[marton-bod]

Build failure is due to the flaky Spark tests:

org.apache.iceberg.spark.extensions.TestCopyOnWriteDelete > testDeleteWithSerializableIsolation[catalogName = spark_catalog, implementation = org.apache.iceberg.spark.SparkSessionCatalog, config = {type=hive, default-namespace=default, clients=1, parquet-enabled=false, cache-enabled=false}, format = avro, vectorized = false] FAILED
    java.lang.RuntimeException: Failed to get table info from metastore default.table

        Caused by:
        org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed)

            Caused by:
            java.net.SocketException: Broken pipe (Write failed)

[pvary]

What do you think about changing the title @marton-bod?

I think the title below describes the change better:

Hive: Push Iceberg table property values to HMS table properties

Thanks,
Peter

[marton-bod]

Overall I agree, I think that describes the crux of the change better. I went with the original title because we also relax the initial set of properties we push down from HMS to Iceberg in this patch. However, until we have a pre/postAlter hook on the HiveMetaHook, this is mostly a one-way street indeed.

[pvary]

Thanks @marton-bod for the change!