-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Second phase of reconstruct read/write to adapt hive #222
Second phase of reconstruct read/write to adapt hive #222
Conversation
…ar conflict in trino
…ve-2 # Conflicts: # spark/v3.1/spark/src/main/java/com/netease/arctic/spark/ArcticSparkCatalog.java # spark/v3.1/spark/src/main/java/com/netease/arctic/spark/writer/ArcticSparkTaskWriterBuilder.java # spark/v3.1/spark/src/test/java/com/netease/arctic/spark/hive/TestAdaptHiveWriter.java
@@ -310,7 +310,9 @@ protected TableMetaStore.Builder getMetaStoreBuilder() { | |||
storageConfigs.get(CatalogMetaProperties.STORAGE_CONFIGS_KEY_TYPE))) { | |||
String coreSite = storageConfigs.get(CatalogMetaProperties.STORAGE_CONFIGS_KEY_CORE_SITE); | |||
String hdfsSite = storageConfigs.get(CatalogMetaProperties.STORAGE_CONFIGS_KEY_HDFS_SITE); | |||
String hiveSite = storageConfigs.get(CatalogMetaProperties.STORAGE_CONFIGS_KEY_HIVE_SITE); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does base arctic catalog need hive site config?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because TableMetaStore generate hive-site location need hive site config
@@ -106,6 +106,11 @@ | |||
<version>${iceberg.version}</version> | |||
<scope>test</scope> | |||
</dependency> | |||
|
|||
<dependency> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why dose hive moduel need this dependency?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because hive client will use mysql jdbc in some case
@@ -0,0 +1,396 @@ | |||
/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate licence
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1
* Add flink adapt hive UT, Fix core HiveMetastoreClient bug, Resolved jar conflict in trino * merge spark conflict * fix ut * fix flink ut * fix spark ut * fix trino ut * fix trino ut * fix trino ut * fix flink ut * fix flink ut * Add some annotation
Add flink adapt hive UT, Fix core HiveMetastoreClient bug, Resolve jar conflict in trino
link issue #151