New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[HUDI-2057][HUDI-5499] CTAS Generate An External Table When Create Managed Table #6419
base: master
Are you sure you want to change the base?
Conversation
@hudi-bot run azure |
8b8e052
to
2f1e53d
Compare
@@ -174,7 +174,7 @@ trait ProvidesHoodieConfig extends Logging { | |||
|
|||
val enableHive = isUsingHiveCatalog(sparkSession) | |||
|
|||
withSparkConf(sparkSession, catalogProperties) { | |||
withSparkConf(sparkSession, catalogProperties ++ extraOptions) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a test that should fail before this fix, to guard any breaking changes in the future?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't figure out how to implement the test case.Do you have any ideas?
@xushiyan : can you assist the author and help take this home. |
496a41c
to
515e9f7
Compare
| ) | ||
| location '/tmp/$tableName2' | ||
| AS | ||
| select 1 as id, 'a1' as name, 10 as price, 1000 as ts |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please move this to a proper unit test. Bundle validation is not the right place to execute such tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@xushiyan Hello, the purpose here is to test the type of the sync hive table, but sync hive cannot be realized in unit test
This problem has been fixed in this PR before:#3146
Later, the problem was discovered again,But it was fixed by this pr: #3998
The last time this problem reappeared was due to this pr: #5178,The reason is that
extraoptions
is not transferred to spark confChange Logs
CTAS Generate An External Table When Create Managed Table
Impact
CTAS Generate An External Table When Create Managed Table
Risk level (write none, low medium or high below)
none
Contributor's checklist