[CARBONDATA-3369] Fix issues during concurrent execution of Create table If not exists #3198
Conversation
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3074/ |
|
Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11338/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3273/ |
...2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonCreateTableCommand.scala
Outdated
Show resolved
Hide resolved
| } | ||
|
|
||
| // No need to throw for create if not exists | ||
| if (!ifNotExistsSet) { |
There was a problem hiding this comment.
But in old code, this check was not handled. where it was handled in old code.
There was a problem hiding this comment.
In old code this check was missing, and so "create table if not exists" throwing table already exists exception.
|
Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11378/ |
|
Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3313/ |
|
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3114/ |
|
Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11381/ |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3118/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3316/ |
|
retest this please |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3122/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3321/ |
|
Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11386/ |
|
LGTM |
…ble If not exists Create table if not exists has following problems if run concurrently from different drivers Sometimes It fails with error Table <db.table> already exists. Create table failed driver still holds the table with wrong path or schema. Eventual operations refer the wrong path Stale path created during create table is not deleted for ever [After 1.5.0 version table will be created in a new folder using UUID if folder with table name already exists] This PR fixes above 3 issues. This closes #3198
…ble If not exists Create table if not exists has following problems if run concurrently from different drivers Sometimes It fails with error Table <db.table> already exists. Create table failed driver still holds the table with wrong path or schema. Eventual operations refer the wrong path Stale path created during create table is not deleted for ever [After 1.5.0 version table will be created in a new folder using UUID if folder with table name already exists] This PR fixes above 3 issues. This closes apache#3198
Create table if not exists has following problems if run concurrently from different drivers
This PR fixes above 3 issues.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
Any interfaces changed?
No
Any backward compatibility impacted?
No
Document update required?
NA
Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
Yes
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.