[HOTFIX] Concurrent insert test case failure fix #3610
Conversation
kunal642
force-pushed
the
concurrent_insert_testcase_fix
branch
from
ca152b0
to
7471f26
Compare
|
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/228/ |
| @@ -290,7 +290,7 @@ case class CarbonLoadDataCommand( | |||
| FileFactory.mkdirs(metadataDirectoryPath) | |||
| } | |||
| } else { | |||
| carbonLoadModel.setSegmentId(System.currentTimeMillis().toString) | |||
| carbonLoadModel.setSegmentId(System.nanoTime().toString) | |||
There was a problem hiding this comment.
Many times have observed (mainly from user environment) that , two concurrent tasks can get same nano time. So, we use UUID in these scenario. But for segmentID we cannot use as it may be casted to long.
Need other solutions or go back to old look up logic before #3601 for non-transactional tables
There was a problem hiding this comment.
Added lookup for non-transactional also..
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1930/ |
kunal642
force-pushed
the
concurrent_insert_testcase_fix
branch
from
7471f26
to
6a39669
Compare
|
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/231/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1933/ |
|
retest this please |
|
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/233/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1935/ |
|
LGTM |
Why is this PR needed?
System.currentTimeMillis() is giving same results for 2 insert due to which 1 load is deleting the temp folder for another load.
What changes were proposed in this PR?
Does this PR introduce any user interface change?
Is any new testcase added?