[CARBONDATA-3334] fixed multiple segment file issue for partition

[kunal642]

Problem:
During partition load, while writing merge index files the FactTimestamp in load model is being changed to current timestamp due to which a new file with mergeindex entry is written.

Solution:
Set new timestamp if FactTimestamp in load model is 0L(meaning nothing is set).

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

• Any interfaces changed?

• Any backward compatibility impacted?

• Document update required?

• Testing done
  Please provide details on
  - Whether new unit test cases have been added or why no new tests are required?
  - How it is tested? Please attach test report.
  - Is it a performance related change? Please attach the performance test report.
  - Any additional information to help reviewers in testing this change.

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[CarbonDataQA]

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11083/

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2823/

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2824/

[CarbonDataQA]

Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11084/

[CarbonDataQA]

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3056/

[kunal642]

retest this please

[CarbonDataQA]

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3057/

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2825/

[CarbonDataQA]

Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11085/

[xuchuanyin]

Feel like this modification is not elegant...
The modification still confuse my even I've read the description of the PR...

It is the MergeIndexWriter for partition loading that cause the problem, but you modify the loadModel instead. It seems that the writer and loadModel have some negotiations which makes the code complicated especially when the loadModel is at the beginning of loading while the MergeIndexWriter is at almost the end of loading.

[kunal642]

Hi xuchuanyin,
The problem was that before launching the job to merge index files we are changing the fact timestamp sue to which a new segments file is written with mergeindex details.
Example:-
load writes 0_t1.segment
while the mergeindex will write 0_t2.segment.

My fix is that if the load has already identifier a timestamp for the segments file then mergeindex should use the same.

I understand the check does not look good but the other solution would be to remove the 0_t1.segment file from merge index flow. i dont think that way would be clean either

[kunal642]

retest this please

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2873/

[CarbonDataQA]

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11133/

[CarbonDataQA]

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3104/

[ravipesala]

retest this please

[ravipesala]

LGTM

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2876/

[CarbonDataQA]

Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3107/

[CarbonDataQA]

Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11136/

[kunal642]

@ravipesala build passed..please merge