[HUDI-1644] Do not delete older rollback instants as part of rollback…

[satishkotha]

What is the purpose of the pull request

Archival can take care of removing old rollback instants cleanly

Brief change log

• rollback instants are cleaned up by archival process. Trying to delete them during rollback is not needed. This code also has many bugs: a) it removes newer rollback instants instead of older ones b) it doesnt clean up inflight/requested files, so not compatible with timeline layout version 1.

Verify this pull request

Only code deletions, existing tests cover functionality

Committer checklist

• Has a corresponding JIRA in PR title & commit

• Commit message is descriptive of the change

• CI is green

• Necessary doc changes done or have another open PR

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[codecov-io]

Codecov Report

Merging #2610 (b108e6f) into master (be257b5) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2610      +/-   ##
============================================
- Coverage     51.27%   51.26%   -0.01%     
+ Complexity     3241     3238       -3     
============================================
  Files           438      438              
  Lines         20126    20112      -14     
  Branches       2079     2079              
============================================
- Hits          10320    10311       -9     
+ Misses         8954     8949       -5     
  Partials        852      852              

Flag	Coverage Δ	Complexity Δ
hudicli	36.87% <ø> (ø)	0.00 <ø> (ø)
hudiclient	100.00% <ø> (ø)	0.00 <ø> (ø)
hudicommon	51.30% <ø> (-0.05%)	0.00 <ø> (ø)
hudiflink	46.85% <ø> (ø)	0.00 <ø> (ø)
hudihadoopmr	33.16% <ø> (ø)	0.00 <ø> (ø)
hudisparkdatasource	69.71% <ø> (ø)	0.00 <ø> (ø)
hudisync	49.62% <ø> (ø)	0.00 <ø> (ø)
huditimelineservice	66.49% <ø> (ø)	0.00 <ø> (ø)
hudiutilities	69.59% <ø> (+0.15%)	0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ	Complexity Δ
...c/main/java/org/apache/hudi/common/fs/FSUtils.java	47.80% <ø> (-1.52%)	57.00 <0.00> (-4.00)
...e/hudi/common/table/log/HoodieLogFormatWriter.java	78.12% <0.00%> (-1.57%)	26.00% <0.00%> (ø%)
...apache/hudi/utilities/deltastreamer/DeltaSync.java	71.07% <0.00%> (+0.35%)	53.00% <0.00%> (+1.00%)
...in/java/org/apache/hudi/utilities/UtilHelpers.java	65.69% <0.00%> (+1.16%)	33.00% <0.00%> (ø%)

[n3nash]

@satishkotha High level looks good to me, can we confirm if we have a equivalent test case on the archiving of rollback instants that simulates the same behavior of not leaving behind any rollback instants ?

[satishkotha]

@satishkotha High level looks good to me, can we confirm if we have a equivalent test case on the archiving of rollback instants that simulates the same behavior of not leaving behind any rollback instants ?

@n3nash Looks like we dont have unit tests for both clean/rollback instants. I created https://issues.apache.org/jira/browse/HUDI-1645 as a follow up.

[n3nash]

@satishkotha Okay, let's take up that task after this so that both go into the next release together.

[n3nash]

LGTM