Enable FileSplits to be obtained directly from InputFormat

[vinothchandar]

• SplitLoader calls ipf.getSplits() if annotated with @UseFileSplitsFromInputFormat
• CustomInputFormats will need to define annotation with the above simple name, to use this

This is followup to PR #6969

[vinothchandar]

hi @electrum , PR as we discussed before.. Have tested this in house, against HoodieInputFormat.

@zhenxiao cc

[vinothchandar]

Ping Ping. :)

[vinothchandar]

@electrum Do you have any suggestions?

[zhenxiao]

@electrum @martint we've using this PR in production for a few months, could you please take a review when you are free?

[vinothchandar]

hi, any updates on this?

[vinothchandar]

@electrum @martint Just another friendly ping..

[vinothchandar]

@electrum Can you please take a look at the PR?

[electrum]

Sorry for the delay on this. I'll merge it after the 0.167 release.

[vinothchandar]

@electrum No worries.. I will rebase it again, when you are ready. Just lmk.

[electrum]

We can use streams for this:

return Arrays.stream(inputFormat.getClass().getAnnotations())
        .map(Annotation::annotationType)
        .map(Class::getSimpleName)
        .anyMatch(name -> name.equals("UseFileSplitsFromInputFormat"));

[electrum]

Merged, thanks!

[vinothchandar]

Thanks @electrum