Added the anonymous credentials provider to S3PinotFS

[rajagopr]

The anonymous credentials provider is useful to ingest from public buckets for test and demo purposes.

Testing

Tested locally with the following sample job spec.

executionFrameworkSpec:
    name: 'standalone'
    segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
    segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
    segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
    segmentMetadataPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentMetadataPushJobRunner'
jobType: SegmentCreationAndTarPush 
inputDirURI: 's3://my.public.bucket/'
outputDirURI: '/path/to/output/dir'
overwriteOutput: true
pinotFSSpecs:
    - scheme: s3
      className: org.apache.pinot.plugin.filesystem.S3PinotFS
      configs:
        region: 'us-west-2'
        anonymousCredentialsProvider: true
recordReaderSpec:
    dataFormat: 'csv'
    className: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReader'
    configClassName: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReaderConfig'
tableSpec:
    tableName: 'myTable'
    tableConfigURI: 'http://localhost:9000/tables/myTable'
pinotClusterSpecs:
    - controllerURI: 'http://localhost:9000'
pushJobSpec:
  pushAttempts: 2
  pushRetryIntervalMillis: 1000

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 63.72%. Comparing base (59551e4) to head (02d5e8d).
Report is 1611 commits behind head on master.

Files with missing lines	Patch %	Lines
.../org/apache/pinot/plugin/filesystem/S3PinotFS.java	0.00%	2 Missing ⚠️
...a/org/apache/pinot/plugin/filesystem/S3Config.java	66.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #14889      +/-   ##
============================================
+ Coverage     61.75%   63.72%   +1.97%     
- Complexity      207     1471    +1264     
============================================
  Files          2436     2708     +272     
  Lines        133233   151360   +18127     
  Branches      20636    23360    +2724     
============================================
+ Hits          82274    96458   +14184     
- Misses        44911    47662    +2751     
- Partials       6048     7240    +1192     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (+99.99%)	⬆️
integration	100.00% <ø> (+99.99%)	⬆️
integration1	100.00% <ø> (+99.99%)	⬆️
integration2	0.00% <ø> (ø)
java-11	63.70% <40.00%> (+1.99%)	⬆️
java-21	63.62% <40.00%> (+2.00%)	⬆️
skip-bytebuffers-false	63.72% <40.00%> (+1.97%)	⬆️
skip-bytebuffers-true	63.60% <40.00%> (+35.87%)	⬆️
temurin	63.72% <40.00%> (+1.97%)	⬆️
unittests	63.72% <40.00%> (+1.97%)	⬆️
unittests1	56.30% <ø> (+9.41%)	⬆️
unittests2	34.04% <40.00%> (+6.31%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.