Add a MinIO plugin replicating S3 plugin's functionality

[aku]

Feature description

Add s3 option for a io.kestra.plugin.fs plugin to be able to upload/download files from/to s3 storages.
Currently, you have AWS S3 plugin but it does not seem to support a custom s3 endpoint

[anna-geller]

Can you try the following?

• https://kestra.io/plugins/plugin-aws/tasks/s3/io.kestra.plugin.aws.s3.upload#endpointoverride
• https://kestra.io/plugins/plugin-aws/tasks/s3/io.kestra.plugin.aws.s3.downloads#endpointoverride

This should work, if not, this might be a bug.

[immutablesemaphore]

For anyone who comes across this, here is one way that worked with MinIO locally

- id: upload_to_minio
  type: "io.kestra.plugin.aws.s3.Upload"
  compatibilityMode: true
  bucket: my-bucket
  from: '{{ outputs.someStep.outputFiles["some-file.json"] }}'
  key: "some-file.json"
  accessKeyId: "xxx"
  secretKeyId: "xxx"
  endpointOverride: "http://172.22.0.3:9000"
  region: "eu-west-1"

[aku]

Minio works, thanks. Probably, documentation can be improved.
Currently, it discourages use of endpointOverride and compatibilityMode options

This property should normally not be used except for local development.

which maximizes compatibility with S3-compatible services but restricts uploads and downloads to 2GB

[immutablesemaphore]

For my tests, I was able to remove compatibilityMode and it still ran against a local MinIO without an issue. It was an unnecessary inclusion in this case.

[loicmathieu]

Fixed by kestra-io/plugin-minio#2

[agoddard]

fyi it seems that compatibilityMode: true is required with some S3 endpoints, I tested against CloudFlare R2 and got software.amazon.awssdk.services.s3.model.S3Exception: STREAMING-UNSIGNED-PAYLOAD-TRAILER not implemented (Service: S3, Status Code: 501, Request ID: null) when it was not set, but things worked fine once it was set.

[anna-geller]

added the info here, thx @agoddard kestra-io/plugin-aws@a7cd02d