S3 dependencies

[johnpoth]

Solves #2132 and part of #1227. To use Maven dependencies hosted in a S3 repository the workflow would be something like this:

1. Create a custom Maven settings.xml as a Secret in Kubernetes containing the S3 credentials & host information:

<servers>
  <server>
    <id>minio-release</id>
    <username>291cafe6-eceb-43dc-91b3-58be867d9da2</username>
    <password>e383fed0-4645-45f6-acea-65f3748b96c8</password>
    <configuration>
      <wagonProvider>s3</wagonProvider>
      <s3Provider>minio</s3Provider>
      <endpoint>https://minio-tenant-1-hl.minio-tenant-1.svc.cluster:4430</endpoint>
    </configuration>
  </server>
  <server>
    <id>minio-snapshot</id>
    <username>291cafe6-eceb-43dc-91b3-58be867d9da2</username>
    <password>e383fed0-4645-45f6-acea-65f3748b96c8</password>
    <configuration>
      <wagonProvider>s3</wagonProvider>
      <s3Provider>minio</s3Provider>
      <endpoint>https://minio-tenant-1-hl.minio-tenant-1.svc.cluster:4430</endpoint>
    </configuration>
  </server>
</servers>

...

<repositories>
  <repository>
    <id>minio-release</id>
    <name>MinIO Release Repository</name>
    <url>s3://maven/release</url>
  </repository>
  <repository>
    <id>minio-snapshot</id>
    <name>MinIO Snapshot Repository</name>
    <url>s3://maven/snapshot</url>
  </repository>
</repositories>

2. Optionally create the S3 repository CA as a Secret in Kubernetes
3. Reference all the above plus the S3 wagon when installing camel-k, so something like:
   kamel install --maven-settings secret:camel-k-minio-maven-settings/maven-settings --maven-ca-secret minio-ca/minio-ca --maven-build-extension fi.yle.tools:aws-maven:1.4.2
4. Reference the dependencies hosted in S3 just like any other dependencies e.g
   `kamel run examples/languages/Sample.java --dependency=mvn:hosted:ons3:1.1

Tested with MinIO operator 4.0.11

Thanks !

Release Note

NONE

[astefanutti]

Awesome, I've left a couple of comments. Thanks!

[astefanutti]

Also, that'd be awesome to have a doc written about your MinIO operator testing, maybe in a section of the https://camel.apache.org/camel-k/latest/configuration/maven.html page.

[johnpoth]

Thanks for reviewing ! I'll add some documentation to https://camel.apache.org/camel-k/latest/configuration/maven.html ASAP; the hardest part was finding where Kubernetes hides the CA it uses to sign CertificateSigningRequest...

[astefanutti]

Thanks for reviewing ! I'll add some documentation to https://camel.apache.org/camel-k/latest/configuration/maven.html ASAP; the hardest part was finding where Kubernetes hides the CA it uses to sign CertificateSigningRequest...

Ah right, it uses the Kubernetes TLS API. I think the location of the key/CA used to signed the certificate is configurable and can differ from one k8s distribution to the other.

On OpenShift, this would be a very good case for using the service signing certificate service, to generate the certificate, as the CA is automatically mounted into Pods. It seems it's possible to provide our own certificate to the MinIO Operator: https://docs.min.io/minio/k8s/tutorials/transport-layer-security.html#id2. As this is a possible solution to hosting customs dependencies and beans, It'd be valuable to also try/document that approach for downstream.

[johnpoth]

Ah right as you pointed out perhaps it's best to leave the "How do I make certificates work in Kubernetes"™ part out of it as it depends on the Kubernetes/S3 distributions and how they were setup...

The OpenShift/MinIO/Camel-k CA setup you described looks really promising!

[astefanutti]

Ah right as you pointed out perhaps it's best to leave the "How do I make certificates work in Kubernetes"™ part out of it as it depends on the Kubernetes/S3 distributions and how they were setup...

Yes, both approaches would be worth documenting.

The OpenShift/MinIO/Camel-k CA setup you described looks really promising!

+1

Let me know if you prefer to have that PR merged, or you prefer to have it left open to work on the documentation.

[johnpoth]

Added, thanks !

[astefanutti]

Awesome!

[astefanutti]

convered -> covered