Allow S3 scheme for HelmRepository source

[rmakram-ims]

It is possible to create a helm repo inside an S3 bucket. It would be nice to be able to use this instead of the Bucket source.

[phillebaba]

@stefanprodan metioned in another issue with the same question but for GCP that a possible workaround is to use Chartmuseum.
#206 (comment)

I actually think this is a better solution to run Chartmuseum inside the cluster than having to implement support for every Chart repository source in source-controller. You can follow the s3 guide to setup chartmuseum or if you want to use their helm chart (very meta) you can use the following values.

env:
  open:
    AWS_SDK_LOAD_CONFIG: true
    STORAGE: amazon
    STORAGE_AMAZON_BUCKET: "bucket-name"
    STORAGE_AMAZON_PREFIX: ""
    STORAGE_AMAZON_REGION: "region-name"
serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: "role-arn"
securityContext:
  enabled: true
  fsGroup: 65534

Given that you deployed Chartmuseum in the flux-system namespace the following HelmRepository will work.

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: helm-charts
  namespace: flux-system
spec:
  interval: 1m
  url: http://chartmuseum-chartmuseum:8080

I hope this solves your problem 😃

[rogermakram]

Thanks @phillebaba for your answer.
I was wondering if you had any opinions on using the AWS ECR as a source for helm charts, I know that this is experimental.
https://docs.aws.amazon.com/AmazonECR/latest/userguide/push-oci-artifact.html
Thanks

[phillebaba]

So personally, if I have total control of the environment I would chose to store the charts in git, which allows me to use the non-compiled charts. Its simple and just works, but I get that its not an option for everyone.

I have a love hate relationship with OCI, I was very positive to the work being done with projects like CNAB which would standardize binary storage. The issue with Helm charts in OCI is that its become a mess of different implementations with plugins for all the different registry solutions. And now there is a "official" OCI feature in Helm that sometimes works and sometimes doesn't. For a while I was using the OCI feature with AKS but I had to stop, as it would stop working randomly between different Helm releases, making it impossible to debug.

If I were you I would wait until the feature has a stable release and just stick with S3 until then. Is there some feature that exists in ECR that does not exist in S3? If you really want to use ECR I would look into if there is a project like Chartmuseum that supports OCI backends or figure out how to get Chartmuseum to support OCI backends. I am not sure how easy/hard that would be to do.

[rogermakram]

The only reason I was looking into using ECR is to have only one storage location for both charts and images, which makes it easier in terms of permissions etc.
Using git is the simplest solution and does the job very well, but then this would only work if we have connectivity to this git repo; some of our work is done on a privately hosted VCS server with no connectivity to the EKS cluster.
Anyways, I understand that it would be a mess to add support for every chart repository source in this project, but could there be a mechanism to allow for importing helm plugins?
thanks

[stefanprodan]

We can't use OCI in source-controller since the Helm packages are marked as internal upstream. We can consider implementing this after Helm makes the code available for consumption, I guess this will happen after it's no longer experimental.