How to use a custom Kamelet Repository?

[k0pper]

I am trying to work with a custom kamelet repository, instead of https://github.com/apache/camel-kamelets.

Using the helm chart of Camel K: https://artifacthub.io/packages/helm/camel-k/camel-k, I am trying to set the Property in the IntegrationPlatform Resource of Camel K to modify the repositories property.

My questions are:

• What kind of repository does this field expect? Is a github-repository with the same structure as apache/camel-kamelets enough?
• According to the IntegrationPlatformSpec, the repositories field is an array, why? Is it possible to set multiple kamelet repisitories?
• Could you document how to set custom repositories?
• What is the expected way to modify the IntegrationPlatform.yaml? I am trying to change the kamelets repository with helm install --namespace camel-k --create-namespace --generate-name --set platform.build.registry.address=${REGISTRY_ADDRESS} --set platform.build.registry.insecure=true --set platform.kamelet.repositories[0].uri=https://github.com/k0pper/custom-kamelets camel-k/camel-k but I get this error: zsh: no matches found: platform.kamelet.repositories.uri[0]=

I am highly thankful for getting support at this case. My general concern is that Kamelets are capable of doing a lot of stuff I feel like, but it is barely documented.

[k0pper]

Update: The command helm install --namespace camel-k --create-namespace --generate-name --set platform.build.registry.address=${REGISTRY_ADDRESS} --set platform.build.registry.insecure=true --set "platform.kamelet.repositories[0].uri=github.com/k0pper/custom-kamelets" camel-k/camel-k worked and the repository is set inside the IntegrationPlatform.yaml inside the cluster, but when I view the Kamelets inside the cluster I still have the default Kamelets from the default repository without the custom one

~ ➤ k get kamelet -n camel-k
NAME                               PHASE
avro-deserialize-action            Ready
avro-serialize-action              Ready
aws-cloudwatch-sink                Ready
aws-ddb-streams-source             Ready
aws-ec2-sink                       Ready
(...)

[k0pper]

... When trying to deploy a KameletBinding:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: my-earthquake-log
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: earthquake-source
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: log-sink

I get the error that the URI for the kamelet repository was invalid:

{"level":"info","ts":1637588929.720187,"logger":"camel-k.trait","msg":"Apply trait: init"}                                                                                            
{"level":"info","ts":1637588929.7207434,"logger":"camel-k.trait","msg":"Apply trait: camel"}                                                                                          
{"level":"info","ts":1637588929.7224216,"logger":"camel-k.trait","msg":"Apply trait: kamelets"}                                                                                      
{"level":"error","ts":1637588929.7234652,"logger":"controller-runtime.manager.controller.integration-controller","msg":"Reconciler error","name":"my-earthquake-log","namespace":"camel-k","error":"error during trait customization: invalid uri: github.com/k0pper/custom-kamelets","errorVerbose":"invalid uri: github.com/k0pper/custom-kamelets\nerror during trait customization

[k0pper]

/bump

The definition of a custom repository seems as an intended feature to be used. Can someone take a look?

[squakez]

I think the problem is related the format of the external repository. It must be github:owner/repo, so, in your case:

  kamelet:
    repositories:
    - uri: github:k0pper/custom-kamelets

[squakez]

@astefanutti @nicolaferraro @oscerd I haven't found any documentation, but some bit on the code to understand how to use it. Are we supporting this feature? If supported, we can add some note to documentation. Also, I think we should simplify the way to configure a repository, ie, providing a kamel kamelet add-repo command or something similar. Wdyt?

[oscerd]

Not sure if this is supported or wanted at the moment. By the way, if so, we should add a much cleaner command and made it easier.

[k0pper]

Can you take a look at my repository https://github.com/k0pper/custom-kamelets and tell me why it's not finding the kamelet in my repository?

{"level":"error","ts":1643721322.4948833,"logger":"controller.integration-controller","msg":"Reconciler error","reconciler group":"camel.apache.org","reconciler kind":"Integration","name":"my-earthquake-log","namespace":"camel-k","error":"error during trait customization: kamelets log-sink found, my-earthquake-source-two not found in repositories: (Kubernetes[namespace=camel-k], Github[owner=k0pper, repo=custom-kamelets, path=, ref=])"

[k0pper]

I figured it out by myself.

For everyone also wanting to implement an own Kamelet-Catalog: it works, but you have to find a fitting service which can handle that.

My fix:

Change the integrationplatform resource yaml to

  kamelet:
    repositories:
    - uri: github:YOUR_GITHUB_USERNAME/YOUR_REPO_NAME/PATH_TO_KAMELETS_FOLDER

(e.g. github:k0pper/custom-kamelets/kamelets)

After that, when you deploy a KameletBinding, it looks for all repositories that exist (the official kamelet catalog + your custom one). GitHub seems unfitting though, because the API Rate Limit exceeded pretty quickly, I don't know why.

{"level":"error","ts":1643722341.1441336,"logger":"controller.integration-controller","msg":"Reconciler error","reconciler 
group":"camel.apache.org","reconciler kind":"Integration","name":"my-earthquake-log","namespace":"camel-
k","error":"error during trait customization: GET https://api.github.com/repos/k0pper/custom-kamelets/contents/kamelets: 
403 API rate limit exceeded for <ip address hidden>. (But here's the good news: Authenticated requests get a higher rate limit. 
Check out the documentation for more details.)

The referenced custom kamelets-repository works.

Alex