Kamelet with id aws-s3-source not found in locations: file:/etc/camel/kamelets/kamelets-bundle-aws-s3-source-pipe-001,classpath:/kamelets

[matzew]

Now trying the aws-s3-source Kamelet, it looks like:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: aws-s3-source-pipe
spec:
  integration:
    dependencies:
    - "camel:cloudevents"
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: aws-s3-source
    properties:
      bucketNameOrArn: "soomebucket"
      accessKey: "the access key"
      secretKey: "the secret"
    dataTypes:
      out:
        format: aws2-s3:application-cloudevents
  sink:
    dataTypes:
      in:
        format: http:application-cloudevents
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: kamelet

However, in the pod for that "aws source" I get the following

2023-12-07 09:39:30,593 ERROR [org.apa.cam.qua.mai.CamelMainRuntime] (main) Failed to start application: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: aws-s3-source
	at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
	at org.apache.camel.support.service.BaseService.doFail(BaseService.java:407)
	at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:2996)
	at org.apache.camel.support.service.BaseService.fail(BaseService.java:336)
	at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:3949)
	at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2075)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:105)
	at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2081)
	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:313)
	at org.apache.camel.quarkus.main.CamelMain.doStart(CamelMain.java:94)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:113)
	at org.apache.camel.quarkus.main.CamelMain.startEngine(CamelMain.java:140)
	at org.apache.camel.quarkus.main.CamelMainRuntime.start(CamelMainRuntime.java:49)
	at org.apache.camel.quarkus.core.CamelBootstrapRecorder.start(CamelBootstrapRecorder.java:45)
	at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot173480958.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot173480958.deploy(Unknown Source)
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: aws-s3-source
	at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:433)
	at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2406)
	at org.apache.camel.quarkus.core.FastCamelContext.doInit(FastCamelContext.java:171)
	at org.apache.camel.support.service.BaseService.init(BaseService.java:78)
	at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2062)
	... 23 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id aws-s3-source not found in locations: file:/etc/camel/kamelets/kamelets-bundle-aws-s3-source-pipe-001,classpath:/kamelets
	at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:421)
	at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:430)
	... 27 more
Caused by: java.lang.IllegalArgumentException: Route template aws-s3-source the following mandatory parameters must be provided: region

But the kamelet is there:

k get kamelets.camel.apache.org aws-s3-source       
NAME            PHASE
aws-s3-source   Ready

I install Camel-K on Minikube, via https://github.com/matzew/eventing-cloud-integration/tree/main/system

[oscerd]

You need to set the region too, if you set accessKey and secretKey

  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: aws-s3-source
    properties:
      bucketNameOrArn: "soomebucket"
      accessKey: "the access key"
      secretKey: "the secret"
      region: "whatever"

It's reported in the error:
Caused by: java.lang.IllegalArgumentException: Route template aws-s3-source the following mandatory parameters must be provided: region

[matzew]

yeah, I just realized that too. Thanks!

After adding the region, I am getting:

aws-s3-source-pipe-7f5bb847b4-hspfx integration 2023-12-07 09:50:54,623 WARN  [org.apa.cam.com.aws.s3.AWS2S3Consumer] (Camel (camel-1) thread #1 - aws2-s3://arn:aws:s3:::SOMETHIG) Error processing exchange. Exchange[XXXXXXXXXXXX-XXXXXX]. Caused by: [org.apache.camel.CamelExecutionException - Missing data type transformer from null to type camel:aws2-s3:application-cloudevents on the exchange: Exchange[]]: org.apache.camel.CamelExecutionException: Missing data type transformer from null to type camel:aws2-s3:application-cloudevents on the exchange: Exchange[]
aws-s3-source-pipe-7f5bb847b4-hspfx integration 	at org.apache.camel.processor.tran

which reminds me of #1777

[oscerd]

yes, it's the same situation.

[matzew]

OK, than I am blocked.

Anyways, thanks for the help on both tickets.

[oscerd]

I hope we are able to release camel-k 2.2.0 soon @squakez

[matzew]

What I also do not fully like, is that the Pipe still returns it is ready:

k get pipes.camel.apache.org 
NAME                 PHASE   REPLICAS
aws-s3-source-pipe   Ready   1

While in the pod has an exception and it could be set to ready:false, with a reason for that, like other kube resources do

[oscerd]

What I also do not fully like, is that the Pipe still returns it is ready:

k get pipes.camel.apache.org 
NAME                 PHASE   REPLICAS
aws-s3-source-pipe   Ready   1

While in the pod has an exception and it could be set to ready:false, with a reason for that, like other kube resources do

This should go in the camel-k repository as issue I guess.

[matzew]

ok! will do

[squakez]

I hope we are able to release camel-k 2.2.0 soon @squakez

Yeah, more than on Camel K this depends on the runtime. I'm planning to cut a release for both during next weeks. Keep tuned.

[christophd]

@matzew here is one thing that you can try to unblock this until the new release is available.

Go to and apply this Kamelet to your local user namespace (where you run the Pipe):
https://github.com/apache/camel-kamelets/blob/4.1.x/kamelets/aws-s3-source.kamelet.yaml

Or overwrite the aws-s3-source Kamelet in the operator namespace with this.

This points to the Kamelet utils library version 4.1.0 which should have the data type available.

Please let me know if this works.

[matzew]

They both run in default namespace, and applying that resource there, does not fix it

[christophd]

Turns out this is related to camel-k#5047 because service discovery mechanism for data types has been broken. Fixed with camel-k#5055