Add AWS CloudWatch Kamelet

[claudio4j]

#116
When regenerating the doc, it updated the aws s3 doc.

[oscerd]

Lgtm cc @nicolaferraro

[nicolaferraro]

Some issues highlighted.. open to different suggestions...

[nicolaferraro]

As general rule, we don't use Camel* headers in Kamelets, since these are higher level objects that hide their Camel engine. Usually we have a few headers, but here we have many more of them, so the mapping may be verbose, since e.g. we may need to detect both e.g. metric-namespace and ce-metric-namespace headers.

One reason is that we want to hide camel, the other is that in some contexts, e.g. knative, you are able to only send cloud-event extensions, which are mapped to headers that start with the "ce-" prefix.

A different issue is with the metric-timestamp header. Consider that (from the doc) CamelAwsCwMetricTimestamp needs to be a java.util.Date. When this sink is bound to a Knative channel or a Kafka topic, all headers are textual, so we need to define the format of the header and its conversion from text to date.

Out of curiosity, is the body of the incoming message used? Maybe we should we use a structured body instead of relying on headers.

[claudio4j]

As general rule, we don't use Camel* headers in Kamelets,

Kamelets should not expose those camel headers,but can set them in - set-property steps ?

The incoming body is empty, the camel-aws2-cw component uses only the values from the header or query parameters.
Would the structured body be something like this ?

{namespace: "my-aws-ns",
metric-name: "my-metric",
metric-value: "1.2",
...
}

Then the to:() directive would use the body structure like the following ?

.to("aws2-cw:${body[namespace]}?name=${body[metric-name]}")

About the timestamp conversion, would it be using the simple("${headerAs(key, type)}") ?

[nicolaferraro]

Yes, we may use more user-friendly names for the properties by remapping them in set-property steps, as other kamelets are doing.

There are many possibilities about the body, we can keep it empty. Or we can expect the body to contain the metric-value header if that is missing. Or, we can detect if the body is a simple value or a structured JSON, in which case we can unmarshal it and use set-header with simple or jsonpath to get camel headers from the structured body.
Anyway, it's also ok to rely on headers for a first release, then add a different logic in the future. Sounds like it's better to improve the component to optionally accept a structured body rather than adding it here.

The timestamp conversion may be possible with simple. But looking here https://camel.apache.org/components/latest/languages/simple-language.html, I see only methods for formatting. I think e.g. "number of milliseconds since epoch" as java.lang.Long can be automatically converted by Camel to java.util.Date, so we may use that format here for simplicity.

[oscerd]

All of these options on the endpoint can be set as headers first. Headers have precedence over the endpoint option. https://github.com/apache/camel/blob/master/components/camel-aws/camel-aws2-cw/src/main/java/org/apache/camel/component/aws2/cw/Cw2Producer.java#L124

So it would be better to set all of this as property or header, like here:
https://github.com/apache/camel-kamelets/blob/master/azure-eventhubs-sink.kamelet.yaml#L57

Especially the metric value would never be constant, so it doesn't make really sense to set it as an endpoint option.

I suggest to set all the dynamic fields as header or property following the example I reported above.

[oscerd]

What is the status of this @claudio4j ?

[claudio4j]

What is the status of this @claudio4j ?

I had other priorities and put this work on hold. But I am back into it today and expect to push the changes soon.

[claudio4j]

@oscerd I pushed the required changes, can you review ?

[oscerd]

@nicolaferraro please have a look again. Looks good to me.

[claudio4j]

@nicolaferraro can you review this kamelet ?

[nicolaferraro]

Thanks, can you check the conflicts?

[claudio4j]

Thanks, can you check the conflicts?

Done. Thanks.

[oscerd]

Thanks.