Support boolean values in additionalProperties of Kafka kamelet

[vedran-kolka-syntio]

Hi! It would be great if additionalProperties in the kafka component would support boolean values when writing in YAML DSL.
Right now, the following route:

- route:
    id: "kafka-to-somewhere"
    from:
      uri: "kafka:{{kafka.my-topic}}"
      parameters:
        additionalProperties.specific.avro.reader: true
        ....

fails with the following exception:

Caused by: java.lang.ClassCastException: class java.lang.String cannot be cast to class java.lang.Boolean (java.lang.String and java.lang.Boolean are in module java.base of loader 'bootstrap')

Stacktrace:

org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:825) ~[kafka-clients-3.2.3.jar:?]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666) ~[kafka-clients-3.2.3.jar:?]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:647) ~[kafka-clients-3.2.3.jar:?]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:627) ~[kafka-clients-3.2.3.jar:?]
        at org.apache.camel.component.kafka.DefaultKafkaClientFactory.getConsumer(DefaultKafkaClientFactory.java:34) ~[camel-kafka-3.20.6-fix.jar:3.20.5-SNAPSHOT]
        at org.apache.camel.component.kafka.KafkaFetchRecords.createConsumer(KafkaFetchRecords.java:245) ~[camel-kafka-3.20.6-fix.jar:3.20.5-SNAPSHOT]     
        at org.apache.camel.component.kafka.KafkaFetchRecords.createConsumerTask(KafkaFetchRecords.java:205) ~[camel-kafka-3.20.6-fix.jar:3.20.5-SNAPSHOT] 
        at org.apache.camel.support.task.ForegroundTask.run(ForegroundTask.java:93) ~[camel-support-3.20.5-SNAPSHOT.jar:3.20.5-SNAPSHOT]
        at org.apache.camel.component.kafka.KafkaFetchRecords.run(KafkaFetchRecords.java:127) ~[camel-kafka-3.20.6-fix.jar:3.20.5-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: java.lang.ClassCastException: class java.lang.String cannot be cast to class java.lang.Boolean (java.lang.String and java.lang.Boolean are in module java.base of loader 'bootstrap')
        at com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializerConfig.getAvroSpecificReader(KafkaAvroDeserializerConfig.java:32) ~[azure-schemaregistry-kafka-avro-1.1.1.jar:?]
        at com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializer.configure(KafkaAvroDeserializer.java:57) ~[azure-schemaregistry-kafka-avro-1.1.1.jar:?]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:718) ~[kafka-clients-3.2.3.jar:?]
        ... 13 more

[davsclaus]

Thanks, yeah we need to improve this - apache kafka client is able to automatic convert these config values, but MS Azure is not able to

[oscerd]

One thing we could do in the future is providing some basic buidling block to connect to different types of schema registry.

[davsclaus]

I created a ticket
https://issues.apache.org/jira/browse/CAMEL-19393

[davsclaus]

Okay this requires some help for MS Azure, so instead of

additionalProperties.specific.avro.reader: true

You would then need to:

additionalProperties.specific.avro.reader: '#valueAs(boolean):true'

This will be in the next releases (3.20.6 onwards)

[davsclaus]

You can use #valueAs(type):actualValueHere in general in camel now that allows this flexibility.

I can see in the source code for MS Azure then they dont have like Kafka a way to automatic convert, or to specify in the code what type the options are. You have to look inside, and some are (Integer) also etc.

[davsclaus]

We could hardcode special conversions inside camel-kafka that check for known types for MS Azure, but then we risk getting out of sync.

[vedran-kolka-syntio]

When You say "in camel now", do You mean this is already available or that it is coming with Camel 3.20.6?

Also, are there any resources where these special references in YAML DSL (#class, #bean, #valueAs, ...) are documented? I am having trouble finding any.

[davsclaus]

Camel uses property binding
https://camel.apache.org/manual/property-binding.html

And it started also earlier before yaml with setting up beans in properties file which has some more smaller snippets
https://camel.apache.org/components/3.20.x/others/main.html#_specifying_custom_beans

[davsclaus]

When You say "in camel now", do You mean this is already available or that it is coming with Camel 3.20.6?
3.20.6 we are adding #valueAs

[davsclaus]

https://issues.apache.org/jira/browse/CAMEL-19395

[davsclaus]

There is also an exisiting option (although intended for confluent only, but its general)

From

additionalProperties.specific.avro.reader: true

To

specificAvroReader: true

That should work on 3.20.4 / 3.20.5 release

[davsclaus]

closing as this is general camel issue and not kamelet