Configs for CamelAzurestorageblobsinkSinkConnector

[tgib23]

Hi team,

I'm trying CamelAzurestorageblobsinkSinkConnector and just confirmed the file is created under the container specified by camel.kamelet.azure-storage-blob-sink.containerName.

But is there any way to specify the path under the container, file name, and file extension?

[tgib23]

Hi @oscerd
Do you think the content of this ticket is to be solved as a consequence of CAMEL-19395?
https://issues.apache.org/jira/browse/CAMEL-19395

[oscerd]

It's not related

[tgib23]

@oscerd
Thanks for clarifying.
Back to my original question, do you know if there is any option to specify the path/filename/extension for Azure Blob Sink?

[oscerd]

Need to double check

[oscerd]

This could be done by specifying a Camel.file header on each message so you could directory/filename.ext

[tgib23]

thanks. i'll check and update here

[tgib23]

@oscerd
I've set the record CamelHeader.CamelFilename and CamelHeader.CamelFilePath as shown below, but it didn't change the behavior. Anything I'm missing? or any other ideas?

[kafka@kafka-team-cluster-kafka-0 kafka]$ ./bin/kafka-console-consumer.sh --topic test-logs --bootstrap-server localhost:9092 --property print.headers=true
CamelHeader.CamelFileName:camel_test,CamelHeader.CamelFilePath:test/year=2023/month=06  {"value":"2023-06-08_07:38:50 test message"}

[oscerd]

It's CamelHeader.CamelAzureStorageBlobBlobName

Please read the documentation: https://camel.apache.org/components/3.20.x/azure-storage-blob-component.html#_endpoint_header_CamelAzureStorageBlobBlobName

[tgib23]

Hi @oscerd
Sorry for asking many times, but setting the message headers hasn't had effect and files are uploaded directly under the container with the file name like E55515147C81BF6-0000000000000001.

Here are the messages again. Do you have any other ideas? I went through the docs, but couldn't find a clue yet.

[kafka@kafka-team-cluster-kafka-0 kafka]$ ./bin/kafka-console-consumer.sh --topic test-logs --bootstrap-server localhost:9092 --property print.headers=true
CamelHeader.CamelAzureStorageBlobOperation:uploadBlockBlob,CamelHeader.CamelAzureStorageBlobBlobName:camel_test,CamelHeader.CamelAzureStorageBlobFileDir:testdir	{"value":"2023-06-13_06:41:03 test value"}
CamelHeader.CamelAzureStorageBlobOperation:uploadBlockBlob,CamelHeader.CamelAzureStorageBlobBlobName:camel_test,CamelHeader.CamelAzureStorageBlobFileDir:testdir	{"value":"2023-06-13_06:42:03 test value"}
...

[oscerd]

I think it is overriden by the Kamelet: can you try with

CamelHeader.file=camel_test ?

Also please note the fileDir header is related to the downloadBlobToFile operation and not the uploadBlockBlob.

[tgib23]

@oscerd

Thanks! it worked, and I can express the folder hierarchies, like CamelHeader.file:year=2023/month=06/day=16/file_name.

Where can I find the possible other message headers configurations? I'd like to try them.

[oscerd]

Hello @tgib23

For the particular use case, the file header is specified in the Kamelet block of code here: https://github.com/apache/camel-kamelets/blob/main/kamelets/azure-storage-blob-sink.kamelet.yaml#L70-L86

So it will be mapped to the following header: CamelAzureStorageBlobBlobName

So this is a case where the Kamelet overrides the value, with a header passed to it.

Talking in general the headers for this component are here: https://camel.apache.org/components/3.20.x/azure-storage-blob-component.html#_message_headers

Don't forget that you always need to prefix CamelHeader. to the header you're going to use.

[oscerd]

I'm closing this, but please re-open or report more issues if needed.