Does Strimzi-bridge works with AWS-MSK ?

[sumanta1974]

Does Strimzi-bridge works with AWS-MSK , with all version and all protocol , strimzi bridge is not able to create consumer in AWS MSK

[ppatierno]

It should works with any Kafka cluster. I don't know AWS-MSK if you need to configure authentication as well and how you are doing with the bridge. Having bridge configuration, how you are deploying it, what you see in the log and anything else would be a good starting point.

[sumanta1974]

we are not seeing any error and we are using mTLS as per AWS documentation and as suggested by CNCF group https://docs.aws.amazon.com/msk/latest/developerguide/msk-authentication.html , but even after connection , it is not able to create consumer

Subscribing alpha-summaryBase-test-cg1-c1 to the Strimzi Bridge
curl -d {"name":"alpha-summaryBase-test-cg1-c1","auto.offset.reset":"latest","format":"json","enable.auto.commit":false,"fetch.min.bytes":512,"consumer.request.timeout.ms":3000} -H Content-Type: application/vnd.kafka.v2+json -k -X POST https://localhost:8090/test/alpha1/consumers/alpha-summaryBase-test-cg1
{"instance_id":"alpha-summaryBase-test-cg1-c1","base_uri":"://api-staging.arity.com/alpha1/consumers/alpha-summaryBase-test-cg1/instances/alpha-summaryBase-test-cg1-c1"}curl -i --data-binary {"topics": [ "iotp.alpha.trip.summaryBase.processed.json" ]} -H Content-Type: application/vnd.kafka.v2+json -k -X POST https://localhost:8090/test/alpha1/consumers/alpha-summaryBase-test-cg1/instances/alpha-summaryBase-test-cg1-c1/subscription
HTTP/1.1 204 No Content
Server: nginx
Date: Tue, 21 Feb 2023 04:23:04 GMT
Connection: keep-alive

[ppatierno]

AFAICS the consumer was created ... and the address was returned in base_uri
You also subscribed.
What were you expecting?

Now you should start to consume.
Takes into account that messages will show up after the first 2-3 polls because the initial ones are needed to the consumer to join the consumer group.

Refer to the official documentation to get how it works here https://strimzi.io/docs/bridge/latest/

PS: please reply not suggest another answer. It's not how Github discussion works.

[sumanta1974]

I am expecting consumer to be created , can you join a zoom call as I can't share all log here

[sumanta1974]

I am seeing response

02/21/2023 04:27:01 AM|INFO |healthy | [1752083214] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:27:10 AM|INFO |healthy | [1254859480] HEALTHY Request: from 127.0.0.1:58668, method = GET, path = /healthy
02/21/2023 04:27:10 AM|INFO |healthy | [1254859480] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:27:20 AM|INFO |healthy | [915300340] HEALTHY Request: from 127.0.0.1:52272, method = GET, path = /healthy
02/21/2023 04:27:20 AM|INFO |healthy | [915300340] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:27:30 AM|INFO |healthy | [524683944] HEALTHY Request: from 127.0.0.1:38644, method = GET, path = /healthy
02/21/2023 04:27:30 AM|INFO |healthy | [524683944] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:27:40 AM|INFO |healthy | [804017309] HEALTHY Request: from 127.0.0.1:43540, method = GET, path = /healthy
02/21/2023 04:27:40 AM|INFO |healthy | [804017309] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:27:50 AM|INFO |healthy | [231626865] HEALTHY Request: from 127.0.0.1:58520, method = GET, path = /healthy
02/21/2023 04:27:50 AM|INFO |healthy | [231626865] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:00 AM|INFO |healthy | [1678058075] HEALTHY Request: from 127.0.0.1:56614, method = GET, path = /healthy
02/21/2023 04:28:00 AM|INFO |healthy | [1678058075] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:10 AM|INFO |healthy | [1194373762] HEALTHY Request: from 127.0.0.1:38724, method = GET, path = /healthy
02/21/2023 04:28:10 AM|INFO |healthy | [1194373762] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:20 AM|INFO |healthy | [1764311032] HEALTHY Request: from 127.0.0.1:57204, method = GET, path = /healthy
02/21/2023 04:28:20 AM|INFO |healthy | [1764311032] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:30 AM|INFO |healthy | [1033413389] HEALTHY Request: from 127.0.0.1:40420, method = GET, path = /healthy
02/21/2023 04:28:30 AM|INFO |healthy | [1033413389] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:40 AM|INFO |healthy | [38899309] HEALTHY Request: from 127.0.0.1:45526, method = GET, path = /healthy
02/21/2023 04:28:40 AM|INFO |healthy | [38899309] HEALTHY Response: statusCode = 200, message = OK
02/21/2023 04:28:50 AM|IN

[ppatierno]

Again you added a new answer instead of replying, please don't do that

I am expecting consumer to be created what do you mean? where? Did you read my answer? Did you read the documentation?

[sumanta1974]

I was able to see consumer created after few GET operations , you can close the issue

[ppatierno]

That's exactly what I said before you didn't follow.
Reporting here again to mark as the answer.

AFAICS the consumer was created ... and the address was returned in base_uri
You also subscribed.
What were you expecting?

Now you should start to consume.
Takes into account that messages will show up after the first 2-3 polls because the initial ones are needed to the consumer to join the consumer group.