chore: replaced kafka test images#2372
Conversation
abhilash-sivan
changed the title
docker-compose-base.yaml
Outdated
|
|
||
| zookeeper: | ||
| image: confluentinc/cp-zookeeper:7.9.0 | ||
| image: confluentinc/cp-zookeeper:7.9.1 |
There was a problem hiding this comment.
Aligned the image versions used in the sidecar for local development.
Since this Docker Compose setup is intended for developers and the project is open source, I have not changed the image reference in the Docker Compose file. Keeping the public image avoids forcing open-source contributors (who don’t have access to our internal registry) to manually locate and replace the image for their local usage.
I strongly believe this is the correct approach. The only required change was in the sidecars, and that has been completed.
There was a problem hiding this comment.
Oki.
Did you try these btw?
public.ecr.aws/aktosecurity/confluentinc-cp-kafka:7.9.1-1-ubi8
public.ecr.aws/aktosecurity/confluentinc-cp-zookeeper:7.9.2-1-ubi8
There was a problem hiding this comment.
No, haven't tried. any other non-official images
Can we use this in sidecars as well ? or just for dev use
aryamohanan
left a comment
aryamohanan
left a comment
There was a problem hiding this comment.
The Kafka image update also requires updating the image references in the Tekton tasks too
Yes, updated now! Let the pipeline run |
abhilash-sivan
force-pushed
the
chore-replace-kafka-images
branch
from
253a2ef to
a716de4
Compare
abhilash-sivan
changed the title
| } | ||
| }, | ||
| { | ||
| "name": "schema-registry", |
There was a problem hiding this comment.
Removed Schema Registry as it does not appear to be required for running the tests.
| kafka-topics: | ||
| image: confluentinc/cp-kafka:7.9.0 | ||
| image: public.ecr.aws/aktosecurity/confluentinc-cp-kafka:7.9.1-1-ubi8 | ||
| pull_policy: if_not_present |
There was a problem hiding this comment.
added pull policy for all test images
abhilash-sivan
force-pushed
the
chore-replace-kafka-images
branch
from
865c285 to
d354ba8
Compare
aryamohanan
left a comment
aryamohanan
left a comment
There was a problem hiding this comment.
Added minor comments. Preapproving
docker-compose-base.yaml
Outdated
| SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'zookeeper:2181' | ||
| SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: "PLAINTEXT://kafka:29092" | ||
| SCHEMA_REGISTRY_DEBUG: 'true' | ||
| # schema-registry: |
There was a problem hiding this comment.
qs: Could we remove it completely?
| sidecars: | ||
| - name: zookeeper | ||
| image: confluentinc/cp-zookeeper:7.9.1 | ||
| image: public.ecr.aws/aktosecurity/confluentinc-cp-zookeeper:7.9.2-1-ubi8 |
There was a problem hiding this comment.
qs: Could we use the same image version every where? like 7.9.2-1-ubi8? and why we can't use the latest (v8)image version?
There was a problem hiding this comment.
There are compatibility concerns with the latest v8 image since it is aligned with Apache Kafka v4.
However, our current dependencies use older client versions (ranging from 0.9 to 2.12). While some of them are compatible with Kafka v3, upgrading directly to Kafka v4 may introduce unexpected issues.
Additionally, during local testing, I noticed an issue with the v8 image. I haven’t investigated it in detail yet, but it adds some uncertainty.
For now, we can proceed with the latest available minor version in the 7.9 line:
public.ecr.aws/aktosecurity/confluentinc-cp-kafka:7.9.4-1-ubi8
This corresponds to Confluent Platform 7.9.x, which is stable and aligned with Kafka 3.x.
We can plan to move to the v8 (Kafka 4.x–based) image later if required, once compatibility and testing are validated.
abhilash-sivan
force-pushed
the
chore-replace-kafka-images
branch
from
7f1654c to
e621eeb
Compare
abhilash-sivan
changed the title
|
3 participants
refs https://jsw.ibm.com/browse/INSTA-73380