- Security:
- 2-way SSL handshake between Zookeeper and Kafka Brokers.
- SASL_SSL security protocol between all the brokers.
- SASL_SSL security protocol between Kafka Brokers and its external clients(producers or consumers).
- Producer/Consumer:
- Used Confluent Python wrapper to implement Producer and Consumer applications.
- Implemented SASL_SSL security protocol between Kafka Brokers and Producer/Consumer.
- In producer, we have used the Faker library to generate new names.
- In producer, with security-related configurations, we have also added configurations for batching(batch.max.messages or batch.size), acks, linger.ms, queue size(queue.buffering.max.messages), etc.
- Below is an example showing 50 records sent from the Producer application to the Kafka Cluster and received by the Consumer application.
