Skip to content

Terraform root module which demonstrates deployment and usage of Kafka on HSDP

Notifications You must be signed in to change notification settings

philips-labs/kafka-on-hsdp

Repository files navigation

Kafka on HSDP

This Terraform root module demonstrates usage of a Kafka Cluster provisioned on Container Host. We also spin up a Kafdrop instance as an example app hosted on Cloud foundry

NOTE: authentication/authorization is mTLS-SSL. you will need to create you own certificates.

Prerquisites

Requirements

Name Version
cloudfoundry >= 0.14.2
hsdp >= 0.18.6

Providers

Name Version
cloudfoundry >= 0.14.2

Modules

Name Source Version
kafka philips-labs/kafka/hsdp 0.8.0
zookeeper philips-labs/zookeeper/hsdp 0.8.0

Resources

Name Type
cloudfoundry_app.kafdrop resource
cloudfoundry_route.kafdrop resource
cloudfoundry_domain.domain data source
cloudfoundry_org.org data source
cloudfoundry_space.space data source

Inputs

Name Description Type Default Required
bastion_host Bastion Host to use (region specific) string "gw-eu1.phsdp.com" no
cartel_host Cartel API host (region specific) string n/a yes
cartel_secret Cartel secret to use for Container Host string n/a yes
cartel_token Cartel token to use for Container Host string n/a yes
cf_api Cloud foundry API endpoint (region specific) string n/a yes
cf_domain Default Cloud foundry domain to use for apps string n/a yes
cf_org Cloud foundry ORG name string n/a yes
cf_password Cloud foundry password string n/a yes
cf_region Cloud foundry region string "eu-west" no
cf_space Cloud foundry space to provision Kafdrop in string n/a yes
cf_username Cloud foundry username string n/a yes
kafdrop_key_store_file file path of the kafkadrop key store string n/a yes
kafka_instance_type The EC2 instance type to use for Kafka nodes string "t3.large" no
kafka_key_store_file file path of the kafka key store string n/a yes
kafka_nodes Number of Kafka nodes to spin up number 1 no
kafka_properties_file file path of the kafka properties file string n/a yes
kafka_trust_store_file file path of the kafka trust store string n/a yes
name_postfix Optional postfix to append in routes string "" no
private_key_file SSH private key. Used to access SSH bastion host string n/a yes
ssl_pass one pass for all of the keys and stores string n/a yes
zoo_key_store_file file path of the zookeeper key store string n/a yes
zoo_trust_store_file file path of the zookeeper trust store string n/a yes
zookeeper_instance_type The EC2 instance type to use for Zookeeper servers string "t2.medium" no
zookeeper_nodes Number of Zookeeper nodes to spin up number 1 no

Outputs

Name Description
kafdrop_url n/a

key generation

Look at the (https://github.com/bitnami/bitnami-docker-kafka) for how to. there is a (https://raw.githubusercontent.com/confluentinc/confluent-platform-security-tools/master/kafka-generate-ssl.sh) which provides the necessary results.

example kafka.properties file for kafdrop and possilby your clients

kafka.securityprotocol=SSL
ssl.security.protocol=SSL
ssl.endpoint.identification.algorithm=
ssl.truststore.password=<yourpass>
ssl.truststore.type=JKS
ssl.keystore.password=<yourpass>
ssl.keystore.type=JKS
ssl.key.password=<yourpass>

Contact / Getting help

Post your question on the HSDP Slack #terraform channel

License

License is MIT

About

Terraform root module which demonstrates deployment and usage of Kafka on HSDP

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published