This project to consume Kafka messages using Apache Beam
- Java v8
$ brew cask install java8
- Gradle v5.0
To install gradle please refer
$ ./gradlew clean test
$ ./gradlew build
$ ./gradlew clean build jacocoTestReport
$ ./gradlew clean build check
Fill all pipeline parameters in file deploy_pipeline.sh :
--project=
--kafkaBrokers=
--kafkaTopic=
--downstreamGcs=
--jobName=
--runner=DataflowRunner
--network=
--subnetwork=
--zone=
--tempLocation=
--stagingLocation=
--filesToStage"
run pipeline by execute file:
$ ./deploy_pipeline.sh
Syarif Hidayataullah