TWITTER INDEXER

---

This project is a POC for playing with Spring Kafka, ElasticSearch, and Twitter API. Docker is multi-stage with the layered spring boot jar.

Usage via docker

Twitter-indexer

• cd twitter-indexer-producer
• ./mvnw clean install
• docker-compose -f docker/docker-compose.yml up --build
• create & fill out the docker/.env file with the missing secrets based on the env-vars.tpl template file
• docker-compose -f docker/twitter-indexer-compose.yml up --build

Twitter-consumer

• cd twitter-consumer
• ./mvnw clean install
• docker-compose -f docker/docker-compose.yml up --build
• docker-compose -f docker/twitter-consumer-compose.yml up --build

Note: twitter api secrets are required to fire the service up! env-vars.tpl file contains all of them to show an
example about them. That means that you need a Twitter developer account with a Twitter App.

Tech Stack

• Spring Boot (2.2.5)
• Liquibase
• Postgres
• ElasticSearch
• Twitter developed HBC API
• SchedLock
• Spring Kafka
• Docker
• Maven

Inspirations

• ElasticSearch doc
• Spring Kafka batch listener config
• Multi-stage Dockerfile
• Multi-stage Dockerfile with Spring-boot