ElasticSearch Pagination by using Kafka

Start

Let's first start the ElasticSearch in docker:

docker-compose -f docker-compose-es.yml up -d

Then start your kafka cluster

docker-compose -f docker-compose-kafka.yml up -d

This is just a simple example for kafka docker-compose

Now you can start the application, you can check it by calling localhost:8080 from your browser.

Let's insert some dummy data in your index, by running following line after you start the application. (100K documents)

curl http://localhost:8080/test/init

Start the pagination by calling the kafka producer

curl http://localhost:8080/test

If you check the application log you ll notice the asynchronous call into ElasticSearch by multiple consumers.