Real-Time Analytics on HTTP Access Logs

Clone repository:

git clone git@github.com:mneedham/analysing-log-files.git
cd analysing-log-files

Launch Docker Compose:

docker-compose up

Generate log files:

python apache-fake-log-gen.py

faust -A app worker -l info

Query the access logs stream:

kcat -C -t access -b localhost:9092

Add Pinot Table

docker run -v $PWD/pinot/config:/config \
 --network analysing-log-files_default \
 apachepinot/pinot:0.11.0 \
 AddTable \
 -schemaFile /config/schema.json \
 -tableConfigFile /config/table.json \
 -controllerHost pinot-controller \
 -exec