talk-stream-processing-et-sql

Support pour un talk (format quickie) sur le stream processing avec SQL

Demos

For the demo, first start the log generator

mvn -Ploggen exec:java

streaming-sql

./streaming-sql

Query examples:

SELECT STREAM * FROM weblogs WHERE status = '500';

SELECT STREAM 
TUMBLE_END(rowtime, INTERVAL '10' SECOND),
url,
COUNT(*) AS nb_requests,
SUM(nb_bytes) AS total_bytes  
FROM weblogs 
GROUP BY TUMBLE(rowtime, INTERVAL '10' SECOND), url;

SELECT STREAM
HOP_END(rowtime, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS rowtime,
COUNT(*) AS nb_requests,
SUM(nb_bytes) AS total_bytes
FROM weblogs
GROUP BY HOP(rowtime, INTERVAL '5' SECOND, INTERVAL '10' SECOND);

sqlline

./sqlline
sqlline> !connect jdbc:calcite:model=target/classes/weblogs-model.json admin admin

Storm SQL

Start the env.

./zk-start.sh
./kafka-start.sh
./storm-start.sh

Submit the topology (sql file)

./submit-storm-sql.sh

Start the log generator (class io.millesabords.demo.streamingsql.producer.KafkaLogProducer

Check the logs pushed in the topic errorlogs

./consumer-error.sh