Gas_Station_Demo

SpringBoot+Spark Streaming+Flume+Kafka+Redis+MySql Big Data--Gas Station Management System

👏👏🎉🎉🎉🎉👏👏

Base project made with much ❤️. Contains CRUD, patterns, generated library, and much more!

Table of contents

• What's included
• Bugs and feature requests
• Creators
• Thanks
• Copyright and license

What's included

• Developed data analysis system with Spark, Kafka, ZooKeeper, MySQL, InnoDB, Flume, SpringBoot, and Echarts
• Integrated data distributed on 30 servers to Kafka in JSON using Flume and implemented data ingestion and handling clusters in real-time processing using Kafka.
• -Used ZooKeeper to co-ordinate cluster services.
• Consume live data from Kafka topics with Spark Streaming to create continuous sequences of RDDs.
• -Generated distributed computing Map and Reduce for RDD stream to improve writing speed to MySQL, and implement transaction features with InnoDB.
• -Transfer offsets generated by Spark Streaming to MySQL and Implemented visualization with SpringBoot and Echarts.

MySQL

This repo is using MySQL. We use MySQL to handle CRUD operations over the Kafka offset, and to store station data.

Bugs and feature requests

Have a bug or a feature request? Please first read the issue guidelines and search for existing and closed issues. If your problem or idea is not addressed yet, please open a new issue.

Creators

Azanda Lyu

• https://github.com/Casperoazanda

Thanks

Thanks to all contributors and their support.

If you have an idea or you want to do something, tell me or just do it! I'm always happy to hear your feedback!

Copyright and license

Code and documentation copyright 2020 the authors. Code released under the MIT License.