Distributed application project for the couse of Distributed Systems at AUEB.
Objective of this project was to implement a video streaming/ upload service based on the publisher/ subscriber model.
Users can search for topics (channels, hashtags), subscribe to topics, stream videos of other users and upload/ delete their own videos by using an android application.
Github repository link here.
A small dataset of videos for the users, download on the emulators or the actual devices, here.
Synchronization node. Zookeeper is responsible for storing, updating and distributing the InfoTable to the Brokers.
The Zookeeper is kept as simple as possible for this project, whereas it could be as complicated as in the Apache Kafka platform.
Intermediate node. Each Broker node is responsible for a group of topics, which are assigned using the SHA-1 hashing.
Broker Nodes serve AppNode "client" requests for upload, deletion or streaming (download) of videos.
Brokers, also, request videos from AppNode "publishers" (pull) to satisfy the AppNode "client" request of video steaming from other AppNodes.
In cases of upload/ deletion requests, after handling it, the Brokers have to issue themselves a request to the Zookeeper for the InfoTable to be updated.
Brokers also issue an InfoTable retrieval request each time an AppNode connects to them or requests anything from them, so that they all always have the most updated
version.
Publisher & Subscriber node. The AppNode technically represents a user of the system.
An AppNode node can be both a Publisher (server side, provide videos to clients) and a Subscriber (client side, ask for videos from publishers)
or just Subscribers if the user hasn't published any videos on the platform.
AppNodes as Publishers maintain a server and handle requests from Brokers when a user (another AppNode) requests one of their videos,
in such a case they chunk the video data and send (push) them chunk by chunk to the Broker, which will then send them to the client AppNode.
AppNodes as Subscribers just issue requests to the Brokers for video streaming (download), by searching or subscribing to a topic.
Other requests of AppNodes issued to Brokers are the ones where the user uploads/ deletes a video of theirs, which means they have to update the data about them on the platform.
Infotable is a vital compoment of the system which contains all of the important data for the application to work properly. The Zookeeper is responsible for keeping
this component updated after every update request it receives from the Brokers. The component is distributed to the Brokers each time it's needed, via the Zookeeper.
The important data we mentioned could be:
- The Broker IDs, as they were assigned from the hashing.
- The topics that are assigned to each Broker.
- The available AppNodes that maintain a server side (are Publishers) and the topics for which they have videos published.
- The available videos on the system categorized by topic.
- Any Java distribution over 6.
- Android Studio (for emulator use and download on Android Devices) and/or any Android Device.
- Devices must have minimum SDK version of 26.
- Clone the project to Android Studio.
- Change the IP addresses in the Node.java file, you may add:
- The same IP for all addresses but different ports for each one.
- Different IPs for each address (ports can be the same or different).
- Assign one IP and different ports to Zookeeper and all Brokers, and every similar combination.
- Download the apk of the Android App on your Android devices/ emulators by running the app on multiple devices.
- Make configurations for the mains and run them in this order:
- Zookeeper main configuration.
- Broker1 main configuration.
- Broker2 main configuration.
- Broker3 main configuration.
- While the Zookeeper/ Brokers are running enter a username.
- You're all set!
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.