Multi-threaded Sensor Aggregation System
A Python application simulating a distributed sensor data aggregation platform. This project focuses on implementing concurrency and ensuring thread safety to efficiently handle and process sensor readings from multiple sources across several aggregator nodes.
To design and implement a concurrent system that distributes sensor readings, manages the processing capacity of aggregators, and prevents data corruption using threading mechanisms.
- Concurrent Reading Collection: Uses Python's
threading
module to simulate the collection of sensor readings concurrently. - Capacity-Aware Distribution: Distributes readings among multiple aggregators, each with a defined processing capacity limit.
- Thread Safety with Locks: Implements
threading.Lock
to protect shared data structures (like task queues and aggregator state) and prevent race conditions. - Safe Reassignment: Logic to safely reassign sensor readings to other available aggregators if an aggregator reaches its capacity.
- Threading: Utilizing
threading
to achieve concurrent execution and task handling. - Concurrency Control: Implementing locks (
threading.Lock
) for mutual exclusion on shared resources. - Load Balancing: Strategies for distributing tasks (readings) among workers (aggregators) based on their capacity.
- Optimisation: Discussion on techniques like task batching, thread pooling and load balancing strategies for handling large scales.
- Ensure you have Python installed.
- Execute the main program file (e.g.,
sensor_aggregator.py
).python sensor_aggregator.py
- [cite_start]The simulation will run with 50 sensor readings and 5 aggregators as required by the simulation scenario[cite: 81].