Repository navigation
Project Overview
Christopher Kok1, Seah Bryan1, Ang Jia Yu1, Jurgen Tan1, Muhamed Fauzi Bin Abbas1, Mahesh Raveendranatha Panicker1, Thiruneepan Selvakulasingam1
1Singapore Institute of Technology
In the dynamic environment of educational institutions, canteens play a crucial role in providing students with quick, convenient, and cost-effective meals. However, these canteens often experience fluctuating levels of traffic, especially during peak times such as lunch breaks. This variability in crowd size can lead to challenges in managing wait times and ensuring a smooth dining experience for students.
During peak periods, canteens may become overcrowded, resulting in extended wait times and difficulties in navigating the space. For students with tight schedules—those between classes or with short lunch breaks—this congestion can lead to significant inconvenience.
- Wasted Time: Students may spend an excessive portion of their break time waiting in line or maneuvering through crowded spaces, which reduces their overall relaxation and breaks.
- Increased Stress: The pressure to quickly find a place to eat amid crowding can elevate stress levels and diminish the quality of the lunch break.
- Health and Performance Impact: Students who struggle to find accessible canteens may resort to unhealthy fast food options or skip meals entirely. This not only affects their immediate health but can also impair academic performance and overall well-being.
To tackle these issues, we propose a multi-modal solution designed to enhance canteen crowd management. Our system integrates Wi-Fi signal strength, Bluetooth Low Energy (BLE) device counts, and real-time image analytics using low-cost Raspberry Pi devices. This approach aims to provide accurate, real-time estimates of crowd density and improve overall crowd management strategies.
Our solution involves the deployment of four Raspberry Pi devices, each serving as an edge computing node within the canteen. These devices are equipped with various sensors to collect data and provide real-time insights into crowd density.
-
Wi-Fi Signal Analysis
- Each Raspberry Pi captures Received Signal Strength Indications (RSSIs) from the top 5 strongest Wireless Access Points (APs) in the vicinity. The attenuation of 2.4 GHz Wi-Fi signals by human bodies can be used as a proxy for estimating crowd density. Increased crowd density results in higher attenuation and a noticeable reduction in signal strength [1].
-
Bluetooth Low Energy (BLE) Device Detection
- BLE devices, commonly found in smartphones and wearable technology, are detected by the Raspberry Pi devices. By counting the number of BLE devices, we estimate the number of individuals present. The number of BLE devices correlates closely with the crowd size.
-
Image Analytics
- Each Raspberry Pi is equipped with a webcam that captures real-time images of the canteen. These images are analyzed using the YOLOv8 object detection model, which counts the number of people present. YOLOv8 is trained to detect and count individuals, providing an additional data point for estimating crowd size.
Initially, data was collected from the Foodlge canteen at Nanyang Polytechnic, where the Raspberry Pi devices were deployed. However, midway through the data collection phase, the Foodlge canteen was closed for renovation.
Therefore, we had to pivot and collect data from another canteen. This change necessitated the retraining of our models and the validation of results using a diverse dataset from two different canteens.
Training data was collected over a period of slightly under 7 hours across 2 days at Nanyang Polytechnic's Foodlge canteen. Additionally, 3 hours of data were collected at a separate venue to serve as a validation set.
During both the data collection and deployment phases, a Python script was utilized to automate the data collection process. This script executed various programs:
- nmcli (Network Manager CLI): To collect Wi-Fi data, including RSSI values from the top 5 strongest APs.
- bluetoothctl: To gather BLE data, capturing the number of BLE devices detected.
- OpenCV: To capture images from the connected webcam, facilitating real-time image analytics for people counting.
Manual counts of people in the canteen at various times of the day provided ground truth data. This manually collected data was used to train and validate our models.
The data collected by the Raspberry Pi devices is transmitted to a fog computing device, which aggregates the information. A Gaussian Process Regression (GPR) model is then applied to this aggregated data to estimate the distribution of crowd density. The GPR model provides both mean and uncertainty estimates, offering a probabilistic approach to understanding crowd dynamics.
- Signal Attenuation: Human bodies cause attenuation of 2.4 GHz Wi-Fi signals, which can be leveraged to estimate crowd density.
- BLE Device Counts: The number of detected BLE devices scales with the number of people in the vicinity.
- Real-Time Image Processing: The YOLOv8 model processes images to count people, providing immediate data for crowd density analysis.
Preliminary experiments were conducted over two days to evaluate the performance of various regression models. We employed a 3-Fold Cross Validation approach to assess model robustness and generalizability. Due to the incomplete data pipeline at this stage, only Wireless AP RSSI values were available for model training.
| Model | Fold 1 | Fold 2 | Fold 3 |
|---|---|---|---|
| Linear Regression | 0.49 | 0.78 | 0.00 |
| Polynomial Regression | 0.45 | 0.71 | -0.15 |
| Support Vector Regression | 0.00 | 0.00 | -0.99 |
| Decision Tree Regression | -0.88 | 0.79 | -1.19 |
| Random Forest Regression | 0.12 | 0.54 | 0.00 |
| Kernel Ridge Regression | 0.48 | 0.84 | 0.01 |
| Linear Ridge Regression | 0.49 | 0.78 | 0.00 |
| Gaussian Process Regression | 0.48 | 0.84 | 0.01 |
| Lasso Linear Regression | 0.79 | -0.08 | -0.25 |
| Elastic Net Regression | 0.32 | -0.08 | -0.15 |
The cross-validation results revealed substantial variability in model performance across different folds. Models such as Support Vector Regression and Decision Tree Regression exhibited significant fluctuations, while Kernel Ridge Regression and Gaussian Process Regression showed more consistent results.
To further validate the models, additional data was collected from a different canteen. The models were retrained and evaluated using this new dataset.
| Model | Training Set | Validation Set |
|---|---|---|
| Linear Regression | 0.99 | -1.21 |
| Gaussian Process Regression | 0.95 | 0.72 |
| Kernel Ridge Regression | 0.99 | -1.87 |
| Lasso Linear Regression | 0.99 | -1.75 |
The Gaussian Process Regressor (GPR) consistently demonstrated strong performance on the validation set, despite having a lower 𝑅² value on the training set. This suggests that the GPR model, while less overfitted to the training data, provides robust and reliable estimates of crowd density on new data. The GPR model's ability to predict both mean and uncertainty allows for a comprehensive understanding of crowd dynamics, which can be visualized in web and mobile dashboards for improved decision-making.
Our approach to crowd management has shown promising results in different canteen environments. The model trained on one canteen demonstrated its effectiveness when validated with data from another canteen, suggesting potential scalability to various crowded venues such as gyms, concert halls, and public events.
- For Operators: Enables efficient resource allocation and manpower management through predictive analytics.
- For Diners: Provides real-time occupancy information, enhancing the dining experience and facilitating better crowd dispersion.
Several areas offer opportunities for further development:
- Higher Precision Sensors: Incorporating advanced sensors and dedicated Wi-Fi antenna peripherals could enhance prediction accuracy and reliability.
- Exploration of Additional Frequency Bands: Investigating other radio frequency ranges (e.g., 433 MHz, 868 MHz) could provide additional data sources for crowd estimation [2].
- Increased Density of Edge Devices: Deploying a higher density of edge devices could enable more granular crowd density estimation, offering deeper insights into venue dynamics.
[1] H. Hongwei, S. Wei, X. Youzhi, and Z. Hongke, "The effect of human activities on 2.4 GHz radio propagation at home environment," in 2009 2nd IEEE International Conference on Broadband Network & Multimedia Technology, Beijing, China: IEEE, Oct. 2009, pp. 95–99. doi: 10.1109/ICBNMT.2009.5347820. Available: IEEE Xplore. [Accessed: Mar. 11, 2024]
[2] A. Kaya, S. Denis, B. Bellekens, M. Weyn, and R. Berkvens, "Large-Scale Dataset for Radio Frequency-Based Device-Free Crowd Estimation," Data, vol. 5, no. 2, p. 52, Jun. 2020. doi: 10.3390/data5020052.