The goal of this project is to implement a Video Surveillance System for Traffic monitoring, that allows to monitor the current state of the roads, being this the traffic density, mean speed and some other features of interest.
The first block of the system consists in a Background Subtraction algorithm based on adaptive Gaussian model defined in RGB space. On this step, we separate the vehicles from the environment.
The second block is a Foreground Improvement algorithm. The mask obtained in the previous step is improved with a morphology pipeline composed by a hole-filling step, area filtering, an opening and finally a closing.
The third block is a Vehicle Tracker based on Kalman Filter. The previous steps are executed for each frame independently; with this step, we obtain the path that follows each vehicle along the different frames.
Finally, the fourth block performs different Road Statistics. It estimates the speed of each detected vehicle and checks if it is below the speed limit of the road and some statistics per lane. Specifically it computes the total number of vehicles on each lane, the density of vehicles and the average speed.
Check the README file in the run folder for further details on how to prepare the running environment and how to
run the code.
- Segmentation metrics. Understand precision & recall.
- Implement Precision and Recall
- Implement F1-Score
- Compute these metrics for sequences A & B
- Provide an interpretation of the different values obtained for Test A and Test B.
- Temporal Analysis of the Results
- Graph 1: True Positive & Total Foreground pixels vs #Frames.
- Graph 2: F1-Score vs Time.
- Provide an interpretation of the graphs obtained for Test A and Test B.
- Quantitative evaluation of Optical Flow
- Compute Mean Squared Error in Non-occluded areas (MSEN)
- Compute Percentage of Erroneous Pixels in Non-occluded areas (PEPN)
- Analysis and visualization of errors
- Desynchronization of the sequence
- Force de-synchronized results for background subtraction in Highway sequence
- Study and comment the results
- Optical Flow visualization
- Propose a simplification method for a clean visualization
- Gaussian Modelling
- Create a Gaussian function to model each background pixel using the first 50% of the video sequence
- Segment the foreground pixels for the second 50% of the video sequence
- Evaluate the segmentation: Compute F1-score vs alpha curve
- Evaluate the segmentation: Precision-Recall vs alpha curve, Area Under the Curve
- Adaptive Modelling
- Segment the foreground pixels for the second 50% of the video sequence and left background adapt
- Optimize alpha and rho with grid search to maximize F1-score
- Compare Gaussian Modelling vs. Adaptive Modelling for all three sequences using F1-score and Area Under the Curve
- Compare with state-of-the-art implementations
- Run different state-of-the-art Background Subtraction methods: e.g.: OpenCV BackgroundSubtractorMOG, BackgroundSubtractorMOG2, BackgroundSubtractorLSBP
- Evaluate precision vs recall to comment which method (single Gaussian programmed by you or state-of-the-art) performs better
- Evaluate the sequences than benefit more one algorithm and try to explain why
- Color Sequences
- Update the implementation to support color sequences: Decide which colorspace to use, number of Gaussians per pixel, etc.
- Hole filling
- Choose the best configuration from week 2
- Post-process with hole filling (indicate used library and link to code)
- Experiment with 4 and 8 connectivity
- Compute AUC and gain for each video sequence
- Provide qualitative interpretation
- Area filtering
- AUC vs number of pixels
- arg max P (AUC) (indicate used library and link to code)
- Additional morphological operations
- Explore with other morphological filters to improve AUC for foreground pixels (indicate used library and link to code)
- Shadow removal
- Implement some shadow removal existing techniques
- Improvements of this week
- Compare the precision / recall curves
- Update the AUC and compute the gain
- Optical Flow
- Optical Flow with Block Matching
- Block Matching vs Other Techniques:
- Try OpenCV Farneback implementation
- Try FlowNet 2.0 implementation
- Video Stabilization
- Video Stabilization wih Block Matching
- Block Matching Stabilization vs Other Techniques
- Stabilize your own video
- Vehicle tracker
- Vehicle tracker with Kalman filter
- Vehicle tracker with other tools
- Speed estimator
- Estimate the speed of the vehicles
- Road statistics
- Check the speed limit
- Check the road density of vehicles
- Compute the average speed per lane