An OpenCV approach to CCTV
[OpenCV]
[Acknowledgements]
[Issues]
Blink records frames from a device like normal recording software, but only keeps those that change significantly from the frame in the previously recorded event to minimize space requirements.
It utilizes SSIM, Flat Difference, and PSNR to detect how different two images are and only records the frame if that difference exceeds a threshold set through the arguments.
Each time that happens, the program then identifies objects in the image and their location. All identified objects are labeled with a number, weight of identification, and a outline depicting the region of interest. For any human detections using the Single Shot Detector or the YOLO detector, it will try to predict if any two people are too close for social-distancing by estimating the relative distance of every detected person from all other people it found in the image.
Finally, it will alert the user as to the time of the event, event number, and then saves this image to a separate file labeled with that event number.
OpenCV (4.3.0-7)
pacman -S opencv (optional dependencies)Please see dependencies listed here.
Classification & Detection Files
To use the Haar-Cascade Classification functionality it is required to have a cascade file. In my own testing, I used the cascade file created by Rainer Lienhart that comes with opencv-samples from the AUR for front-facing human faces. These files are also available here, in the OpenCV repository.
Alternatively, to use the Single-Shot-Detector or YOLO detector functionality it is required to have the class labels as well as both the model and network configuration in whatever file format you choose. Please see the documentation for information on supported backends in OpenCV. A good place to look for these is the Model Zoo and the Tensor Flow repositories. Depending on what you use, you will need to set the scaling for your images and mean value used in the detection. In my own testing I've used MobileNetv2 for the SSD and YOLOv3-416 for YOLO, both of which have performed well in general application.
./driver -<OPTION>=<VALUE>./driver -<MULTI-VALUE-OPTION>=<VALUE>,<VALUE>,...e.g. Capture an image every 1 millisecond interval on devices 0 & 2 with a percent difference threshold of 15%
./driver -i=1000000 -t=15 -d=/dev/video0,/dev/video2| Option | Full Name | Description |
|---|---|---|
d |
device | Sets the device used to read frames from |
i |
interval | Sets the interval between capturing frames in units of nanoseconds |
m |
method | Defines which method of calculating image difference is used (0: FTPD -- 1: SSIM) |
f |
fthresh | Set the pixel-difference percentage threshold for FTPD |
t |
threshold | Specify the percent difference threshold to use for what qualifies as 'different images' |
n |
type | Set the detection method (-1: NONE -- 0: SSD -- 1: HCC) |
c |
classifier | Set the path to the file(s) used during classification |
p (req: SSD) |
humanLID | Specify the index of 'human' in class labels given for SSD |
a (req: HCC) |
neighbours | Set the minumum amount of neighbours for HCC detection |
s (req: HCC) |
scale | Set the image scaling to use for HCC detection |
b (req: HCC) |
blur | Specify whether to blur identified objects for HCC |
r (req: HCC) |
rotation | Specify whether to try rotation for HCC |
Provided in the repository is a performance testing and resource consumption testing utility, which monitors the amount of resources used by Blink during sixty seconds of runtime. To perform this testing, please first modify the values in the Makefile to reflect the values you would like to use for testing.
SSD_FILES = -c=<Configuration>,<Model>,<Labels>
YOLO_FILES = -c=<Configuration>,<Model>,<Labels>
HCC_FILES = -c=<Classifier>
DET_NONE = -n=-1
DET_SSD = -n=0 $(SSD_FILES)
DET_HCC = -n=1 $(HCC_FILES)
DET_YOLO = -n=2 $(YOLO_FILES)
DEVICE = ...
MISC = ...Finally, all that's left is run make perftest and check the resulting resource_usage.png file generated, which will show the graph of the resources used plotted against time.
'Blink' was originally meant as a educational resource for OpenCV that turned into my Senior Project. As such, it is heavily commented and there may be certain sections with verbose documentation.
A good portion of the comments for SSIM() & PSNR() refer to the formulas, most if not all of which can be found in the two Wikipedia pages below:
To learn how to calculate SSIM in a timely and efficient manner with the matrices in OpenCV as well as use the cascade classifiers provided, I referenced the pages below: