Skip to content
Post processing video magic with ZoneMinder: find missing objects, blend multiple events, annotate videos, and more
Python
Branch: master
Clone or download
Latest commit a814987 Aug 29, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
sample annotate video Jun 15, 2019
zmMagik_helpers fix label #5 Aug 29, 2019
.gitignore initial GPU integration Jun 26, 2019
LICENSE Initial commit Jun 6, 2019
README.md mocord notes for 1.34 Jun 28, 2019
config.ini.sample add minimum alarmed frame option Jun 30, 2019
magik.py fix label #5 Aug 29, 2019
requirements.txt align annotate with blend, using new gpu code Jun 27, 2019

README.md

What

zmMagik will be a list of growing foo-magic things you can do with video images that ZM stores. Probably...

Features

As of today, it lets you:

  • Blend multiple events to quickly see how the day went. Imagine compressing 24 hours of video into 1 minute with object overlays. Gadzooks!

this video is blended from 2 days worth of video. Generated using python ./magik.py -c config.ini --monitors 11 --blend --display --download=False --from "2 days ago"

  • Annotate recorded ZoneMinder videos. Holy Batman!

generated using python ./magik.py -c config.ini --eventid 44063 --dumpjson --annotate --display --download=False --onlyrelevant=False --skipframes=1

  • Find an image fragment inside multiple events. For example, someone stole your amazon package. Crop a picture of an event with that package and then ask zmMagik to search for events where this package went missing. Great Krypton!

generated using python ./magik.py -c config.ini --find trash.jpg --dumpjson --display --download=False --from "8am" --to "3pm" --monitors 11

Why

  • I came home one day to see my trash can cover went missing. I thought it would be fun to write a tool that could search through my events to let me know when it went missing. Yep, it started with trash talking

  • Andy posted an example of how other vendors blend multiple videos to give a common view quickly. I thought it would be fun to try

  • One thing leads to another and I keep doing new things to learn new things..

Limitations

  • Only works with video mp4 files. Did not bother adding support for JPEG store
  • Very Beta. Also, if you don't have a GPU, make sure you play with the flags to optimize skipframes, detection mode, resize
  • Multi-server most likely won't work

Installation

# needs python3, so you may need to use pip3 if you have 2.x as well
git clone https://github.com/pliablepixels/zmMagik
cd zmMagik
# you may need to do sudo -H pip instead for below, if you get permission errors
pip install -r requirements.txt

If you are using yolo extraction, you also need these files and make sure your config variables point to them

wget https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg
wget https://pjreddie.com/media/files/yolov3.weights
wget https://raw.githubusercontent.com/pjreddie/darknet/master/data/coco.names

Examples

General note: do a python ./zmMagik -h to see all options. Remember, you can stuff in regularly used options in a config file and override on CLI as you need. Much easier.

  • Create a blended event for monitor 11 in specified time range. config.ini has your ZM usename/password/portal etc so you don't need to type it in every time
python ./magik.py --monitors=11 --from "yesterday, 7am" --to "today, 10am" --blend -c config.ini
  • Search when an object (image) went missing:
python ./magik.py --monitors=7 --present=False --from "today, 7am" --to "today, 7pm" --find "amazonpackage.jpg" -c config.ini

Note that amazonpackage.jpg needs to be the same dimensions/orientation as in the events it will look into. Best way to do this is to load up ZM, find the package, crop it and use it.

FAQ

  • How do I use GPU acceleration?

    • See GPU section below
  • What is "mixed" background extraction?

    • This is the default mode. It uses the very fast openCV background subtraction to detect motion, and then uses YOLO to refine the search to see if it really is an object worth marking. Use this mode by default, unless you need more speed, in which case, use "backround_extraction"
  • Using "background_extraction" mode isn't that great

    • Yes, that's why you should use "mixed"
    • Some tips:
      • Use masks to restrict area
      • Use --display to see what is going on, look at frame masks and output
      • Try changing the learning rate of the background extractor
      • See if using a different Background extractor for fgbg in globals.py helps you (read this)
      • Fiddle with kernel_clean and kernel_fill in globals.py
  • Using "Yolo" or "mixed" extraction mode is great, but it overlays complete rectangles

    • Yes, unlike "background_extraction" yolo doesn't report a mask of the object shape, only a bounding box
    • I'll add masked R-CNN too, you can try that (will be slower than Yolo)
    • Maybe you can suggest a smarter way to overlay the rectangle using some fancy operators that will act like its blending?
  • find doesn't find my image

    • Congratulations, maybe no one stole your amazon package
    • Make sure image you are looking for is not rotated/resized/etc. needs to be original dimensions

GPU FAQ

As of today, OpenCV DNN doesn't support GPU acceleration for NVIDA (no idea about non NVIDIA). This may change soon with a work in progress contribution to get it working with CUDA. Till that happens, we need to directly use a GPU enabled version for YoloV3. I went with the darknet fork maintained by AlexyAB

Simply put:

  • Compile it with GPU

  • Make sure it is actually using GPU

  • then set gpu=True and darknet_lib=<path/to/filename of gpu accelerated so>

  • If you need help compiling darknet for GPU and CUDA 10.x, see simpleYolo

  • Do NOT use darknet lib directly for a CPU compiled library. It is terribly slow (in my tests, OpenCV was around 50x faster)

  • How much GPU memory do I need?

    • The YoloV3 model config I use takes up 1.6GB of GPU memory
    • Note that I use a reduced footprint yolo config. I have 4GB of GPU memory, so the default yolov3.cfg did not work and ate up all my memory. This is my modified yolov3.cfg section to make it work:
[net]
batch=1
subdivisions=1
width=416
height=416
<and then all the stuff that follows>
  • How much speed boost can I expect with GPU?

    • Here is a practical comparison. I ran a blend operation on my driveway camera (modect) for a full day's worth of alarmed events. I used 'mixed' mode, which first used openCV background subtraction and then YOLO if the first mode found anything. This was to be fair to the CPU stats when compared. It grabbed a total of 27 video events:
    python ./magik.py --blend --from "1 day ago"  --monitors 8 -c ./config.ini --gpu=True --alarmonly=True --skipframes=1
    
    Total time: 250.72s
    

    I then ran it without GPU: (Note that I have libopenblas-dev liblapack-dev libblas-dev configured with OpenCV to improve CPU performance a lot)

    python ./magik.py --blend --from "1 day ago"  --monitors 8 -c ./config.ini --gpu=False --alarmonly=True --skipframes=1
    
    Total time: 1234.77s
    

    Thats a 5x improvement

    • On my 1050 Ti, YoloV3 inferences drops to 120ms or less, compared to 2-3 seconds on GPU
    • That being said, blending/annotating involves:
      • reading frames (A)
      • processing frames (B)
      • writing frames (C)
    • GPU affects point B. If you are reading very large events, A & C will still take its own time. You likely won't see a big improvement there. If there are many objects (B), then obviously, GPU performance improvements will have a huge impact. To make A & C faster:
      • use resize
      • use skipframes
      • use alarmonly=True
      • If you use mocord in ZM, then starting ZM 1.34, set EVENT_CLOSE_MODE to "alarm". That will create a new event when an alarm occurs, and close it when the alarm closes. That will help you speed things up a lot
      • All that being said, I'm using a threaded opencv pipeline to read frames which does improve read performance compared to before (credit to imutils)
You can’t perform that action at this time.