OpenVINO™ Toolkit - DL Streamer repository
This repository is a home to Deep Learning (DL) Streamer. DL Streamer is a streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines. It ensures pipeline interoperability and provides optimized media, and inference operations using Intel® Distribution of OpenVINO™ Toolkit Inference Engine backend, across Intel® architecture - CPU, iGPU and Intel® Movidius™ VPU. DL Streamer prebuilt binaries can be installed with the Intel® Distribution of OpenVINO™ Toolkit installer.
Here's the canonical video analytics pipeline consturctued using DL Streamer. It performs detection and classification operations on a video stream, using face detection and emotion classification deep learning models. The results of this pipeline are demoed in the above video clip:
gst-launch-1.0 filesrc location=cut.mp4 ! decodebin ! videoconvert ! gvadetect model=face-detection-adas-0001.xml ! gvaclassify model=emotions-recognition-retail-0003.xml model-proc=emotions-recognition-retail-0003.json ! gvawatermark ! xvimagesink sync=false
The complete solution leverages:
- Open source GStreamer* framework for pipeline management
- GStreamer* plugins for input and output such as media files and real-time streaming from camera or network
- Video decode and encode plugins, either CPU optimized plugins or GPU-accelerated plugins based on VAAPI
- Deep Learning models converted from training frameworks TensorFlow*, Caffe* etc. from Open Model Zoo And, the following elements in the DL Streamer repository:
In addition, the solution uses the following Deep Learning-specific elements, also available in this repository:
- Inference plugins leveraging OpenVINO™ Toolkit for high-performance inference using deep learning models
- Visualization of the inference results, with bounding boxes and labels of detected objects, on top of video stream
Please refer to Elements for the complete DL Streamer elements list.
GStreamer is an open source framework licensed under LGPL. See license terms. You are solely responsible for determining if your use of Gstreamer requires any additional licenses. Intel is not responsible for obtaining any such licenses, nor liable for any licensing fees due, in connection with your use of Gstreamer
- Refer to OpenVINO™ Toolkit Hardware sections - Intel® Processors, Intel® Processor Graphics and Intel® Movidius™ Vision Processing Unit (VPU)
- On platforms with Intel Gen graphics, see the gstreamer-vaapi for hardware accelerated video decode and encode requirements
- Intel® Distribution of OpenVINO™ Toolkit Release 2021.4
- GStreamer* framework 1.16.2 or above (recommended version is 1.18.4)
- Operating system: Ubuntu 18.04, Ubuntu 20.04, CentOS 7
- Start here: Install Guide
- DL Streamer Tutorial
- Introduction to DL Streamer: Ready, Steady, Stream: Introducing Intel® Distribution of OpenVINO™ toolkit Deep Learning Streamer
- Audio event detection synchronized with video based object detection using DL Streamer: AI Beyond Computer Vision with the Intel® Distribution of OpenVINO™ toolkit
- YouTube Videos:
For additional documentation, please see wiki and don't miss the documentation indexed on the right side of the wiki home page.
Develop in the Cloud
Try DL Streamer with Intel® DevCloud. You can build your pipeline, test and optimize for free. With an Intel® DevCloud account, you get 120 days of access to the latest Intel® hardware — CPUs, GPUs, FPGAs. No software downloads. No configuration steps. No installations. Check out DL Stramer Tutorial on Intel® DevCloud.
Other Useful Links
Video Analytics Serving: Video Analytics Serving is a python package and microservice for deploying optimized media analytics pipelines. It supports pipelines defined in DL Streamer and provides APIs to discover, start, stop, customize and monitor pipeline execution.
The reference media analytics applications, provided by Open Visual Cloud, that leverage DL Streamer elements:
Intel® Edge Software Hub packages that include DL Streamer:
Reporting Bugs and Feature Requests
Report bugs and requests on the issues page
How to contribute
Pull requests aren't monitored, so if you have bug fix or an idea to improve this project, post a description on the issues page.
* Other names and brands may be claimed as the property of others.