2022.2 Pre-release 2
Pre-release
Intel® Deep Learning Streamer Pipeline Framework 2022.2 Pre-release 2
Intel® Deep Learning Streamer (Intel® DL Streamer) Pipeline Framework is a streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines. It ensures pipeline interoperability and provides optimized media, and inference operations using Intel® Distribution of OpenVINO™ Toolkit Inference Engine backend, across Intel® architecture, CPU, and iGPU.
This release includes Intel® DL Streamer Pipeline Framework elements to enable video and audio analytics capabilities, (e.g., object detection, classification, audio event detection), and other elements to build end-to-end optimized pipeline in GStreamer* framework.
The complete solution leverages:
- Open source GStreamer* framework for pipeline management
- GStreamer* plugins for input and output such as media files and real-time streaming from camera or network
- Video decode and encode plugins, either CPU optimized plugins or GPU-accelerated plugins based on VAAPI
- Deep Learning models converted from training frameworks TensorFlow*, Caffe* etc. from Open Model Zoo (OMZ)
- The following elements in the Pipeline Framework repository:
| Element | Description |
|---|---|
| gvadetect | Performs object detection on a full-frame or region of interest (ROI) using object detection models such as YOLOv4, MobileNet SSD, Faster-RCNN etc. Outputs the ROI for detected objects. |
| gvaclassify | Performs object classification. Accepts the ROI as an input and outputs classification results with the ROI metadata. |
| gvainference | Runs deep learning inference on a full-frame or ROI using any model with an RGB or BGR input. |
| gvaaudiodetect | Performs audio event detection using AclNet model. |
| gvatrack | Performs object tracking using zero-term, or imageless tracking algorithms. Assigns unique object IDs to the tracked objects. |
| gvametaaggregate | Aggregates inference results from multiple pipeline branches |
| gvametaconvert | Converts the metadata structure to the JSON format. |
| gvametapublish | Publishes the JSON metadata to MQTT or Kafka message brokers or files. |
| gvapython | Provides a callback to execute user-defined Python functions on every frame. Can be used for metadata conversion, inference post-processing, and other tasks. |
| gvawatermark | Overlays the metadata on the video frame to visualize the inference results. |
| gvafpscounter | Measures frames per second across multiple streams in a single process |
| gvaactionrecognitionbin | Performs full-frame action recognition inference using action-recognition-0001’s/driver-action-recognition-adas-0002’s encoder and decoder models. |
For the details of supported platforms, please refer to System Requirements section.
For installing Pipeline Framework with the prebuilt binaries or Docker* or to build the binaries from the open source, please refer to Intel® DL Streamer Pipeline Framework installation guide
New in this Release
| Title | High-level description |
|---|---|
| Intel® Data Center GPU Flex Series Beta support | Validated on Intel® Data Center GPU Flex Series 140 and 170 with pipelines/models/videos from the Intel® DL Streamer Pipeline Zoo, Pipeline Zoo Models and Pipeline Zoo Media repositories |
| New element object_track supporting object tracking on GPU | device=GPU in gvatrack device=GPU discontinued, instead use vaapipostproc ! object_track spatial-feature=sliced-histogram device=GPU ! vaapipostproc |
| New element watermark_sycl supporting inference results visualization on GPU | device=GPU in gvawatermark device=GPU discontinued, instead use vaapipostproc ! watermark_opencv attach-label-mask=true ! watermark_sycl ! vaapipostproc |
New property labels-file in gvainference/gvadetect/gvaclassify elements |
Allows to pass labels as .txt file |
New property scale-method in gvainference/gvadetect/gvaclassify elements |
Allows to select scale method used in pre-processing |
| Compatibility with OpenVINO™ Toolkit 2022.2 Pre-release | Pipeline Framework has been updated to use the 2022.2.0.dev20220829 version of the OpenVINO™ Toolkit |
| YOLOv5 Support | Added YOLOv5 postprocessing support |
Changed in this Release
Deprecation Notices
- Deprecated device=GPU in gvatrack and gvawatermark
Known Issues
| Issue | Issue Description |
|---|---|
| DMABuf memory not working | Media driver fails processing DMABuf memory. Workaround is to use VASurface memory, replace GStreamer caps video/x-raw(memory:DMABuf) with video/x-raw(memory:VASurface) |
| Artifacts on watermark_sycl | Running inference results visualization on GPU via watermark_sycl may produce some partially drawn bounding boxes |
| gvawatermark fails if DPC++ environment initialized | Workaround is to add property device=CPU gvawatermark device=CPU or not initialize DPC++ environment if using gvawatermark for visualization of inference results on CPU |
Fixed issues
| Issue # | Issue Description | Workaround | Affected platforms |
|---|---|---|---|
| Backwards compatibility issues when using gvapython and GST 1.18. | Added support to preserve legacy compatibility with gvapython element. | All |
System Requirements
Please refer to Intel® DL Streamer documentation.
Installation Notes
There are several installation options for Pipeline Framework:
- Build Pipeline Framework from source code
For more detailed instructions please refer to Intel® DL Streamer Pipeline Framework installation guide.
Samples
The samples folder in Intel® DL Streamer Pipeline Framework repository contains command line, C++ and Python examples.
Legal Information
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
The products and services described may contain defects or errors which may cause deviations from published specifications. Current characterized errata are available on request.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© 2022 Intel Corporation.