2022.2.0

Intel® Deep Learning Streamer Pipeline Server Release v1.0

Intel® Deep Learning Streamer (Intel® DL Streamer) Pipeline Server is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Intel® DL Streamer Pipeline Server is based on Intel® DL Streamer Pipeline Framework and FFmpeg Video Analytics.

What's Changed

Title	Description
Kubernetes Sample	Microk8s based sample has been replaced by a "vanilla" Kubernetes equivalent with enhanced features
Action Recognition	This preview pipeline has been temporarily removed while we migrate to Pipeline Framework Architecture 2.0
Improved Security	By default source and destination data from pipeline requests is removed from metadata and pipeline status. It can be optionally included via server configuration.

What's New

Title	Description
OpenVINO™ Toolkit 2022.2 support	Now using intel/dlstreamer:2022.2.0-ubuntu20 as base image which includes the latest version of OpenVINO™ Toolkit.
Improved Kubernetes sample	Significant refactor of sample to provide the following new features: Helm install and configuration via values.yaml RTSP and WebRTC re-streaming from cluster Access to shared model, pipeline and extension folders through Persistent Volumes REST API support for listing all pipeline instances in cluster Optional https security via HAProxy Controller and sidecar docker images to manage the Pipeline Server routing and deployments for multiple Pipeline Server deployment GPU support for Kubernetes via Intel GPU Plugin
Add message field to REST pipeline status	Status now contains details of pipeline error
Improve REST API security	Added sample for securing server with Nginx and improve validation of requests
Intel® Data Center GPU Flex Series (preview)	Added a sample previewing use of this GPU for AI visual inference

What's Fixed

Description	Issue
Pipeline failure in some multi-GPU systems	#98
Intermittent 30s delay in pipeline start during multi-stream sessions	#104
Kubernetes deployment fails if no_proxy contains *	#105
Server crashes after several minutes when using RTSP camera and GPU inference	#111
Parameter substitution in pipeline template crashes the server	#117

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38
Models can be picked up from previous build	#71
Difficult to get normalized coordinates for spatial analytics parameters	#87
Yolo-v3-tiny-tf model with INT-8 precision gives bad inferences	#116

Tested Base Images

Supported base images are listed in the Building Intel® DL Streamer Pipeline Server document.

* Other names and brands may be claimed as the property of others.

v0.7.2-beta

Intel® Deep Learning Streamer Pipeline Server Release v0.7.2

Intel® Deep Learning Streamer (Intel® DL Streamer) Pipeline Server, formerly known as Video Analytics Serving, is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Intel® DL Streamer Pipeline Server is based on Intel® DL Streamer Pipeline Framework and FFmpeg Video Analytics.

What's Changed

Title	Description
Python package and module names	Package name changes vaserving -> server vaclient -> client Module name changes vaserving.py -> pipeline_server.py vaclient.py -> pipeline_client.py Applications that use pipeline server Python modules directly must be updated to use new names.

What's New

Title	Description
OpenVINO 2022.1 support	Now using intel/dlstreamer:2022.1.0-ubuntu20 as base image.
Deployment time pipeline configuration	Pipeline parameter default value can be set by environment variable using syntax: "default": "{env[DETECTION_DEVICE]}" This is particularly useful with Kubernetes deployments or with Docker Compose.
GPU support for Kubernetes	By using deployment time pipeline configuration the Kubernetes sample now automatically runs inference on GPU if accelerator is available.
WebRTC support	Added WebRTC as a frame destination.
Extended inference device support	Added support for HETERO, MULTI and AUTO devices
More reference models and pipelines	Added person and vehicle specific pipelines and models for improved accuracy

What's Fixed

Description	Issue
Some public models from Open Model Zoo do not produce inference results	#89
When interrupting run of multiple streams pipeline client prints fps of last stream not average	#106
GPU inference fails on 12th Gen Intel® Core™ systems	#108
Memory leak on pipeline stop	#112

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38
Models can be picked up from previous build	#71
Difficult to get normalized coordinates for spatial analytics parameters	#87
Pipeline failure in some multi-GPU systems	#98
Intermittent 30s delay in pipeline start during multi-stream sessions	#104
Kubernetes deployment fails if no_proxy contains *	#105
Client is incompatible with older versions of the service	#107
Server crashes after several minutes when using RTSP camera and GPU inference	#111

Tested Base Images

Supported base images are listed in the Building Intel® DL Streamer Pipeline Server document.

* Other names and brands may be claimed as the property of others.

v0.7.1-beta

Intel® Deep Learning Streamer Pipeline Server Release v0.7.1

Intel® Deep Learning Streamer (Intel® DL Streamer) Pipeline Server, formerly known as Video Analytics Serving, is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Intel® DL Streamer Pipeline Server is based on Intel® DL Streamer and FFmpeg Video Analytics.

What's Changed

Title	Description
Product name change	Video Analytics Serving is now called Intel® Deep Learning Streamer Pipeline Server as it is part of the Intel® DL Streamer product suite.
Breaking API change: Pipeline instances are now uuid strings	Pipeline instances created by different services can now be uniquely identified. Applications that depended on pipeline instances being integer values must be updated to handle strings.

What's New

Title	Description
Kubernetes Load Balancing Sample	Show how to use MicroK8s with the HAProxy load balancer to distribute work across pods in a cluster
REST API endpoint to list all pipeline instances	Endpoint GET /pipelines/status returns all pipeline instances as an array of status objects.
REST API status and stop endpoints no longer require pipeline name and version	The following endpoints have been added. GET /pipelines/{instance_id} Get {instance_id} summary DELETE /pipelines/{instance_id} Stop {instance_id} GET /pipelines/status/{instance_id} Get {instance_id} status The new endpoints have equivalent functionality to the following which are deprecated. GET /pipelines/{name}/{version}/{instance_id} Get summary for {instance_id} DELETE /pipelines/{name}/{version}/{instance_id} Stop {instance_id} GET /pipelines/{name}/{version}/{instance_id}/status Get {instance} status
VA Client enhancements	The following features have been added to support the Kubernetes sample. Use remote service Start multiple streams to help measure stream density Display results from MQTT and Kafka metadata destinations

What's Fixed

Description	Issue
Prevent pipeline instances from resetting	#58
REST API for status and stop ignores pipeline name and version	#92
EdgeX sample fails when run from behind a proxy	#97
REST service fails to start due to soft_unicode import error	#101

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38
Models can be picked up from previous build	#71
Difficult to get normalized coordinates for spatial analytics parameters	#87
Some public models from Open Model Zoo do not produce inference results	#89
Pipeline failure in some multi-GPU systems	#98
Intermittent 30s delay in pipeline start during multi-stream sessions	#104
Kubernetes deployment fails if no_proxy contains *	#105
VA Client reports incorrect average fps across multiple streams	#106

Tested Base Images

Supported base images are listed in the Building Intel(R) DL Streamer Pipeline Server document.

* Other names and brands may be claimed as the property of others.

v0.7.0-beta

Video Analytics Serving (VA Serving) is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.7.0-beta

Title	Description
Standalone microservice available in docker hub	A docker image of the VA Serving REST service is available at intel/video-analytics-serving. The ready to use image contains the following reference pipelines and can also be used as the basis for derivative microservices. object_detection Detect and label objects object_classification Add meta-data to object detections such as subtype and color object_tracking As object_classification adding tracking identifier to metadata audio_detection Analyze audio streams for events such as breaking glass or barking dogs. action_recognition Classifies general purpose actions in input video such as tying a bow tie or shaking
Expanded source customization based on request (including transparent support for webcam)	Previously reference pipelines required changes to work with different types of camera sources. Now pipelines can be reused without modification with the proper source derived from the request thus supporting a wider range of cameras including webcams and 'GigE' industrial cameras.
Edge AI Extension	Added HTTP protocol support Improved gRPC performance by up to 2x over v0.6.1 by threaded input handling It has been moved to independent repository https://github.com/dlstreamer/edge-ai-extension
HDDL-R accelerator support for Ubuntu 20.04 container	HDDL-R requires additional dependencies no longer in the the OpenVINO base image – these have been added back.
OpenVINO 2021.4.2 support	Updated DL Streamer base image to openvino/ubuntu20_data_runtime:2021.4.2.
VA Client improvements	Numeric parameters are now supported. Results are now displayed from slow starting pipelines.

Issues Resolved by This Release

Description	Issue
RTSP re-streaming plays back at frame processing rate, not encoded rate.	#68
Docker build fails if no_proxy setting contains spaces	#88

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38
Models can be picked up from previous build	#71
Difficult to get normalized coordinates for spatial analytics parameters	#87
Some public models from Open Model Zoo do not produce inference results	#89
REST API for status and stop ignores pipeline name and version	#92
EdgeX sample fails when run from behind a proxy	#97
Pipeline failure in some multi-GPU systems	#98

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

v0.6.1-beta

Video Analytics Serving (VA Serving) is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.6.1-beta

Title	Description
Documentation updates	Added more examples to extensions developer guide and customizing pipeline requests. Removed LVA references from Edge AI Extension Module README
OpenVINO 2021.4.1 support	Updated DL Streamer base image to OpenVINO 2021.4.1

Issues Resolved by This Release

Description	Issue
Tracking pipeline can drop results if tracking-type parameter is set to “short-term”	#72
Audio inference fails on some platforms	#79
VA Client outputs blank lines if watermarking enabled in spatial analytics pipelines	#80
How to use web camera source	#83
Error when trying to execute object line crossing pipeline	#84
Zone counting pipeline can hang when watermarking is enabled	#86

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
RTSP re-streaming plays back at frame processing rate, not encoded rate.	#68
Models can be picked up from previous build	#71
Docker build fails if directory name contains spaces	#38
Difficult to get normalized coordinates for spatial analytics parameters	#87
Docker build fails if no_proxy setting contains spaces	TBD
Spatial Analytics pipelines do not generate events with default parameters	TBD
MQTT clientID metadata destination property is not supported	TBD

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

v0.6.0-beta

Video Analytics Serving (VA Serving) is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.6.0-beta

Title	Description
Spatial Analytics	GVA Python powered pipeline extensions deliver zone counting and line crossing spatial analytics.
Extension Developer Guide	Guide for developing extensions.
Action Recognition	Added GStreamer and Edge AI Extension Module pipelines with a general-purpose action recognition composite model, consisting of encoder and decoder parts, trained on Kinetics-400 dataset.
Frame Record and Retrieve Sample	Added a sample that inserts a frame identifier into metadata allowing an application to retrieve corresponding frame from a frame store.
Edge AI Extension Module Improvements	Added spatial analytics and action recognition pipelines Properties in media stream descriptor can be added to the extensions field in results Removed scaling from topology for improved detection accuracy as DL Streamer will automatically scale to match selected model Added XPU operations file that runs three pipelines executing detection inference on CPU, GPU and VPU (NCS2) respectively.
OpenVINO 2021.4 support	GStreamer version updated to 1.18. Added action recognition models (feature preview)

Issues Resolved by This Release

Description	Issue
Build fails if UID is not 1000	#61
Intermittent error during concurrent pipeline tear-down in Edge AI Extension Module	#69
Tracking pipeline can drop results if tracking-type parameter is set to short-term	#72
vaclient gives misleading error if server not running	#74

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
VA Client outputs blank lines if watermarking enabled in spatial analytics pipelines	#80
Audio inference fails on some platforms	#79
RTSP re-streaming plays back at frame processing rate, not encoded rate.	#68
Models can be picked up from previous build	#71
Docker build fails if directory name contains spaces	#38

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

v0.5.0-beta

Video Analytics Serving (VA Serving) is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.5.0-beta

Title	Description
RTSP Re-streaming Preview	Experimental RTSP re-streaming support. Endpoint is defined as part of pipeline request.
Updated GStreamer Reference Pipelines	REST service now includes object tracking, classification, tracking and audio detection pipelines which have simplified and more consistent definition files.
Update AI Extension to support Azure Video Anaylzer (AVA)	AI extension now supports AVA so clients can select pipeline and its parameters via extension configuration field, enabling multiple different pipelines to run concurrently on different accelerators.
Added Reference Client	Added the “vaclient” command line sample application for issuing REST requests.
Improved EdgeX Sample	Showcases how to build and run a lean EdgeX DL application as independent microservices, integrated with EdgeX Network.
Improved Documentation Flow	Documentation has better flow from getting started to request customization, enabling hardware accelerators and finishing with changing pipeline model.
Updated OpenVINO Support	Updated DL Streamer base image to OpenVINO 2021.3.

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
RTSP re-streaming plays back at frame processing rate, not encoded rate.	#68
Intermittent error during concurrent pipeline tear-down in Live Video Analytics AI extension sample	#69
Memory leak if using gvametaconvert in pipeline	#70
Models can be picked up from previous build	#71
Tracking pipeline can drop results if tracking-type parameter is set to “short-term”	#72
Build fails if UID is not 1000	#61
Docker build fails if directory name contains spaces	#38

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

v0.4.1-beta

Video Analytics Serving (VA Serving) is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.4.1-beta

Title	Description
Hardware accelerator support	Updated VA Serving REST microservice and the Edge AI Extension sample to support Intel® Neural Compute Stick 2 and HDDL-R cards as inference devices.
Edge AI Extension Module	Updated to the latest version of gRPC AI Extension for Live Video Analytics on IoT Edge which includes a new tracking id metadata for object tracking.
Model Download Tool (MDT)	Added a shell script to provide a consistent environment and improved developer experience for downloading the models from Open Model Zoo.
Model-proc auto-selection	VA Serving can auto-select model-proc based on the model name. If a model-proc is not configured for an inference element in a pipeline, VA Serving will search the model-procs downloaded by MDT and select the appropriate one.

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

v0.4.0-beta

Video Analytics Serving is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFmpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.4.0-beta

Title	Description
Programmatic Input Sources and Destinations	Applications can now supply frames directly to pipelines and receive inference results. See the app_source_destination sample for more information.
Edge AI Extension Module	samples/lva_ai_extension contains a gRPC AI Extension for Live Video Analytics on IoT Edge by Microsoft*.
Model Download Tool	Introduction of Model Download Tool to fetch and organize deep learning models that power your pipelines. Models are downloaded at docker build time and may be stored/updated at a local or remote location.
Pipeline and Model Versioning	Pipeline and model versions are no longer restricted to integer values so can have more meaningful descriptions.
EdgeX Integration Sample	samples/edgex_bridge contains a sample demonstrating how to integrate video analytics with the Linux Foundation EdgeX Foundry.
Record and Playback Sample	samples/record_playback contains a sample demonstrating how to record incoming streams in parallel with inference and to playback results later.
Reduced Docker Build Time	Default docker builds use pre-built images from Docker Hub. All supported base images use Intel® distribution of OpenVINO™ Toolkit v2021.1.

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Docker build fails if directory name contains spaces	#38
dconf permissions	#45

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.

Release Notes (v0.4.0-alpha-preview)

Video Analytics Serving is a python package and microservice for deploying hardware optimized media analytics pipelines. It supports pipelines defined in GStreamer* or FFMpeg* media frameworks and provides APIs to discover, start, stop, customize and monitor pipeline execution. Video Analytics Serving is based on Intel® Distribution of OpenVINO™ Toolkit - DL Streamer and FFmpeg Video Analytics.

New and Changed in Release v0.4.0-alpha-preview

This is a preview of an upcoming v0.4.0-alpha release that extends functionality as follows:

Title	Description
Programmatic input sources and destinations	Applications can now supply frames directly to pipelines and receive inference results. See the app_source_destination sample for more information.
Live Video Analytics AI extension	A sample lva_ai_extension with gRPC AI Extension that works with Microsoft* Live Video Analytics.
Model download tool	Introduction of Model Download Tool to fetch deep learning models that power your pipelines. Models are downloaded at docker build time so longer need to be checked in
Pipeline and model versioning	Pipeline and model versions are no longer restricted to integer values so can have more meaningful descriptions
Additional samples	EdgeX integration. Synchronized recorded video and inference result playback
Additional base images	Intel(R) distribution of OpenVINO(TM) Toolkit runtime container (v2020.4) is now a supported base image. Version 2021.1 will build and run but has not been validated.

Known Issues

Known issues can be found as GitHub issues. If you encounter defects in functionality, please submit an issue.

Description	Issue
Pre-built base images do not support audio detection	#31
Docker build fails if directory name contains spaces	#38

Tested Base Images

Supported base images are listed in the Building Video Analytics Serving document.

* Other names and brands may be claimed as the property of others.