Introduction to the Video Processing Service Toolkit

Disclaimer

The Video Processing Service Toolkit contains pre-production samples intended to accelerate your implementation of a solution. However, the samples are not intended for production without modification. In addition to modifying pipelines to fit your use cases, you must harden security of endpoints and other critical tasks to secure your solution.

Overview

The VPS Toolkit is a MIP SDK toolkit that allows you to forward video from an XProtect camera device to a GStreamer pipeline. Video and/or metadata output from the GStreamer pipeline can be brought back into the XProtect VMS through a VPS Driver-based device. However, this is not mandatory; it is up to you how to consume the output of the GStreamer pipeline.

Below is shown an overview diagram of an XProtect VMS setup where the VPS Toolkit is processing video and sending it back into the XProtect VMS.

The orange arrows illustrate how video and metadata flows through the system originating from a real camera. Video and metadata is captured by a Camera Driver and forwarded to be optionally recorded on disk and optionally sent as a live feed to the Smart Client. This is how the flow of video and metadata normal works in a XProtect VMS.

The VPS Toolkit adds the ability to forward the feed (currently only video) unmodified through a VPS Driver. The VPS Driver pushes video to a VP Service which can then process the video using a GStreamer pipeline. Optionally, the video and/or metadata output from the GStreamer pipeline can be sent back through the VPS Driver into the XProtect VMS. This is shown in the diagram with the blue arrows. The feed coming back from the VP Service is received by the VPS Driver and is exposed through a new camera device and a new metadata device. From here on, the video and/or metadata feeds can be used just as if they had come from a physical camera device. For example, you can record the feed and watch both the live feed and recorded data in the Smart Client.

The VPS Toolkit provides a Management Client plugin named VPS Configuration to help configure this setup. The plugin offers a convenient way to select camera sources and specify the VP Service and GStreamer plugin to process the feeds.

The MIP SDK includes both ready-to-use binaries and source code for VP Service and VPS Configuration.

VPS Driver

VPS Driver is an integral part of XProtect version 2020 R1 and later.

The VPS Toolkit is not supported in XProtect releases before version 2020 R1.

VP Service

VP Service is an ASP .NET Core 2.1 web application. As such, it runs unmodified on Microsoft and Linux.

Vps2GStreamer

The Vps2GStreamer component is the binding between GStreamer and the C# service part of VPS. It is a C++ component including a P/Invoke interface with a few functions, PutData and GetData being the most important ones. PutData is used to push data from the C# part of the VPS into the GStreamer part, and GetData to get data out. Setting up and configuring the GStreamer pipeline happens in the private method SetupGStreamer of the Vps2GStreamer class.

GStreamer plugins

VPS Toolkit includes these GStreamer plugins:

• vpspasstru
• vpsjpegtranscoder
• vpsboundingboxes

vpspasstru simply passes video directly from input to output without modifying anything. If you use this plugin, you will basically just get a camera device in XProtect that shows the same video as the input camera device, a little delayed.

vpsjpegtranscoder will decode the video it receives (e.g. H.264) and encode it to JPEG. Visually, the experience will be similar to that of the vpspasstru plugin, but the video format of the new camera device will always be JPEG, no matter what the input format is.

vpsboundingboxes appends a bounding box in ONVIF format to each frame. Visually, in the Smart Client you will see a green bounding box moving across the view item for the new camera device. vpsboundingboxes also demonstrates how to pass parameters to a GStreamer plugin.

The plugins provided use standard GStreamer elements, and there is no GPU acceleration involved.

For further samples of interesting plugins, check out the NVIDIA DeepStream SDK (https://developer.nvidia.com/deepstream-sdk). Note that DeepStream is for Linux only.