WebRTC - JavaScript client

This sample uses a RESTful API for the signaling required to establish a WebRTC connection with an XProtect VMS through the API Gateway.

Playback support in BETA
Support has been added for specifying playback time, speed, and whether gaps should be skipped when creating a WebRTC session.
To be considered beta for this release.

Prerequisites

Set up WebRTC sample

You can open the sample webpage index.html in a browser directly from the sample directory, or host the sample directory WebRTC_JavaScript on a web server.

If you intend to serve the sample directly from the sample directory:

  1. Configure API Gateway CORS settings.

If you intend to serve the sample from a web server:

  1. Copy the WebRTC_JavaScript sample directory to a host with a web server, for example the API Gateway host.
  2. Configure the web server to serve the sample directory at, for example, /WebRTC_JavaScript.
  3. If the sample webpage is not served from the same host URL as the API Gateway, you must configure API Gateway CORS settings.

Configuration files

API Gateway configuration files are located in the installation location, by default %ProgramFiles%\Milestone\XProtect API Gateway\.

These configuration files are relevant for the sample:

Do not edit appsettings.json manually. This file is created by the product installer and maintained by the Server Configurator.

If needed, create appsettings.production.json and add configuration overrides to this file. This file will not be removed or changed by product updates.

Use a validating editor to edit configuration files. Most popular code editors support JSON and XML syntax validation, either by default or through extensions.

Syntax errors in the API Gateway configurations files result in 502 Bad Gateway or 503 Service Unavailable server errors and will show up in the Windows Application event log and the IIS request log.

API Gateway CORS settings

If the sample webpage is not served from the same origin host URL as the API Gateway, the browser will block requests to the API Gateway from the sample scripts unless the API Gateway is configured to support CORS (Cross-Origin Resource Sharing).

CORS is disabled by default. You enable and configure CORS support by creating and editing appsettings.production.json.

  1. Create appsettings.production.json (if not already created).

  2. For development and test purposes, you can use a very permissive policy:

    {
  "CORS": {
    "Enabled": true,
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Headers": "*",
    "Access-Control-Allow-Methods": "*"
  }
}

    This will allow calls from any origin, including a local file system, to the API Gateway.

    In a production environment, this is not secure, and CORS should be configured correctly in order to keep the system secure.

  3. Restart the IIS, or at least recycle VideoOS ApiGateway AppPool.

For more information about CORS, please refer to Cross-Origin Resource Sharing (CORS).

Running

WebRTC - JavaScript client
  1. Open index.html in a browser
  2. Enter the URL of the API Gateway.
    Usually, the API Gateway is installed on the same host as the management server, that is, the API Gateway will be something like https://managementServer.example.com/api.
  3. Enter the CameraId of a camera that supports H.264.
    See Get the CameraId.
  4. Optionally, enter a StreamId. See Get a StreamId.
  5. Optionally, enter an ISO 8601-formatted datetime string in Playback Time to request video playback instead of live video.
  6. Optionally, select Skip Gaps to skip gaps between video sequences in video playback.
  7. Optionally, enter a number in Playback Speed. The default is 1.0, and any value > 0 and ≤ 32 is valid.
  8. Enter Username and Password of a basic user with access to the camera.
  9. Optionally, enter a STUN server address, for example stun:stun1.l.google.com:19302.
  10. Optionally, enter a TURN server address, username, and credential (password).
  11. Select Start to establish the connection.

Get a CameraId

You can use the Management Client to get a CameraId:

  1. In the Site Navigation pane, select Servers and then select the recording server.
  2. Select a camera that support H.264 in the Overview pane.
  3. Select the Info tab in the Properties pane.
  4. Ctrl+Click the video image in the Preview pane.
    The camera ID will be displayed at the bottom of the Properties pane.

Get a StreamId

Cameras may have multiple streams. You can specify a StreamId to request a specific stream. If the connection is initiated without a StreamId, the camera’s default stream will be used.

If an invalid StreamId is specified, a WebRTC session cannot be initiated, and the diagnostics log will display an error message.

You can use the Management Client to get a StreamId:

  1. In the Site Navigation pane, select Servers and then select the recording server.
  2. Select the camera hardware in the Overview pane.
  3. Unfold the camera hardware and select the camera device.
  4. Select the Streams tab in the Properties pane.
  5. Ctrl+Click the video image in the Preview pane.
  6. Select the Stream XML button.
  7. A small XML document describing available streams will open in your default XML application.
  8. Use the value of a <referenceid> element of an H.264 as StreamId.

Description

Please look at the main.js code while reading the following steps:

  1. In initiateWebRTCSession(), the session is initiated by a POST request to API/REST/v1/WebRTC/Session.
    The request body contains
  2. The response contains the newly created sessionId and the offerSDP which is used to update the RTCPeerConnection object pc.
  3. An answerSDP is created based on pc, and it is sent to the session by a PUT request to API/REST/v1/WebRTC/Session.
  4. Next, ICE candidates will be exchanged in two methods: addServerIceCandidates() polls the API Gateway server while the connection state is new or checking, and SendIceCandidate() is called every time pc finds an ICE candidate.
  5. Once ICE candidates have been exchanged, WebRTC will try to establish a connection between the peers.

The bearer token expires (default after 1 hour). Code for getting and refreshing the token can be found in tokenCommunication.js; see the methods getToken and refreshToken.

For more information about the signaling involved in establishing a WebRTC connection, please refer to: WebRTC API, Signaling and video calling

Playback of recorded video [BETA]

If a playback start time is provided, recorded video is streamed instead of live video. The API Gateway will start streaming at the requested time. If there’s no video recorded at that time and skip gaps is enabled, the stream will immediately forward to the first video sequence after the requested time.

Playback is controlled by including an optional PlaybackTimeNode object when initiating the session. PlaybackTimeNode is an object taking these values:

How to calculate time for current frame

  1. In start() in main(),
    • frameStartTime is set to either Date.now or the requested playback time. frameStartTime will be considered the start of streaming.
    • A callback function is registered for the <video> player. The function will be called next time a frame has been received.
    • The WebRTC session is initiated.
  2. In the callback function, the rtpTimestamp is extracted from WebRTC frame metadata. rtpTimestamp is the time since streaming started in milliseconds.
    To get frameDate (the date and time for the current frame), the rtpTimestamp value is added to frameStartTime.
  3. The function is registered again to be called next time a frame has been received.

To register the callback function, the sample uses requestVideoFrameCallback() (currently a W3C Draft Community Group Report) for most browsers, and requestAnimationFrame() for FireFox:

When there are gaps between video sequences:

In any case, the rtpTimestamp for the next video sequence jumps as if frames had been streamed during the gap.

STUN and TURN server addresses

To help establish a connection through NATs, WebRTC uses STUN (Session Traversal Utilities for NAT) and/or TURN (Traversal Using Relays around NAT) servers.

A STUN server is used to discover the public IP address and port number of a device behind a NAT.

A TURN server is used to relay traffic between peers when a direct connection is not possible due to firewall or NAT restrictions.

The sample supports specifying one of each in the user interface. More than one STUN and/or TURN server can be specified, but the sample only supports up to one of each.

The values passed on from the user interface are used in:

No default STUN or TURN server URLs are configured API Gateway-side. To do so, the URLs for STUN and TURN servers can be defined in appsettings.production.json:

{
  "WebRTC": {
    "iceServers": [ 
      { "url": "stun:mystun.zyx:3478"}, 
      { "url": "turn:myturn.zyx:5349"} 
    ] 
  }
}

While it is no longer necessary to send the configuration to the API Gateway during signaling (in body.iceServers in initiateWebRTCSession() method), setting body.iceServers when creating a RTCPeerConnection instance is still required.

TURN servers that require username and credential
appsettings.production.json cannot be used for a TURN server that require username and credential.

For more information about STUN and TURN, see WebRTC API STUN and WebRTC API TURN.

Trickle ICE

The API Gateway fully supports trickle ICE, but the sample keeps checking on new candidates from the server only while the ICE gathering state is new or checking. Once the connection between peers has been established, the sample stops polling the API Gateway.

To support getting candidates from the server at any time during the connection, the API Gateway must be polled periodically during the whole duration of the connection.

WebRTC features in browsers

Each browser has different levels of support for WebRTC features. The sample makes use of the Adapter.js library which allows for improved browser compatibility when using WebRTC API.

Limitations and workarounds

WebRTC connection on a local network uses mDNS

To prevent private IP addresses from leaking from a local network when running WebRTC applications, modern browsers by default send mDNS (multicast DNS) addresses as ICE Candidates to the signaling server.

API Gateway support for mDNS

The signaling server running in the API Gateway supports resolving mDNS addresses when running on a Windows version with native support for mDNS. Native support for mDNS was introduced in Windows version 1809 (October 2018) or later, and is available in any recently updated Windows Server 2019 or Windows 10 installations, and all Windows Server 2022 and Windows 11 installations.

WebRTC connections across routers in a local network

mDNS relies on multicast which by default will not pass through routers. This means that in enterprise environments, mDNS will fail in many cases:

The signaling server running in the API Gateway supports a workaround for connections across routers on a local network. The signaling server will attempt to get the client’s local IP network address from X-Forwarded-For and Remote_Addr headers in the HTTP request and use that to add an ICE Candidate with higher priority than the ICE Candidate with the mDNS address. This will not work in all cases; on some networks, X-Forwarded-For is removed and Remote_Addr will not contain the local IP address of client.

Disable browser mDNS support

As a last resort, you can try disabling browser mDNS support to force the browser to reveal the local IP network address in WebRTC connections.

In Chromium-based browsers, mDNS support can be disabled by opening chrome://flags or edge://flags and setting Anonymize local IPs exposed by WebRTC to Disabled.

Troubleshooting tips

CORS errors

Symptoms

Cause

The sample is not served from same host server URL as the API Gateway, and CORS support has not been enabled.

Remedy

Enable CORS support as described in API Gateway CORS settings.

Cause

Errors are sometime presented in the browser as CORS error without being actual CORS issues. If you see a CORS error message in the browser, it could be related to configuration issues in the IIS.

Remedy

Open your browser Developer tools and select the Network tab. If it is not an CORS error, the actual error will be shown here in the messages received before the CORS error.

No connection through a symmetric NAT firewall

Symptoms

Cause

WebRTC cannot create a connection through a symmetric NAT firewall without using a TURN (Traversal Using Relays around NAT) server. Without using a TURN server, a WebRTC connection is possible only within the symmetric NAT firewall, that is, both your browser and the API Gateway must be behind the firewall.

Check with your system administrator if you are behind a symmetric NAT firewall, or run the test described here: Am I behind a Symmetric NAT?.

Remedy

Specify a TURN server.

See STUN and TURN server addresses.

Server errors

Symptoms

Cause

Syntax errors in the appsettings configuration files will prevent the API Gateway from starting.

Remedy

Do not edit appsettings.json manually. This file is created by the product installer and maintained by the Server Configurator.

Open and edit appsettings.production.json in a validating editor. For more information, see Configuration.

IIS crashes

Symptoms

Remedy

Try temporarily enabling 32-Bit Applications in the IIS application pool VideoOS ApiGateway AppPool. If WebRTC is the cause of the crash, enabling 32-Bit Applications may provide you with more detailed error information.

The sample demonstrates

Using