ImageServer - requests and responses

An ImageServer session uses a TCP socket connection to an XProtect Recording Server.

A Recording Server may be located on the same computer as the Management Server, but it can be located on any computer. The address of the Recording Server for at given device can be retrieved using ServerCommandService.GetConfiguration().

The default port number is 7563. Using your socket library, connect to this address using TCP. This connection can be kept open throughout the entire session with a device.

You do not need to close the socket until you are done with that device. All ImageServer requests consist of a small serialized XML document with a root element named <methodcall>. Likewise, many responses — but not all — consist of a serialized <methodresponse> XML document.

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <methodname>connect</methodname>
</methodcall>
<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <methodname>connect</methodname>
</methodresponse>

The terminology from here on will be to refer to any <methodcall>xyz</methodcall> as the "xyz request" and to refer to <methodresponse>xyz</methodresponse> as the "xyz response".

List of Requests and their Responses:

connect – Simple

The first thing to send after having successfully opened a TCP socket is always a connect request.

Request - Simple example

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>1</requestid>
  <methodname>connect</methodname>
  <username>dummy</username>
  <password>dummy</password>
  <cameraid>[guid]</cameraid>
  <connectparam>id=[guid]&amp;connectiontoken=[token]</connectparam>
</methodcall>

Request - Simple example with StreamId

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>1</requestid>
  <methodname>connect</methodname>
  <username>dummy</username>
  <password>dummy</password>
  <cameraid>[guid]</cameraid>
  <connectparam>id=[guid]&amp;streamid=[guid]&amp;connectiontoken=[token]</connectparam>
</methodcall>

The id is the guid of the device, and the streamid is the guid for a specific stream as configured in XProtect. The order of the parameters must be maintained.

connect – More options

The <alwaysstdjpeg> element allows you to force transcoding to JPEG.

It is recommended to specify no to receive the raw data in the format produced by the camera. If you set the <alwaysstdjpeg> element to yes, all live video data will be transcoded and returned in standard image/jpeg format.

If you have not forced transcoding, you will receive the raw camera data embedded in Milestone's "generic byte data" headers. Such data may be a lot more compact to transfer and will consume less CPU on the server, but currently we do not provide any library with which you can decode it.

The <clientcapabilities> node specifies what the client supports.

The <privacymaskversion> element can be specified along with the <privacymask> element to specify whether the client understands newer versions of the privacy mask format. If not specified or if the client specify '0', the server will send the privacy mask in the simple format specifying only the size of the grid followed '1' for each area not to be shown. If specifying '1' the server will send the privacy mask as a base64 encoded xml string if supported by the server.

The <multipartdata> element tells the server if the client supports the interpretation of multiple data packages in a single response. This will allow the server to send all the frames of an MPEG GOP as one response and also to optimize in other situations.

The <datarestriction> element tells the server if the client implements data restriction.

The <transcode> node contains elements for configuring the transcoding of the video data. It does not explicitly force transcoding.

The <allframes> element in the <transcode> node applies to cameras with an MPEG stream. If set to no, which is the default, all sub-frames are skipped when transcoding, and only key-frames are transcoded. Typically, this will result in an effective frame rate of 1 per second. If set to yes, all sub-frames may be transcoded, and the effective transcoded frame rate may be higher than 1.

The elements <width>, <height>, <keepaspectratio> and <allowupsizing> in the <transcode> node enable the application to specify the output image size measured in absolute pixels.

The <timerestriction> element is used when export is about to begin. The purpose is to limit the audit logging to a single entry as long as the video frames being retrieved are within the specific start and end time stamp. When this entry is not available, the server will make one line in the audit log for each minute of video being accessed.

Connect request – the full syntax

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>connect</methodname>
  <username>[text]</username>
  <password>[text]</password>
  <cameraid>[guid or cameraname]</cameraid>
  <alwaysstdjpeg>yes/no</alwaysstdjpeg>               (Recommended)
  <connectparam>[text]</connectparam>
  <clientcapabilities>                                (optional)
    <privacymask>yes/no</privacymask>                 (optional)
    <privacymaskversion>0/1</privacymaskversion>      (optional, def=0)
    <multipartdata>yes/no</multipartdata>             (optional)
    <datarestriction>yes/no</datarestriction>         (optional)
  </clientcapabilities>
  <transcode>                                         (optional)
    <allframes>yes/no</allframes>                     (optional, def=no)
    <width>[number]</width>                           (optional, def=unaltered)
    <height>[number]</height>                         (optional, def=unaltered)
    <keepaspectratio>yes/no</keepaspectratio>         (optional, def=no)
    <allowupsizing>yes/no</allowupsizing>             (optional, def=no)
  </transcode>
  <timerestriction>                                   (only one log and control client behaviour – time forward only)
    <starttime>[milliseconds since epoc]</starttime>
    <endtime>[milliseconds since epoc]</endtime>
  </timerestriction>
</methodcall>

Response

<?xml  version="1.0" encoding="UTF-8"?>
<methodresponse>
  <requestid>[actual]</requestid>
  <methodname>connect</methodname>
  <connected>yes/no</connected>
  <errorreason>[text]</errorreason>                   (only if connected =  no)
  <alwaysstdjpeg>[actual]</alwaysstdjpeg>
  <camera>
    [camera info]
  </camera>
  <clientcapabilities>
     <privacymask>[actual]</privacymask>
     <privacymaskversion>[actual]</privacymaskversion>
     <multipartdata>[actual]</multipartdata>
     <datarestriction>[actual]</datarestriction>
  </clientcapabilities>
  <servercapabilities>
     <connectupdate>[actual]</connectupdate>
     <adaptivestreamingversion>[version]</adaptivestreamingversion>
  </servercapabilities>
  <transcode>
     <allframes>[actual]</allframes>
     <width>[actual]</width>
     <height>[actual]</height>
    <keepaspectratio>[actual]</keepaspectratio>
     <allowupsizing>[actual]</allowupsizing>
  </transcode>
</methodresponse>

connectupdate

Before every response, the server repeatedly validates the token you passed with the initial connect request or the previous connectupdate request.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>connectupdate</methodname>
  <connectparam>[text]</connectparam>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>connectupdate</methodname>
  <connected>yes/no</connected>
  <errorreason>[text]</errorreason>                   (only if connected =  no)
</methodresponse>

goto

Sets the database pointer for the currently connected camera to a specific timestamp and returns a recorded image close to the timestamp.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>goto</methodname>
  <time>[milliseconds  since UNIX epoch]</time>
  <compressionrate>[number]</compressionrate>        (optional)
  <keyframesonly>yes/no</keyframesonly>              (optional)
</methodcall>

Response - Live and browse without multipart

ImageResponse
RequestId: [number]
PrivacyMask: none/grid;3x3;110;110;000/[base64 encoded xml]  (first time and when mask changes)
Restriction: none/time
Prev: [milliseconds since UNIX epoch]
Current: [milliseconds since UNIX epoch]
Next: [milliseconds since UNIX epoch]
SequenceNumber: [number] (if requested)
MotionLevel: [number] (if requested)
Content-length: [number]
Content-type: image/jpeg
 
[binary data]

You should note that this response is basically different from the other ImageServer responses, in that it is not formatted as an XML document. Instead, it is formatted as an HTTP response, having its initial HTTP identification line with the HTTP status code replaced by an initial line with just the identifier string ImageResponse.

The length of the [binary JPEG data] is equal to the Content-length parameter.

The value returned in the Current header line indicates the actual time for the image. This may be different from the time you requested an image from.

Response - Browse with multipart

ImageResponse
RequestId: [number]
PrivacyMask: none/grid;3x3;110;110;000/[base64 encoded xml]  (first time and when mask changes)
Restriction: none/time
Prev: [milliseconds since UNIX epoch]
Current: [milliseconds since UNIX epoch]
Next: [milliseconds since UNIX epoch]
SequenceNumber: [number] (if requested)
MotionLevel: [number] (if requested)
Content-type: multipart/related; boundary=frontier
 
--frontier
Content-length: [number]
Content-type: application/x-genericbytedata-octet-stream
 
[binary data]
--frontier
Content-length: [number]
Content-type: application/x-genericbytedata-octet-stream
 
[binary data]
.
.
.
--frontier
Content-length: [number]
Content-type: application/x-genericbytedata-octet-stream
 
[binary data]
--frontier--
 

Privacy mask

The PrivacyMask will be returned the first time or whenever the mask changes. The mask can be one of the following three values:

The privacy mask XML conforms to privacy_protection_mask_schema.xsd. The following shows an example of the content of a privacy mask.

<?xml version="1.0" encoding="utf-8"?>
<mask>
  <methods>
    <solid value="1" removable="false">
      <color>
        <red>255</red>
        <green>0</green>
        <blue>0</blue>
      </color>
    </solid>
  </methods>
  <grid size="8x8">
    <row index="1" values="0100000000000000" />
    <row index="3" values="0000000000010100" />
  </grid>
</mask>

The <methods> element contains a number of method specific child elements such as <solid> where each method can have zero or more method specific parameters. A method always has a unique value attribute, which is the value that is written in the values attribute of a row in the grid.

The removable attribute indicates whether or not the grid cells for the method may be lifted by the user or not.

The <grid> element has the size attribute which specifies the number of rows and columns in the mask and thus the number of areas/cells.

The <row> element has two attributes, index and values.

The byte value indicates which method to use for the area/cell where the method is found by looking up the method with the specified numeric value - e.g. the method /mask/methods/*[@value='numeric-value'].

A mask with no rows is valid and is considered an empty mask.

next

Moves the database pointer for the currently connected camera to the next image and returns the image. The element <time> is optional. If not included, the current value is used.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>next</methodname>
  <time>[milliseconds  since UNIX epoch]</time>       (optional)
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

previous

Moves the database pointer for the currently connected camera to the previous image and returns the image. The element <time> is optional. If not included, the current value is used.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>previous</methodname>
  <time>[milliseconds  since UNIX epoch]</time>       (optional)
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

nextsequence

Moves the database pointer for the currently connected camera to the first image in the next sequence and returns the image. The element <time> is optional. If not included, the current value is used.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>nextsequence</methodname>
  <time>[milliseconds  since UNIX epoch]</time>       (optional)
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

previoussequence

Moves the database pointer for the currently connected camera to the first image in the previous sequence and returns the image. The element <time> is optional. If not included, the current value is used.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>previoussequence</methodname>
  <time>[milliseconds  since UNIX epoch]</time>       (optional)
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

begin

Moves the database pointer for the currently connected camera to the first image in the database and returns the image. The response includes timestamps for the previous, the returned and the next image in the database and the image data in binary format.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>begin</methodname>
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

end

Moves the database pointer for the currently connected camera to the last image in the database and returns the image. The response includes timestamps for the previous, the returned and the next image in the database and the image data in binary format.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>end</methodname>
  <compressionrate>[number]</compressionrate>         (optional)
</methodcall>

Response

See the goto response.

alarms – Get recorded sequences between two points in time

This requests information about all the current camera's recorded sequences within the specified period of time.

Request

<?xml version="1.0" encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>alarms</methodname>
  <starttime>[milliseconds since UNIX epoch]</starttime>
  <stoptime>[milliseconds since UNIX epoch]</stoptime>
</methodcall>

Response

<?xml version="1.0" encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>alarms</methodname>
  <dblimit startTime="[milliseconds since UNIX epoch]" endTime="[milliseconds since UNIX epoch]"/>
  <alarms>
    <alarm  startTime="ms" alarmTime="ms" endTime="ms" numImages="[number]"/>
    <alarm  startTime="ms" alarmTime="ms" endTime="ms" numImages="[number]"/>
    ...
  </alarms>
</methodresponse>

The <dblimit> element contains the timestamp of the first and last image in the database.

alarms – Get max N recorded sequences around a point in time

This requests information about the current camera's recorded sequences around the specified center time.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>alarms</methodname>
  <centertime>[milliseconds since UNIX epoch]</centertime>
  <numalarms>[max number of alarms to return before and after center time]</numalarms>
  <timespan>[max time before and after center to look for alarms]</timespan>
</methodcall>

Response

Same as the response listed in the first example of using Alarms. Using this variant of the request, you may compute what the possible maximum size of the response is.

live

Start live feed form the currently connected camera.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>live</methodname>
  <compressionrate>[number]</compressionrate>  (optional)
  <sendinitialimage>yes/no</sendinitialimage>  (optional)
  <attributes [text]/> (optional)
  <adaptivestreaming>  (optional, if stated one option must be supplied)
    <resolution>       (option #1)
      <widthhint>[number]<widthhint/>   (range 1 - Int32.Max)
      <heighthint>[number]<heighthint/> (range 1 - Int32.Max)
    </resolution>
    <maxresolution/>   (option #2)
    <disabled/>        (option #3)
  </adaptivestreaming>
</methodcall>
Option Value Comments
frameraterelative Integer N Send every N'th frame only
framerateabsolute Integer N Send as close to N fps as possible (future enhancement)
framerate full/medium/low Reduce frame rate by dropping frames
motiononly yes/no Send only image sequence where motion is detected

Example: <attributes framerate="full" motiononly="true" />.

framerateabsolute is a future enhancement. It will overrule framerate when implemented.

frameraterelative overrules framerate.

The three values for the framerate option aim to send all images at full level (no filter), around one image pr. second at low level and a rate in between those two at medium level. The frame reduction scheme is implemented by simply dropping frames coming from the monitor pipe, not taking time into account.

The scheme for dropping frames is listed in the following table, where 0 = drop all frames, 1 = never drop frames, x > 0 = send 1 out of x frames.

Stream type/Value Low Medium Full
JPEG 20 4 1
MPEG - IFrame 1 1 1
MPEG - PFrame 0 0 1
Option Value Comments
resolution Integer N
Integer N		 Use the optimal stream, which doesn't require upscaling, according to width and height
maxresolution none Use the highest resolution stream (width*height)
disabled none Don't use Adaptive streaming

Responses

The result of sending the live request is a series of responses of two different types, GoTo or LivePackage. Each response ends with a double CR-LF, which means four bytes with decimal values 13, 10, 13, 10. The GoTo response is described in the GoTo section.

The LivePackage responses arrive with regular intervals, so you should keep your socket receive active at all times.

Each LivePackage response contains status information for the currently connected camera. This includes information about camera-to-server connection problems, database errors, recording status and motion status. The package has the following format.

<?xml version="1.0"  encoding="UTF-8"?>
<livepackage>
  <status>
    <statustime>[milliseconds since UNIX  epoch]</statustime>
    <statusitem id="[Number]" value="[Text]" description="[Text]"/>
    ...
  </status>
</livepackage>

The status time given for each information package is the (server) time at which the values of the status items have been read out. The time format for the status time is similar to all other time stamps from the ImageServer. The currently supported status items are listed in the table below. The description part of each status item is optional. A live information package will always contain exactly one status time and may contain from zero to all of the status items.

ID Description Value Comments
1 Camera live feed started 0, 1 1 if live feed from the camera is started. 0 if not.
2 Live feed motion 0, 1 1 if the live feed contains motion. 0 if not.
3 Live feed recording 0, 1 1 if the live feed is being recorded. 0 if not.
4 Live feed event notification 0, 1 1 if there was an event notification during the live feed. 0 if not.
5 Camera connection lost 0, 1 1 if the connection between the camera and the server is lost. 0 if not.
6 Database fail 0, 1 1 if accessing the database failed. 0 if not.
7 Server is running out of disk space 0, 1 1 if the server is running out of disk space. 0 if not.
100 Client Live feed stopped 0, 1 1 if live feed to client is stopped. 0 when live feed is started.

live – Change live adaptive streaming

Update options for Adaptive streaming for currently connected camera.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>live</methodname>
  <adaptivestreaming>  (optional, if stated one option must be supplied)
    <resolution>       (option #1)
      <widthhint>[number]<widthhint/>   (range 1 - Int32.Max)
      <heighthint>[number]<heighthint/> (range 1 - Int32.Max)
    </resolution>
    <maxresolution/>   (option #2)
    <disabled/>        (option #3)
  </adaptivestreaming>
</methodcall>
Option Value Comments
resolution Integer N
Integer N		 Use the optimal stream, which doesn't require upscaling, according to width and height
maxresolution none Use the highest resolution stream (width*height)
disabled none Don't use Adaptive streaming

Responses

None.

changelivecompression rate

Immediately changes the compression rate for the live stream.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>changelivecompressionrate</methodname>
  <compressionrate>[number]</compressionrate>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>changelivecompressionrate</methodname>
</methodresponse>

This interpretation is valid for all commands in this API which have an element named <compressionrate>. Special values are 101-104:

stop

Stops live feed streaming.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>stop</methodname>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>stop</methodname>
</methodresponse>

ptz

A "PTZ (Pan, Tilt Zoom)" command is executed on the camera you are connected to.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>ptz</methodname>
  <ptzcommand>[ptzcommand]</ptzcommand>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>ptz</methodname>
</methodresponse>

The [ptzcommand] is one of these text strings:

The camera will perform one step in the given direction, or move to home position for that command.

ptzcenter

A "PTZ Move" command is executed on the camera you are connected to.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>ptzcenter</methodname>
  <refwidth>[number]</refwidth>
  <refheight>[number]</refheight>
  <centerx>[number]</centerx>
  <centery>[number]</centery>
  <zoom>[number]</zoom>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>ptzcenter</methodname>
</methodresponse>

ptzrectangle

A "PTZ Rectangle" command is executed on the camera you are connected to.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>ptzrectangle</methodname>
  <refwidth>[number]</refwidth>
  <refheight>[number]</refheight>
  <left>[number]</left>
  <top>[number]</top>
  <right>[number]</right>
  <bottom>[number]</bottom>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>ptzrectangle</methodname>
</methodresponse>

preset

A "PTZ Move to Preset" command is executed on the camera you are connected to.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>preset</methodname>
  <presetname>[presetid]</presetname>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>preset</methodname>
</methodresponse>

output

Activates a named output on the camera you are connected to

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>output</methodname>
  <outputname>[outputid]</outputname>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>output</methodname>
</methodresponse>

aviinformation

Returns AVI export related information about the recorded images within a specified time span for the currently connected camera.

Request

<?xml version="1.0"  encoding="UTF-8"?>
<methodcall>
  <requestid>[number]</requestid>
  <methodname>aviinformation</methodname>
  <start>[milliseconds  since UNIX epoch]</start>
  <stop>[milliseconds  since UNIX epoch]</stop>
</methodcall>

Response

<?xml version="1.0"  encoding="UTF-8"?>
<methodresponse>
  <requestid>[number]</requestid>
  <methodname>aviinformation</methodname>
  <width>[number]</width>
  <height>[number]</height>
  <depth>[number]</depth>
  <fps>[float]</fps>
</methodresponse>