Quantum

REST Services API

The Service/Jobs API allows CatDV Server to be used by external components as a simple queuing mechanism to allow background jobs to be scheduled and executed, and also to provide feedback to users on the progress of these background jobs.

It is anticipated that multiple different external components will be using the job queue at the same time and to facilitate this, we introduce the concept of a ‘service’, which is a representation of some external component that runs independently and wishes to add and remove items from the queue. CatDV places no constraints on the implementation of the service, save that it registers itself with CatDV Server by adding an entry into the Service table.

Each service has a type and all jobs created by that service are tagged with that type. The values stored against each entry in the Jobs table are mostly defined by the service itself, with only a few standard fields.

Endpoint: /services

Represents the collection of all services on the server:

Object Definitions

Service

The Service object represents an external service that wishes to use the job queue.

Field

Type

Description

serviceType

String

Name of the type of service provided by this service. E.g. Amazon S3. This will be used to tag the Jobs belonging to this service

serviceUID

String

A unique identifier for the service. To ensure uniqueness it should include the domain name of the organisation that created the service as well as some locally unique identifier. E.g squarebox.catdv.plugin.s3archive.in_server

name

String

Short descriptive name to display to the user in the admin UI.

description

String

Longer description of the service.

createdDate

Date

Date this service entry was created (filled in by system)

lastModifiedDate

Date

Last time this record was updated (filled in by system)

statusCode

Number

Traffic light status for the service. (0 – Unknown, 1 – Red (Error), 2 – Yellow (Warning), 3 – Green (OK) ). Used to implement traffic light status display similar to that used in CatDV’s Control Panel

status

String

Short status message to display next to the traffic light.

statusDetails

String

Longer status message to be displayed on-demand.

statusData

JSON

Additional status information that the service may wish to store.

config

JSON

Location for the Service to store any configuration information that it needs (server addresses, user names etc.).

data

JSON

Any additional data that the service needs to store.

jobDataDesc

HTML/Template

An HTML template used to display the ‘data’ field of the Job entries for this service. (see Job Data Templates section for details)

Operations

GET /services[?serviceType=<service_type>][&serviceUID=<service_uid>]

Returns a list of services optionally filtered by serviceType and serviceUID.

GET /services/<service_id>

Returns the Service object specified by ‘service _id’.

POST /services

Adds a new Service object to the database. The HTTP POST data should contain a JSON representation of the Service object.

PUT /services/<service_id >

Updates the Service object specified by the service _id. HTTP POST data should contain a JSON object populated with the fields to be updated.

Endpoint: /jobs

Represents the collection of all jobs on the server:

Object Definitions

Job

The Job object represents a job in the queue.

Field

Type

Description

serviceType

String

Type of service that created this Job.

jobType

String

Type of this Job. Values are specific to the Service that created the Job (e.g. Archive, Backup etc.)

userID

Number

ID of the user who initiated this job.

clipID

Number

(Optional) ID of the clip this job relates to, where relevant

createdDate

Date

Date this job was created (filled in by system)

lastModifiedDate

Date

Last time this record was updated (filled in by system)

description

String

Human readable description of this job.

priority

Number

Indicates the priority of this job compared to other jobs. Zero is normal priority – positive numbers indicate raised priority and negative number reduced priority.

status

String

Short status message to display to user.

statusCode

Number

Basic status information for the job

-1 – Failed 0 – Queued 1 – Running 2 – Complete

allocatedToService

String

ServiceUID of the service that is processing this job. Initially null, when the job is in the queue, and then set to the UID of the service that picks the job off the queue and executes it.

allocatedDate

Datetime

When this job was allocated to the service.

data

JSON

Any data the service wishes to store against the job. May be displayed to the user by applying the template specified in the service’s jobDataDesc field (see Job Data Templates section for details)

JobResult

Represents the results form one execution of a job. It is possible to re-submit a failed job, resulting in multiple JobResult entries for one Job.

Field

Type

Description

jobID

String

ID of job that created this Job.

Succeeded

True/False

Was this execution successful

createdDate

Date

Date this job result was created (filled in by system)

ouput

String

Textual describing the result of the execution

Operations

GET / jobs[?queries=<num_queries>]…

Returns an array of the jobs on the server matching the, optionally specified, criteria.

The Jobs endpoint allows you to retrieve multiple result subsets, each independently filtered and sorted, that are concatenated together to form the final result set.

The optional ‘queries’ parameter specifies the number of these result subsets that you require and each of the other parameters (jobType, status etc.) takes a comma separated list of values specifying the filter or sorting values for each subset. If the ‘queries’ parameter is not present then a single result set it returned and the other parameters revert to taking single values.

Parameter

Type

Description

queries

number

Specifies the number of result sub-sets that should be retrieved and returned as a single combined result set.

serviceType

string[]

The type(s) of the Service that created the job entry (see the /services endpoint for more details)

jobType

string[]

The type(s) of Job that should be returned. The values for this field are defined by the service that created the job entry.

parentJobID

number[]

The value(s) of the parentJobID for the Job entries.

status

string[]

Value(s) of the status field for the Jobs to be returned

sortBy

string[]

Name(s) of the fields that this result set/subset should be sorted by.

sortDir

string[]

Direction (ASC or DESC) for sort the result set/subset by.

GET /jobs/< job_id >

Returns the job with the specified ‘job_id’.

GET /jobs/< job_id >/results

Returns a list of the JobResults associated with the Job specified by ‘job_id.

GET /jobs/<job_id>/results/<result_id>

Returns a the JobResult with the specified ‘result_id’ that belongs to the Job specified by ‘job_id’.

POST /jobs

Adds a new Job object to the queue. The HTTP POST data should contain a JSON representation of the Job object.

POST /jobs/<job_id>/results

Adds a new JobResult object to the job specified by ‘job_id’. The HTTP POST data should contain a JSON representation of the JobResult object.

PUT /jobs/< job_id >

Updates the Job object specified by the job_id. HTTP POST data should contain a JSON object populated with the fields to be updated.

PUT /jobs/< job_id>/results/<result_id>

Updates the JobResult object specified by result_id that belongs to the Job specified by ‘job_id’. HTTP POST data should contain a JSON object populated with the fields to be updated.

Job Data Templates

As far as possible the Services/Jobs API places as few restrictions on the data that the service can store against each Job. However, it is useful to be able to display detailed status information to the user, and to this end the Service object may include an HTML template to allow the admin UI to format the status information held against each Job.

This template is stored in the Service’s ‘jobDataDesc’ field and holds a template to format the job’s data object, which is assumed to be in JSON.

This is an example template

<h2>${result}</h2> <table width="100%" border="1"> <tr><th>Name</th><th>Format</th></tr> <%for(clip:clips)%> <tr><td>${clip.name}</td><td>${clip.format}</td></tr> <%endfor%> </table>

The template is essentially HTML with some additional tags to allow the HTML to display data extracted from the JSON data object stored in the Job object.

To display a single value the following variable reference may be used:

${variable_name}  ${object_variable_name.propertyName}    

This substitutes the value of the variable into that position.

If the JSON contains an array of values these can be iterated over using the following tag:

<%for(loop_variable:array_variable>%> <%endfor%>

This will repeatedly apply the template contained within the open and close tags setting the loop_variable to each value in the array. Note this loop_variable may be an object.

Parts of the template can be optionally emitted based on the following tags

<%if(variable)%> <%else%> <%endif%>

If the specified variable is set then this outputs the first part otherwise the else part.

Example

For example – if the value of the Job’s data object was:

{ "clips": [ { "name": "clip 1" , "format": "mp4"},  { "name": "clip 2" , "format": "mp4"}, { "name": "clip 3" , "format": "mp4"} ], "result" : "Success" }

And the template from jobDataDesc was:

<h2>${result}</h2> <table width="100%" border="1"> <tr><th>Name</th><th>Format</th></tr> <%for(clip:clips)%> <tr><td>${clip.name}</td><td>${clip.format}</td></tr> <%endfor%> </table>

Then the result would be:

Success

Name

Format

clip 1

mp4

clip 2

mp4

clip 3

mp4