Get job type specification

type

• id: a job type id from jobspec::list

• Full job specification in JSON format

• returns the jobspec JSON

Get supported job types

list

none

• list of job type ids

• This is a list of the supported processing steps that a workflow can orchestrate.

• A workflow modeler could use this listing to feed the palette selection of supported processing steps.

• The name is used for display purposes in usage of jobspec::list.

• Full specifications use the jobspec::type call

Create a job specification

add

• spec: JSON specification of job

• job_spec_id

• Specification JSON documented here: Job and HySDS-IO Specifications

Remove a job specification

remove

• id: jobspec id

Get hysds-io specification

type

• id: a job type id from hysdsio::list

• Full job specification in JSON format

• returns the hysds-io JSON specification

Get supported hysds-io specs

list

none

• list of job type ids

• This is a list of the supported processing input-to-param wirings

• A workflow submitter could use this listing to feed inputs in standard HySDS format (ES entries) into the job's parameters.

• The name is used for display purposes in usage of hysdsio::list.

• Full specifications use the hysdsio::type call

Create a hysds-io specification

add

• spec: JSON specification of hysds-io

• job_spec_id

• Specification JSON documented here: Job and HySDS-IO Specifications

Remove a hysds-io specification

remove

• id: hysds-io id

Gets the list of registered containers.

list

none

• List of all registered container ids

Registers a new container

add

• name

• url

• version

• container_id

• container names shall be unique, which includes version

Removes a container

remove

• container_id

Gets more detailed information for a container

info

• container_id

• a json of the container description

Gets the allowed queues to send jobs to.

list

none

• List of all queue names that the jobs can be submitted to.

• Each queue will be backed by an auto-scaling group of workers.

Submits a new job to RM

submit

• type: a job type from jobspec/list

• queue: name of queue for this job to be submitted to. from queue/list
  
  

• priority: the relative priority of the job. default=0 (optional)
  
  

• tags: list in JSON format of machine job tags (optional)
  
  

• context: the input parameter set context for this job

• job_id (uuid)

• Priority follows the documented job priority specification

• The given job type is used internally to look up the corresponding Container that it is registered with.

• localize_urls is inside context

Gets the status of the job state.

status

• job_id

• job state from set:

  • job-queued

  • job-started

  • job-failed

  • job-completed

  • job-offline

• This method is intended to be light-weight to call to frequently in polling for checking of job states.

• The job state transitions are documented in the job state transition page.

Gets the complete job information

info

• job_id

• dict of:

  • job name

  • job id

  • job type

  • context

  • status

  • queue

  • timestamp

  • worker

  • priority

  • work dir url

  • tags

  • exit code

• Additional details are in the ES doc and work dir url

• The original context is stored to facilitate job retries.

Gets a list of all jobs in RM

list

• page_size: jobs per page (for pagination of result set) default=100

• offset: starting job offset in result set (for pagination of result set) default=0
  
  

• list of job ids

• There may be millions of jobs returned. Therefore it is good to require a pagination of the result set.

• Results is limited to just enable quick listing jobs with sufficient for minimal management

• For more detailed information, can call job::info(jobid) to get more details.

(Pending Implementation) Stops a running job.

stop

• job_id

• Revokes a queued and running job.

• Stopped job would still be in system.

• Stopping a job that has completed, failed, or offline has no affect.

(Pending Implementation) Purges a job from the RM

purge

• job_id

• Stops and removes a job from the RM

(Pending Implementation) Adds a user tag to a job

user_tag_add

• job_id

• user_tag: the tag to associate with a given job

•  Ensure that this call does not clobber entire job state in backend. e.g. for ES, use doc_as_upsert for updating only the tag field.
  
  

(Pending Implementation) Removes a user tag from a job

user_tag_remove

• job_id

• user_tag: the tag to remove from a given job

•  Ensure that this call does not clobber entire job state in backend. e.g. for ES, use doc_as_upsert for updating only the tag field.