Skip to content

Tasks

Tasks are the primary workhorse of AO's backend.

Task

Task Types

Docker (task:docker)

Docker tasks will be run within a docker container and use stdin and stdout to provide input and capture output. This allows users to use any environment necessary to run analysis and perform tasks.

API (task:api)

A thin wrapper to provide an easy way to integrate custom API calls into AO's system.

Task Detail

The task detail page provides the capability to create and edit tasks. Tasks are instatiated with an event and create a task run

Task Selector

The task selector is responsible for determining where and when tasks can be run.

  • type filter: Task runner will attempt to prefix match on this type

    ex. asset:gen will allow this task to show up for all asset:gen types and their children

  • task contract: Used to filter tasks in projects or scenarios. Certain steps will require a specific contract and this location denotes which contract it fulfills. This will be given more robust support in the future.

    ex. tc:resource:solar requires inputs lat, lng, and will output a resource file

  • ignore type: optionally ignore certain tasks for the task runner

  • queue: which queue to place the task into

Task Parameters

Task parameters allow you to add custom parameters to tasks, perform any mapping on event objects, and provide validation details for events.

Parameter Config

Parameter config is used to configure custom parameters to be used within your task.

This parameter config adds a parameter for min and max acres used in a parcel search

acre_max:
  formtype: number
  key: acre_max
  name: Max Acres
  order: 20
acre_min:
  formtype: number
  key: acre_min
  name: Min Acres
  order: 10

Parameter Map

The parameter map is used to transform the event object before it is passed to the task.

Transform a data object to sync down to workspace

The following example adds a new field to the event object under the property ao_syncDown.

ao_syncDown is an array of item refs that are to be downloaded and stored on the workspace with a given filename. An example array item is

{
    rn: 'team/<team>/file/<fileid>',
    filename: 'test.json'
}

The maps are applied from top to bottom, so the bottom have highest preference and will overwrite previous values.

This example has 3 possible names for the file, a default file name, the name of the data item, and finally a filename specified directly on the event object

The data property map will pass through the data key from the event input

data:
  value: data
syncDownFileA:
  fixed: default-name.data
  key: 'ao_syncDown[0].filename'
syncDownFileB:
  key: 'ao_syncDown[0].filename'
  value: data.parameter.name.name
syncDownFileC:
  key: 'ao_syncDown[0].filename'
  value: filename
syncDownRn:
  key: 'ao_syncDown[0].rn'
  value: data.rn

Validator Config

Validator config is used to determine whether a task is ready to be run.

Validate parameters for either a radius search or a wkt_search

This example has two options it can validate. If either succeeds, the task can be run.

The first looks to ensure there is a lat, lng, and buffer that is used as a radius

The second looks for a wkt geometry and a buffer.

radius_search:
  checks:
    - get: lat
      name: Latitude
    - get: lng
      name: Longitude
    - get: buffer
      name: Buffer
  description: 'Require Lat,Lng and Radius'
  name: Radius Search
wkt_search:
  checks:
    - get: wkt
      name: WKT Geometry
    - get: buffer
      name: Buffer
  description: Require WKT Geometry
  name: WKT Geometry Search

Environment Variables

Custom environment variables can be added to the docker container and used within the arguments for starting the container.

Rest API

Used to configure API calls

  • url: Url for the call
  • checkurl: An endpoint to query before calling the endpoint to check for specific information
  • querystringmap: A config block to map the input event to the query string

This is only found for type task:api

This will be updated to be more versatile in the near future

Runtime

The runtime describes the environment in which your task is run and is mapped to Docker input.

  • arguments: The arguments to pass into the entry point
  • entry point: Entry point into your script
  • docker image: Docker image to use for task
  • docker options: Options to add to docker

This is only found for type task:docker

Docker options is not implemented currently for security concerns

Workspace

The workspace determines which filesystem is mounted and accessible through docker.

  • filter type: Used to filter tasks. Should be in task selector parameters
  • workspace: Which workspace to mount.

This is only found for type task:docker

File

This is a temporary parameter that allows for an easy way to change and update files for a given task. Eventually the file system will be better integrated into the task developer.

  • filename: Filename to store file
  • contents: Python file contents

This is only found for type task:docker

Temporary parameter that is to be deprecated by 2019 pipeline

Event

This is a temporary parameter that allows for an easy test event for a given task. Eventually there will be a way to save, copy, and list custom events for use in building tasks.

  • data: JSON event

Temporary parameter that is to be deprecated by 2019 pipeline

Sample Events

This sample event will download a specific file item to the workspace and save as test.bmp

{
  "ao_syncDown": [
    {
      "filename": "test.bmp",
      "rn": "team/demo-v3/file/-LHK7ldtBTKc5fqIe-zQ"
    }
  ]
}

Notes

Notes can be provided in markdown in the parameter group Notes. It will be formatted and displayed in the task summary.

  • notes: notes provided as markdown

Input

Task Detail Notes Input

Parsed to HTML

Task Detail Notes

Task Run

Task runs are an instatiation of a task with an event.

Output

Output of a task run.

The output object can be used to create items within AO. Currently, the task manager can create file, data, asset, and project types.

Create a project with the task manager

An output ref will be created upon succesful creation of the item

{
"project": [
      {
        "tid": "demo-v3",
        "item": {
            <project item>
        }
      }
    ]
}

Manager Logs

Container Logs

Output

Output Refs

When items are created from a task, an output ref is created that links to the new item.