Verdi

Verdi is composed of workers that pull jobs from the queue and executes them. Once the jobs are completed, the finished products are sent to a storage repository. Additionally, the product metadata is indexed in Elasticsearch.

Note: a “compute fleet” refers to an army of Verdi running in compute workers.

Mozart

Mozart is a resource management service which orchestrates job management across the HySDS hybrid cloud computing architecture.

It manages job queues (RabbitMQ), and job metadata (Elasticsearch). Mozart also hosts information about job status, and the GUI for viewing job status using a faceted view (Figaro).

Factotum

Factotum consists of special workers which handle specific processes that don’t generate data products. This includes non-scaling jobs in the AWS cloud.

GRQ (Geo Region Query)

GRQ catalogs product metadata in Elasticsearch indices. It provides the GeoRegionQuery (GRQ) REST API for publishing and querying the metadata. It also hosts the product faceted view GUI (Tosca) for viewing published products.

Metrics

Displays worker and job metrics. It hosts the Elasticsearch indices for real-time job and worker metrics.

Continuous Integration (CI)

GitHub is checked for updates to containers and publishes them when found.