...
Elastic Compute Cloud (EC2): Amazon’s web-interfaced cloud computing service
The Verdi Nodes are run by EC2
Autoscaling Groups: An EC2 feature that enables customizable scaling
HySDS allows you to specify settings for Autoscaling Groups
Security Groups: Define network settings and firewall on EC2 instances
Elastic Block Store (EBS): Provides block storage to be used with EC2 instances, allows you to choose which file system you want
Simple Cloud Storage Service (S3): Object store, primarily used to store files.
All data products (inputs and outputs) are stored in S3
PGE Code is stored in S3 buckets
Amazon Machine Image (AMI): Used to create virtual machine within EC2. HySDS uses 4 AMIs.
HySDS uses 4 AMIs
GRQ
Factotum
Metrics
Mozart
Elastic search: AWS allows you to integrate Elasticsearch in order to store metadata
All relevant metadata associated with jobs is stored in Elasticsearch. To access elasticsearch in HySDS, open the resource manager and press es:head in the navigation bar
Debugging
If the autoscaling groups are not working correctly, there is likely a problem with Mozart communication.
Autoscaling Group Structure
...