...
Elastic Compute Cloud (EC2): Amazon’s web-interfaced cloud computing service. Resizable compute capacity. Allows you to utilize your OS to run custom applications. EC2 is scalable, allows you to pay only for what you use, and deploy across different availability zones.
The Verdi Nodes are run by EC2
Autoscaling Groups: An EC2 feature that enables customizable scaling
HySDS allows you to specify settings for Autoscaling Groups
Security Groups: Define network settings and firewall on EC2 instances
Elastic Block Store (EBS): Provides block level storage to be used with EC2 instances, allows you to choose which file system you want.
Simple Cloud Storage Service (S3): Object store, primarily used to store files.
All data products (inputs and outputs) are stored in S3
PGE Code is stored in S3 buckets
Snapshots stored in S3
Amazon Machine Image (AMI): Used to create virtual machine within EC2. It contains a snapshot (OS, Application, Application Server), launch permissions, etc. HySDS uses 4 AMIs.:
HySDS uses 4 AMIs
GRQ
Factotum
Metrics
Mozart
Elastic search: AWS allows you to integrate Elasticsearch in order to store metadata
All relevant metadata associated with jobs is stored in Elasticsearch. To access elasticsearch in HySDS, open the resource manager and press es:head in the navigation bar
...