Page Comparison

This guide describes how to create an auto-scaling fleet queue within HySDS.

1. Log into the Mozart instance and navigate to the sds config file.

   Code Block
   vi ~/.sds/config

2. Add an entry for your new auto-scaling queue within the config file following this format:
   -QUEUE_NAME: <YOUR_QUEUE_NAME>
   INSTANCE_TYPES: -<DESIRED_AWS_INSTANCE_TYPES>
   
   e.g.

   Code Block
   - QUEUE_NAME: aria-standard_product-enumerator INSTANCE_TYPES: - r4.8xlarge - r4.16xlarge - r5a.large

3. Stop the Mozart instance. Note that active jobs will be lost.

sds stop mozart -f

4. Update the Mozart instance. This will rewrite the supervisord.conf on Mozart to include the syncer
scripts for your new queues/ASGs so that autoscaling can kick in.

sds update mozart -f

5. Propagate custom settings.

fab -f ~/.sds/cluster.py -R mozart,factotum update_aria_packages

6. Creates code-config bundles for all ASG’s.

sds ship

7. Restart Mozart.

sds start mozart -f

8. Create the ASG.

sds cloud asg create

9. Go to the AWS console and confirm the ASG was created by selecting the ASG dashboard. Your queue should appear on this dashboard. Make sure to set the desired and maximum number of instances for this ASG.

Troubleshooting: Reference your ASG sync log for more insight:

e.g.

tail -f ~/mozart/log/sync_ec2_target_tracking_metric-aria-standard_product-enumerator.log

Want to run all the mozart commands at once?

Just add && between the commands.

sds stop mozart -f && sds update mozart -f && fab -f ~/.sds/cluster.py -R mozart,factotum update_aria_packages && sds ship && sds start mozart -f && sds cloud asg create

How do I make sure this gets populated to Tosca?

Often, you will have followed these instructions creating a queue (or queues) for a specific PGE. When you build a PGE through Jenkins using the HySDS infrastructure (and specify “recommended-queues” e.g. here), the queue will populate in Tosca.

However, sometimes we want to experiment with queue settings and do not want to modify the PGE itself. For example, we may want to explore instance types in the spot market. To ensure that the queue shows up on Tosca and RabbitMQ, make sure to add the queue directly using RabbitMQ. See the figure below after following the creation steps above.

Image Added