Create Auto-Scaling Fleet Queue

This guide describes how to create an auto-scaling fleet queue within HySDS.

1. Log into the Mozart instance and navigate to the sds config file.

   vi ~/.sds/config

2. Add an entry for your new auto-scaling queue within the config file following this format:
   -QUEUE_NAME: <YOUR_QUEUE_NAME>
   INSTANCE_TYPES: -<DESIRED_AWS_INSTANCE_TYPES>
   
   e.g.

   - QUEUE_NAME: aria-standard_product-enumerator INSTANCE_TYPES: - r4.8xlarge - r4.16xlarge - r5a.large

3. Stop the Mozart instance. Note that active jobs will be lost.

4. Update the Mozart instance. This will rewrite the supervisord.conf on Mozart to include the syncer
scripts for your new queues/ASGs so that autoscaling can kick in.

5. Propagate custom settings.

6. Creates code-config bundles for all ASG’s.

7. Restart Mozart.

8. Create the ASG.

9. Go to the AWS console and confirm the ASG was created by selecting the ASG dashboard. Your queue should appear on this dashboard. Make sure to set the desired and maximum number of instances for this ASG.

Troubleshooting: Reference your ASG sync log for more insight:

e.g.

Want to run all the mozart commands at once?

Just add && between the commands.

How do I make sure this gets populated to Tosca?

Often, you will have followed these instructions creating a queue (or queues) for a specific PGE. When you build a PGE through Jenkins using the HySDS infrastructure (and specify “recommended-queues” e.g. here), the queue will populate in Tosca.

However, sometimes we want to experiment with queue settings and do not want to modify the PGE itself. For example, we may want to explore instance types in the spot market. To ensure that the queue shows up on Tosca and RabbitMQ, make sure to add the queue directly using RabbitMQ. See the figure below after following the creation steps above.

Open