When installation of the HySDS framework is complete on your mozart instance (see Installation), we must configure the rest of the cluster instances so that they can talk to each other. We do this using the sdscli command on the mozart instance. The idea is that all code and configuration is centralized on the mozart instance and when ready to deploy updates during the development cycle or when upgrading operations, we can push them easily from a single location.

1. Configure your cluster parameters using sdscli: The sdscli repo was installed on your mozart instance during Installation. Configure your cluster by running:

   cd

   ~

   source ~/mozart/bin/activate sds configure

2. The sds configure command will prompt you for your cluster parameters. A description of the parameters with examples is provided below

...

1. Make sure elasticsearch is up on the mozart and grq instances. You can run the following command to check:

   curl 'http://<mozart/grq ip>:9200/?pretty'
   

   you should get answer back from ES, something like this:

   {
    "status" : 200,
    "name" : "Dweller-in-Darkness",
    "cluster_name" : "resource_cluster",
    "version" : {
      "number" : "1.7.3",
      "build_hash" : "05d4530971ef0ea46d0f4fa6ee64dbc8df659682",
      "build_timestamp" : "2015-10-15T09:14:17Z",
      "build_snapshot" : false,
      "lucene_version" : "4.10.4"
    },
    "tagline" : "You Know, for Search"
   }
   
   

   If you can not connect to elastic search, you need to start ElasticSearch in mozart and grq instances:

   sudo systemctl start elasticsearch
   

2. Ensure mozart component can connect to other components over ssh using the configured KEY_FILENAME. If correctly configured, the sds status all command should show that it was able to ssh into each component to check that the supervisord daemon was not running like below:

   sds status all
   ########################################
   grq
   ########################################
   [100.64.106.214] Executing task 'status'
   Supervisord is not running on grq.
   ########################################
   mozart
   ########################################
   [100.64.106.38] Executing task 'status'
   Supervisord is not running on mozart.
   ########################################
   metrics
   ########################################
   [100.64.106.140] Executing task 'status'
   Supervisord is not running on metrics.
   ########################################
   factotum
   ########################################
   [100.64.106.64] Executing task 'status'
   Supervisord is not running on factotum.
   ########################################
   ci
   ########################################
   [100.64.106.220] Executing task 'status'
   Supervisord is not running on ci.
   ########################################
   verdi
   ########################################
   [100.64.106.220] Executing task 'status'
   Supervisord is not running on verdi.
   

   Otherwise if any of the components show the following error, for example for the grq component:

   ########################################
   grq
   ########################################
   [100.64.106.214] Executing task 'status'
   
   Fatal error: Needed to prompt for a connection or sudo password (host: 100.64.106.214), but abort-on-prompts was set to True
   
   Aborting.
   Needed to prompt for a connection or sudo password (host: 100.64.106.214), but abort-on-prompts was set to True
   
   Fatal error: One or more hosts failed while executing task 'status'
   
   Aborting.
   One or more hosts failed while executing task 'status'
   

   then there is an issue with the configured KEY_FILENAME on mozart or the authorized_keys file under the component's ~/.ssh directory for user OPS_USER. Resolve this issue before continuing on.

3. Update all HySDS components:

   sds update all
   

   If you receive any errors, they will need to be addressed.

4. Start up all HySDS components:

   sds start all
   

5. View status of HySDS components and services:

   sds status all
   

6. During installation, the latest versions of the lightweight-jobs core HySDS package and the verdi docker image was downloaded. Next we import the lightweight-jobs package:

   cd ~/mozart/pkgs
   sds pkg import container-hysds_lightweight-jobs.*.sdspkg.tar
   

7. Finally we copy the verdi docker image to the code bucket (CODE_BUCKET as specified during sds configure). Ensure VERDI_PRIMER_IMAGE url is consistent:

   aws s3 cp hysds-verdi-latest.tar.gz s3://<CODE_BUCKET>/hysds-verdi-latest.tar.gz

   
   

Next Step

Now that you have your HySDS cluster configured, continue on to /wiki/spaces/v3/pages/28803122