Step 5: Running your First "Hello World" Job
Confidence Level TBD This article has not been reviewed for accuracy, timeliness, or completeness. Check that this information is valid before acting on it. |
---|
*With the OPS_USER and OPS_PASSWORD, you can use the https (e.g. https:// ) protocol to gain access to the url.
Integrating a "Hello World" Job Type
For definitions of terminology used, please refer to our terminology reference.
When installation (see Installation) and setup of your HySDS cluster (see Cluster Setup) is complete, you can start integrating job types (PGE or non-PGE) into the system. In this tutorial, we integrate our first job type which will simply echo "Hello World".
On the
mozart
instance, create a directory for your new job typecd ~/mozart/ops
mkdir hello_world
cd hello_world
Let's create the script that runs our job type. Open up a new file
and paste the contents below
and save.
Set the script to be executable
chmod 755 run_hello_world.sh
Test the script by running it
./run_hello_world.sh
Hello World Hello World to STDERR
Now let's create the necessary files for integrating this job into your HySDS cluster. Create the
docker
directory and cd into itmkdir docker
cd docker
Create a Dockerfile for your job type by opening up a new file
Dockerfile.<job_type_name>
vi Dockerfile.hello_world
paste the following into it
and save.
Create a
job-spec
file for your job type by opening up a new filejob-spec.json.<job_type_name>
paste the following into it
and save. For more information on the
job-spec
format, check out Job and HySDS IO Specifications.Create a
hysds-io
file for your job type by opening up a new filehysds-io.json.<job_type_name>
paste the following into it
and save. For more information on the
hysds-io
format, check out Job and HySDS IO Specifications.Change directory back up to the root of your job type directory
and the layout of the contents should be as follows
Now we create a git repository from this directory
Create a new repository on github, copy the repo URL, and push your existing hello_world repository. In this example, I created a new repo under my github user account, https://github.com/pymonger/hello_world.git
Ensure that your HySDS cluster is up and running
If it is not running, run
sds start all
.Now we need to configure our new job type for continuous integration and deployment. Since we're still developing this job type, we'll configure it using the master branch instead of release tags
If jenkins needs to use an OAuth token to access the repo, specify the
-k
option and ensure thatGIT_OAUTH_TOKEN
was specified in$HOME/.sds/config
:You should have output similar to the following
Follow these instructions to configure the webhook on github so that it can trigger rebuild and deployment.
Note: If you run into issues like :
you may have a jenkins-cli.jar version that does not support -http, -auth parameters. You can either get a latest jenkins-cli.jar, or can make the following change line in cluster.py to make it work:
to:
If the build fails with following error (aws encryption type) :
remove the encryption type value as AES256 is default type in cluster.py:
to:
if your build fails with import error for python-pyasn1 or rsa:
you may not have latest version of the packages. use pip to install them (specially in ci and factotum instances):
yum may not install the latest version of pyasn1_modules.
Navigate to your
ci
instance (e.g. http://<your-ci-instance>:8080) to validate that the jenkins job was configuredYou should see your jenkins job named as "container-builder_<your repo name>_<branch>" (e.g. container-builder_gmanipon_hello_world_master).
Verify that the jenkins job runs to completion by manually scheduling a build. The jenkins job will build the docker image, publish it to your S3 code bucket, and register the job types into you HySDS cluster. Click on the green arrow and view the console output to validate.
Verify that your new job type was published to your HySDS cluster. Open up a browser and point it to your
mozart
instance's ElasticSearch head plugin (e.g. http://<your-mozart-instance>:9200/_plugin/head)*Alternatively, you can query for the the "Hello World" job-spec using curl
Now let's try to run our hello_world job type using the Dataset Faceted Search interface (aka
tosca
) running on thegrq
instance. First, let's ingest a dataset so that we have at least 1 dataset in our catalog. On themozart
instance runNavigate to your
grq
instance (e.g. http://<your-grq-instance>/search)* to validate that the AOI dataset was ingestedNext click on the "On-Demand" button. If you don't see the "On-Demand" button, then the AOI dataset wasn't ingested correctly. Otherwise, a modal titled "On-Demand (Process Results)" should pop up as below
Enter the value "test" into the "Tag" field, select "Hello World [master]" for the "Action" field, and select "job_worker-small" for the "Queue" field (should already be selected because it is the "recommended-queue" based on our job-spec.json.hello_world that we composed earlier). You can leave the "Priority" field at "0" for now
Click on "Process Now".
You will then get another modal providing you with a link to the HySDS Resource Management interface (aka
figaro
) for the job you just submitted. Click on the link to open upfigaro
. If the link doesn't open up correctly, there may be a routing issue with the private vs. public IP address in thetosca
configuration. Alternately, navigate to yourmozart
instance (e.g. http://<your-mozart-instance>/figaro)* to view your job. Click on the "Queued" button to facet down to the jobs that are in a "job-queued" stateFor advanced users, navigate to the RabbitMQ Admin interface (e.g. http://<your-mozart-instance>:15672/#/queues) running on your
mozart
instance to view the queued jobs and tasksIt will be stuck in the
job-queued
because there are no workers configured to process jobs from the "job_worker-small" queue.Let's configure your
factotum
instance to start up a worker that will pull jobs from the "job_worker-small" queue. What we will do is copy the default supervisord.conf template for factotum to~/.sds/files
, and modify it to include start up ofcelery
workers for thejob_worker-small
queue:Edit the
supervisord.conf.factotum
fileappend the following lines to the bottom of the file and save
Since there are no jobs running in your cluster, we can safely update the factotum instance
Check the status of
supervisord
on factotumWe're ready to update if it shows
Finally update and start up factotum:
Rerun
sds status factotum
to verify that your new worker forjob_worker-small
isRUNNING
Navigate to your
mozart
instance (e.g. http://<your-mozart-instance>/figaro)* to view your job again. Click on the "Started" button to facet down to the jobs that are in a "job-started" stateWhen the job is finished executing, you should have a job that is in a "job-completed" state
Click on the "work directory" link to view the live work directory on factotum. All STDOUT output will go into the
_stdout.txt
file and all STDERR output will go into the_stderr.txt
. Here's what_stdout.txt
should showand what the
_stderr.txt
should showNote:
If you run into issues like:
it could be due to permission issue in aws. Check the permission tab in 'grfn-v2-ops-product-bucket' to ensure you have the same configuration, specially, the policy file (dont forget to edit the bucket name in your policy file). Also download the index.html file from there and add to your bucket (do necessary edits to reflect your bucket)
That's it! Congratulations, you've integrated a new job type into your HySDS cluster and successfully executed it.
Next Steps
Now that you've integrated your first job type, continue on to Step 5: Integrating a "Hello World" Dataset Type to integrate a dataset produced by this job type.
Related Articles: |
---|
Have Questions? Ask a HySDS Developer: |
Anyone can join our public Slack channel to learn more about HySDS. JPL employees can join #HySDS-Community
|
JPLers can also ask HySDS questions at Stack Overflow Enterprise
|
Page Information: |
---|
Was this page useful? |
Contribution History:
|
Subject Matter Expert: @Gerald Manipon @Hook Hua |
Find an Error? Is this document outdated or inaccurate? Please contact the assigned Page Maintainer: @Lan Dang |