Page Navigation:

Page Navigation:
1 Intro 2 Queues 2.1 rabbitmq_queue_monitor.py 2.2 rabbitmq_queue_monitor_to_sdswatch.sh 3 Connections 3.1 rabbitmq_connection_monitor.py 3.2 rabbitmq_connection_monitor_to_sdswatch.sh

Confidence Level Moderate This article includes input from several JPLers. Multiple subject matter experts can indicate that a page is more frequently reviewed and updated.

Confidence Level Moderate This article includes input from several JPLers. Multiple subject matter experts can indicate that a page is more frequently reviewed and updated.

Intro

RabbitMQ has a web UI to see real-time information about queues, connections, and workers. This Repo: https://github.com/hysds/rabbitmq-utils has tools and examples for calling the RabbitMQ REST API to query for similar information.

Queues

rabbitmq_queue_monitor.py

This script calls RabbitMQ REST API to get a queue info. Uses request module, which has been tested up to python 3.7.

input arguments: --endpoint: the RabbitMQ API endpoint. e.g. "https://mozart.mycluster.hysds.io:15673" see http://e-jobs.aria.hysds.io:15672/api/ --username: username for rabbitmq --passwd: password for rabbitmq --queue: (optional) name of queue. if specified, it will only return info for this queue name. if not specified, then all queues will be shown. --interval: (optional) frequency of how often to check rabbitmq in unit seconds. (default=10)

outputs to stdout:

   timestamp, queue_name, queue_state, messages_ready, messages_unacknowledged

example usage of a specific queue:

$ ./rabbitmq_queue_monitor.py --endpoint="https://mozart.mycluster.hysds.io:15673" --username=guest --passwd=guest --queue=user_rules_dataset --interval=10

2020-05-21T22:37:29+00:00 user_rules_dataset running 0 0

example usage for all queues:

$ ./rabbitmq_queue_monitor.py --endpoint="https://mozart.mycluster.hysds.io:15673" --username=guest --passwd=guest --interval=10
2020-05-21T22:37:29+00:00 aria-job_worker-large running 1 0
2020-05-21T22:37:29+00:00 asf-job_worker-large running 0 0
2020-05-21T22:37:29+00:00 dataset_processed running 0 0
2020-05-21T22:37:29+00:00 factotum-create_aoi-queue running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-apihub_scraper_throttled running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-apihub_throttled running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-asf_throttled running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-global_ipf_scrape running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-large running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-realtime running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-scihub_throttled running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-small running 0 0
2020-05-21T22:37:29+00:00 factotum-job_worker-unavco_throttled running 0 0
2020-05-21T22:37:29+00:00 import_prov_es running 0 0
2020-05-21T22:37:29+00:00 ipf-scraper-scihub running 0 0
2020-05-21T22:37:29+00:00 jobs_processed running 0 0
2020-05-21T22:37:29+00:00 on_demand_dataset running 0 0
2020-05-21T22:37:29+00:00 on_demand_job running 0 0
2020-05-21T22:37:29+00:00 process_events_tasks running 0 0
2020-05-21T22:37:29+00:00 spyddder-sling-extract-asf running 1 0
2020-05-21T22:37:29+00:00 standard_product-s1gunw-topsapp-pleiades running 0 0
2020-05-21T22:37:29+00:00 system-jobs-queue running 0 0
2020-05-21T22:37:29+00:00 urgent-response-job_worker-large running 0 0
2020-05-21T22:37:29+00:00 urgent-response-job_worker-small running 0 0
2020-05-21T22:37:29+00:00 user_rules_dataset running 0 0
2020-05-21T22:37:29+00:00 user_rules_job running 0 0
2020-05-21T22:37:29+00:00 user_rules_trigger running 0 0

rabbitmq_queue_monitor_to_sdswatch.sh

This script calls the rabbitmq tool "rabbitmq_queue_monitor.py" to query the job queues (ignoring celery queues) and outputs to STDOUT log file in SDSWatch log format.

To use, update the settings in the script: RABBITMQ_API_ENDPOINT="https://mozart.mycluster.hysds.io:15673" RABBITMQ_USERNAME="username" RABBITMQ_PASSWD="mypasswd"

Example output:

The output can be redirected a rabbitmq.sdswatch.log file to be picked up by SDSWatch agent and sent to elasticsearch for analysis.

Connections

rabbitmq_connection_monitor.py

This script calls RabbitMQ REST API to get a connection info. Uses request module, which has been tested up to python 3.7.

input arguments: --endpoint: the RabbitMQ API endpoint. e.g. "https://mozart.mycluster.hysds.io:15673" see http://e-jobs.aria.hysds.io:15672/api/ --username: username for rabbitmq --passwd: password for rabbitmq --connection: (optional) name of connection. if specified, it will only return info for this queue name. if not specified, then all connections will be shown. format of this string is "127.0.0.1:46542 -> 127.0.0.1:5672" --interval: (optional) frequency of how often to check rabbitmq in unit seconds. (default=10)

outputs to stdout:

note that

the connection_name is: "client_host:client_port -> server_host:server_port"
send/receive rates are in B/s units.

example usage:

rabbitmq_connection_monitor_to_sdswatch.sh

This script calls the rabbitmq tool "rabbitmq_connection_monitor.py" to query the rabbitmq connection and outputs to STDOUT log file in SDSWatch log format.

To use, update the settings in the script: RABBITMQ_API_ENDPOINT="https://mozart.mycluster.hysds.io:15673" RABBITMQ_USERNAME="username" RABBITMQ_PASSWD="mypasswd"

example of running rabbitmq_connection-sdswatch.sh to stdout:

Related Articles:
Page: Beginner's Guide to HySDS Page: HySDS GUI's Overview Page: RabbitMQ Guide Page: RabbitMQ Utilities

Have Questions? Ask a HySDS Developer:

Anyone can join our public Slack channel to learn more about HySDS. JPL employees can join #HySDS-Community

JPLers can also ask HySDS questions at Stack Overflow Enterprise

Page Information:

Was this page useful?

Yes No

Contribution History:

Topher Allen (1557 days ago)
Hook Hua (1644 days ago)

Subject Matter Expert:

@Lan Dang

@Marjorie Lucas

@Hook Hua

Find an Error?

Is this document outdated or inaccurate? Please contact the assigned Page Maintainer: