RabbitMQ Guide


Confidence Level High  This article been formally reviewed and is signed off on by a relevant subject matter expert. 

Confidence Level High  This article been formally reviewed and is signed off on by a relevant subject matter expert. 

RabbitMQ Overview

RabbitMQ is used in combination with Celery to queue jobs. Logging into RabbitMQ can give you a broader view on the state of the system. Some examples where RabbitMQ can be useful is in determining why a job has been queued for a long time, determining if a job is in the wrong queue, or receiving real-time status of jobs. While Mozart can show the status of a job, it determines this through the data in elasticsearch. If for any reason elasticsearch wasn’t updated properly, Mozart may give you an incorrect status.

Logging in to RabbitMQ

The credentials for RabbitMQ will likely be located in a .netrc file in the Mozart Machine

Using RabbitMQ

  1. Go to the Queues Tab

  2. Type ^(?!celery) under Filter and check Regex to rid of celery related queues.

Messages

  • Ready: Jobs are in the queue ready to be run by a worker

  • Unacked: A worker started a job but hasn’t necessarily finished. There is often a timeout in the case that a job remains in an unacked state for a set time period.

  • Total:

State

  • Running: A worker is listening to the queue

  • Idle: A worker is not listening to the queue. May be due to network and connectivity issues or because the queue doesn’t actually exist.

Using RabbitMQ for Debugging

  • When jobs are in a Ready state but not an Unacked state, it can mean that the Autoscaling Group isn’t working correctly, or that a worker isn’t listening in on the queue.


Related Articles:

Have Questions? Ask a HySDS Developer:

Anyone can join our public Slack channel to learn more about HySDS. JPL employees can join #HySDS-Community

JPLers can also ask HySDS questions at Stack Overflow Enterprise

Search HySDS Wiki

Page Information:

Was this page useful?

Yes No

Contribution History:

Subject Matter Expert:

@Lan Dang

Find an Error?

Is this document outdated or inaccurate? Please contact the assigned Page Maintainer:

@Lan Dang

Note: JPL employees can also get answers to HySDS questions at Stack Overflow Enterprise: