RabbitMQ Guide
Page Navigation: |
---|
Confidence Level High This article been formally reviewed and is signed off on by a relevant subject matter expert. |
---|
RabbitMQ Overview
RabbitMQ is used in combination with Celery to queue jobs. Logging into RabbitMQ can give you a broader view on the state of the system. Some examples where RabbitMQ can be useful is in determining why a job has been queued for a long time, determining if a job is in the wrong queue, or receiving real-time status of jobs. While Mozart can show the status of a job, it determines this through the data in elasticsearch. If for any reason elasticsearch wasn’t updated properly, Mozart may give you an incorrect status.
Logging in to RabbitMQ
The credentials for RabbitMQ will likely be located in a .netrc file in the Mozart Machine
Using RabbitMQ
Go to the Queues Tab
Type ^(?!celery) under Filter and check Regex to rid of celery related queues.
Messages
Ready: Jobs are in the queue ready to be run by a worker
Unacked: A worker started a job but hasn’t necessarily finished. There is often a timeout in the case that a job remains in an unacked state for a set time period.
Total:
State
Running: A worker is listening to the queue
Idle: A worker is not listening to the queue. May be due to network and connectivity issues or because the queue doesn’t actually exist.
Using RabbitMQ for Debugging
When jobs are in a Ready state but not an Unacked state, it can mean that the Autoscaling Group isn’t working correctly, or that a worker isn’t listening in on the queue.
Related Articles: |
---|
Have Questions? Ask a HySDS Developer: |
Anyone can join our public Slack channel to learn more about HySDS. JPL employees can join #HySDS-Community
|
JPLers can also ask HySDS questions at Stack Overflow Enterprise
|
Page Information: |
---|
Was this page useful? |
Contribution History:
|
Subject Matter Expert: @Lan Dang |
Find an Error? Is this document outdated or inaccurate? Please contact the assigned Page Maintainer: @Lan Dang |