HySDS Ops Doc Development Table-Spring '20
- Stirling Algermissen
- Topher Allen
- Lela Rodriguez
Cells with blue background represent input by ARIA/GRFN.
Purple = in progress or ready for feedback
Red = too mission specific, removed
Gold = Needs clarification
Blue = completed, signed-off
count | priority | topic | Status | wiki link | Jira link | comments |
|---|---|---|---|---|---|---|
| I'd like to see the How-To's placeholder on the wiki filled in with: |
|
|
|
| |
1 | 1 | Jira #593: How to track Jobs (thru the system) --nominal flow | 5/12 received tag-up clarification, starting v1 5/14 Reviewed with Marjorie, received more input, revising v2. v2 updated, new image created 6/9 signed off | https://jira.jpl.nasa.gov/browse/NSDS-593
| nominal flow from task to job to …(some of this is probably generic and some mission specific mlucas: approved | |
2 | 1 | JIRA #594: How to manage Jobs (seeing, stopping, removing, re-submitting, following) (aka “lightweight” jobs”) 5/5 TAG UP CLARIFICATION:
| 4/28 Revised v2 6/9 revised, possible updates needed | would like to see more info on tracking the job thru different states in the system mlucas: | ||
3 | 1 | JIRA #595: How to purge all jobs by a time range (See #601, #597) Potentially implemented as a how-to page 6/2 tag up: likely fits in a Step-by-step guide. The key is using the ES query for a timerange. Marjorie is going to follow-up with the correct query string
| 5/14 reviewed with Marjorie, she’s going to follow up 6/9 see query string. Implement as a cheatsheet that includes the timestamp query <-- |
|
| |
4 | 1 | Jira #598 How to identify gaps in data 6/2 probably too mission specific. | 6/2 confirm this is too mission specific with Lela 6/9 Lela agrees. |
|
|
|
5 | 1 | JIRA #597: How to facet on all failed jobs with the same failure message, purge them and then re-submit them (Related to #601) 6/9 After tag up with Marjorie, added How to Retry and How to Revoke stub articles | 4/16 received v2 feedback 4/30 updating based off feedback 6/9 added two articles --> 6/9 pending comment clean-up, otherwise signed-of |
https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/593395739 |
| |
6 | 1 | How to know the list of all the log files involved with HySDS and the top 3 that operators should become familiar with (**Overlap with ARIA/GRFN request below) 5/5 TAG UP CLARIFICATION:
| 6/9 Marjorie will provide me with location of the logs, the types of logs, description of each. (Note: not necessary to list/describe contents of each log file) 6/11 completed v1 based on log files/descriptions provided by Marjorie |
|
| metadata is indeed useful to know about but topic was for HySDS “log” files NSDS-796-ljr |
7 | 1 | JIRA #599: How to know when operators need to use potentially different guis... like celery, rabbitmq, aws and kibana {6/2 And Tosca and Figaro} | 4/30 Started v1, finished minimally viable v1, ready for feedback 5/5 incorporated initial feedback re “how”, added additional images. No SWOT queue, AWS access. v1.5 in progress 6/4 updated with additional GUIs 6/9 Marjorie will fill in blanks with remaining GUI’s 6/11 updated Tosca and Figaro content. | https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/492142629/HySDS+GUI+s+Overview
| looking for the “why” operator would need to access different guis, not just the “how” | |
8 | 1 | JIRA #600: How to tell which products have not been produced yet 5/5 TAG UP CLARIFICATION:
| Confirmed in tag up, Too mission specific |
| Check w Lela for clarification; this might be mission specific(?) | |
9 | 1 | JIRA #601: How to filter on failed jobs due to precondition failure (Note: precondition failure “is when a PGE fails not because of a runtime error but due to missing inputs, corrupt inputs, etc. These show up as failed jobs… Not something easily visible. These aren’t necessarily distinguished from a failed job”) | 5/5 pending v2 revisions
6/9 Marjorie will get more clarification from HySDS team (Gerald, Namrata) | LR: added comment on wiki page link | ||
10 | 1 | How to determine latency between receipt of raw telemetry and production of pipeline products | 4/30 received feedback on this from Namrata; likely too mission specific |
|
| LR: sign off concur too mission specific |
11 | 2 | JIRA #603: How to query Elastic search catalog by filename, ingest time and get a tabular search result 5/5 TAG UP CLARIFICATION:
|
| would like the example expanded to include the GRQ es (not just the mozart one) | ||
12 | 2 | JIRA #604: (Elasticsearch) Like to know how to see all the different indexes and what all can be queried 5/5 TAG UP CLARIFICATION:
| 6/4 reviewed with Marjorie
6/9 revised article, linked to stub article outlining tabular search results --> 6/9 TODO: append webex screenshot and add comment with alternate method Done. | Navigating the Elasticsearch Plug-in head Database
https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/591101963 | add search results image and add structured query example with how to get the tabular results | |
13 | 3 | JIRA #605: Start/stop/restart/status/reset and supervisord role up next | 4/30 pending tag up w Namrata for 5/1 5/10 v1 done, pending tag-up w Marjorie for revisions v2 revised, image added 6/9 reviewed, signed-off |
| ||
14 | 3 | JIRA #606: how to find and use any/all PCM config files (settings.yaml) | 4/30 received feedback on this from Namrata; likely too mission specific |
| LR: sign off concur too mission specific | |
15 |
| JIRA #607: Create/edit/delete trigger rules | 4/28 Finished v1, ready for feedback 5/5 incorporated remaining feedback. Done.. | LR: sign off
| ||
16 |
| JIRA #608: Submit on-demand jobs via Figaro and Tosca | 4/14 Finished v1 (Figaro) 4/10 Finished v1 (Tosca) 4/28 Revised, ready for sign-off | LR: sign off | ||
17 |
| **Description of product log file types (e.g. _context.json, _met.json, _dataset.json) | 4/14 Finished v1 4/28 Revised, ready for sign-off… 5/5 Done. |
| LR: sign off
| |
18 |
| JIRA #610: Diagram of interfaces between HySDS components showing data flow | 4/26 Finished L1 + L2 v1 4/28 Ready for feedback 5/5 working on Marjorie’s feedback 5/10 Uploaded v2 L1 & L2 with revisions, feedback received, working on v3 L1 v3 finished; made text searchable per feedback 6/4 Updated L1 and L2, added CI to L2 6/9 signed-off |
|
| |
19 |
| JIRA #612: Troubleshoot frequently-seen issues 6/2 TAG UP CLARIFICATION: (FYI) the intent is to have a place where people can write up solutions to frequently seen problems in HySDS core. | 5/14 tag up with Marjorie, she’s going to follow up with me with specific examples 6/2 Deemed lower priority per Marjorie 6/9 Needs discussion regarding best implementation for HySDS community |
|
| |
20 |
| JIRA #613: View AWS running instances, buckets, and ASG’s (auto scaling groups) | 4/28 Lan tag-up, working on v1 5/1 Pending AWS access; doesn’t look like I have it. 5/10 Finished v1, Ready for feedback 5/14 Awaiting feedback from Marjorie on v1 6/9 signed-off | View AWS Running Instances, Buckets, and Auto Scaling Groups |
| |
21 |
| JIRA #614: Basic manipulation of AWS instances, buckets, and ASG’s via CLI and GUI Clarification: how to stop an instance, how to terminate an instance, creating a bucket, also describe how to get an IP of an EC2 instance. For ASG’s: knowing how to change desired and maximum capacity. | 5/1 Pending AWS access; doesn’t look like I have it. 6/2 working on v1 6/4 finished v1 6/9 signed-off | https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/575045633 |
| |
22 |
| Define resource types within job management 5/5 TAG UP CLARIFICATION: | 4/28 need clarification (task, job, event, worker, etc) 6/9 incomplete |
|
|
|
23 |
| JIRA #616: Send accessible links of faceted content
| 4/11 finished v2, done. | https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/395083799 | LR: sign off | |
24 |
| Incorporate generic Mozart trigger rules into Operations Docs as appropriate. Link to generic rules on public wiki. surgically delete an entry (exists on old internal documentation) update to public
| 6/2 reviewed with Marjorie, request is tentatively satisfied with linked doc → 6/9 signed-off |
| Lan’s suggestion via Slack | |
Hierarchy of HySDS Operator Manual (current 6/9/20):
Jira dashboard: https://jira.jpl.nasa.gov/secure/Dashboard.jspa#
Have Questions? Ask a HySDS Developer: |
Anyone can join our public Slack channel to learn more about HySDS. JPL employees can join #HySDS-Community  |
JPLers can also ask HySDS questions at Stack Overflow Enterprise |