(TODO: Note - JIRA #597: How to facet on all failed jobs with the same failure message, purge them and then re-submit them. Related to JIRA #601)
Question: what are all of the possible precondition failures? (this is mentioned in Jira 601).
(v2 revisions, 4/16/20, begin below)
Faceting On & Purging Similar Failed Jobs:
Navigate to the Resource Manager (Figaro).
Inside Figaro, facet on “job-failed” in the left-hand column under the status menu (See #1 in image).
Narrow the scope to a specific, unique job failure type by selecting the targeted message in the left-hand column under the error column (#2). {{INSERT UPDATED SCREENSHOTS}}
Note: Multiple job types can share an error message. Users can confirm the faceted error message belongs to only one job type by checking the “type” menu in the left-hand column.
4. After the similar failed jobs have been faceted on, click the “On Demand” button (#3).
5. In the On-Demand window, add a unique key phrase in the “Tag” text box (#4). This helps facilitate job tracking and management.
6. Select “Purge jobs” from the action drop-down menu (#5). {{INSERT SCREENSHOTS}}
7. Modify the remaining job parameters as needed. Click “Process Now” (#6).
Re-Submitting Purged Jobs:
7. After the failed jobs have been purged, remove any previously applied facets in Figaro (#7).
8. Using the key phrase created in step #5, enter the user-defined tag in quotations in the search box (#8). In this example “purge_tag_test” is used. This displays all the failed (and now purged) similar jobs.
9. Click the “On-Demand”, add another unique tag {{example ____ in this example}} (#9) to track the retried jobs. Next, select “Retry Jobs/Tasks” from the action drop-down menu (#10). Last, click “Process Now”. This retries the failed, purged jobs.
10. To facet on the newly retried jobs, remove the “purge_tag_test” facet and add the unique tag created in the previous step {{example _____ }} to facet on the newly retried similar jobs.