Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

reference: https://www.nas.nasa.gov/hecc/support/kb/commonly-used-pbs-commands_174.html

Short-circuiting job

...

worker if not enough time remaining to completed the next

...

job

Recommendations for running on Pleiades:

...

At the beginning of a PBS job wrapper script (invoked by verdi job worker), check if the job has sufficient time allocation remaining to complete the max estimated duration of a job. This ensures that the running job has at least X hours left in the PBS job for the node running the job, and exiting the verdi job worker if the PBS job does not have sufficient time remaining to complete the estimated job duration.

Can run qstat -f $PBS_JOBID and compare the output for resources_used.walltime and Resources_List.walltime to see how much time remains. If insufficient time, then the pbs wrapper script can sigterm gracefully the PID of verdi job worker to trigger verdi job worker to exit the process and therefore end the PBS node.

Lei.Pan@jpl.nasa.gov (Unlicensed) TODO

...