SciFlo Job

As briefly mentioned before SciFlo is a workflow framework. It uses a XML workflow definition document to know the sequence of steps. Each step of a workflow is known as a process. The workflow definition documents defines the flow of the processes and how to run each process. More about the workflow definition documents can be found here: [placeholder]

SciFlo Bash Script

Any workflow job is executed by the run_sciflo.sh script.

It first sets up the BASE_PATH, PYTHONPATH and other environment requirements for the sciflo job.

It then reads in the inputs mentioned below:

1. Location to the adaptation module

2. Path to the directory containing the workflow definition document

3. Name of the Algorithm/Job/PGE to run

MODULE_PATH="$1"
WF_DIR="$2"
WF_NAME="$3"

It exports the adaptation module’s path to the python path so the scripts can resolve dependency imports.

export PYTHONPATH=${MODULE_PATH}:$PYTHONPATH

It constructs the location of the workflow definition document with WF_DIR and WF_NAME i.e. $WF_DIR/$WF_NAME.sf.xml

It runs the run_sciflo.py script with the required inputs.

python $BASE_PATH/run_sciflo.py $WF_DIR/$WF_NAME.sf.xml _context.json output > run_sciflo_$WF_NAME.log 2>&1
STATUS=$?
echo -n "Finished running $PGE run_sciflo.py: " 1>&2
date 1>&2
if [ $STATUS -ne 0 ]; then
  echo "Failed to run $PGE run_sciflo.py" 1>&2
  cat run_sciflo_$WF_NAME.log 1>&2
  echo "{}"
  exit $STATUS
fi

SciFlo Python Script

This is the python script that executes the sciflo job. It takes 2 positional arguments:

1. Workflow definition document - has the file extension .wf.xml

2. Job’s context file aka _context.json

3. Directory to write outputs of the job i.e. output