Chimera
- Namrata Malarout
- Topher Allen
Navigation: |
|---|
This is a workflow concept implemented using SciFLo. It is a basic skeleton for creating a workflows using SciFlo and generalized to be easily adapted by any project. SciFlo is the workflow infrastructure integrated with the HySDS framework.
This concept originated from work on SMAP in the Cloud. The chimera approach came about with our efforts to generalize how we could execute PGEs easily on every project. We recognized 3 general steps that we were implementing over and over again in different ways. So we decided to break them out into a mini 3 step workflow where each step is reusable and is configuration driven. Its development is being carried forward and reused by missions like SWOT and NISAR.
The common steps we identified in a PGE’s execution are:
For each of these steps we have created generic python scripts which are run in workflows.
The “Chimera concept“ is to use the 3 steps in order to execute a PGE. Now using SciFlo, you can choose to create:
Macro Workflows - A workflow executing multiple PGEs. SciFlo would handle the link between two PGEs.
Micro Workflows - A workflow executing only one PGE. We would use external HySDS features such as triggers to trigger a PGE based on its required input being ingested.
On the SMAP mission with HySDS, we have used macro workflows.
But on SWOT and NISAR missions, we are using micro workflows and triggers to connect them.
First place to start is by understanding Sciflo Basics
Contributors:
@Michael Cayanan
@Sujen Shah
@Namrata Malarout
@Frank Greguska
@Gerald Manipon