	Docker	Singularity	Podman
Runs as	root-level daemon	user-space daemon	user-space and daemonless
Platforms supported	AWS, GCP, Azure, desktops, linux; basically broad support	many HPC environments (e.g. NASA Pleiades), linux	currently linux
K8 Support	Yes	N/A	Yes
Python SDK support	Yes - docker-py	Python API	podman-py only the Rest API
HySDS implementation	Yes	No (not yet)	No (not yet)
Run container in container (container-ception)	Yes - mount `docker.sock` file	N/A - (Not sure if possible)	Yes - with `--privileged` flag
resource usage statistics	using shim - cgroups	??	??

Development Effort

1 sprint = 2 weeks * 1 FTE
Update core HySDS code (job_worker.py, container_utils.py, etc.) - 2 sprints minimum
- Core code is due for an upgrade
- To support all 3 container engines without separation of code would require a large refactor
- Break up large functions into smaller more readable functions
  - Maybe use use object-oriented design to de-couple the container engines
  - Parent class branches off into Docker Podman and Singularity with a factory function
Update the verdi code tarball - ~1 sprint
Update the docker images (running the CircleCI job) - ~0.5 sprint
Updating the PCM code (for NISAR, SWOT, etc.) and get them up to speed with the changes to HySDS core - 1-2 sprint minimum
Integration of podman - 1-2 sprint minimum
- Revisiting HySDS core code for integration of podman logic in the job execution
- Testing for full run through of cluster deployment and end-to-end test with podman running all container jobs

...

Versions Compared