Containerization is defined as a form of operating system virtualization, through which applications are run in isolated user spaces called containers, all using the same shared operating system (OS). A container is essentially a fully packaged and portable computing environment.

We are looking into supporting 3 different container engines: Docker (current), Podman and Singularity. But ideally HySDS should be able to support all 3 engines without separation of code (multiple branches)

Docker

The most popular container engine (Do I have to say more?)

HySDS currently runs its jobs through docker.

• https://www.docker.com/

Singularity

SingularityCE is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible

• https://singularity.lbl.gov/

• https://sylabs.io/docs/

Podman

• https://podman.io/

Podman is a daemonless container engine for developing, managing, and running OCI Containers on your Linux System. Containers can either be run as root or in rootless mode. Simply put: alias docker=podman.

It behaves similarly to Docker, the commands are mostly the same:

• podman images - lists all images in host repository

• podman ps - lists all running containers

• podman run - run image in container (all flags work as well, ie. --rm -it --net=host -v, etc.)

Running Podman in Podman

Because Podman is daemon-less, there isn’t a podman or docker .sock file that can be mounted into the running container, instead:

images are stored in the /var/lib/containers directory when Podman is run by the root user. For standard users, images are typically stored in $HOME/.local/share/containers/storage/.

by mounting /var/lib/containers to the container it will give it access to the host (or parent) images

Buildah

Buildah can be used as an alternative to podman to build images

• https://buildah.io/

You can either build using a Dockerfile using podman build or you can run a container and make lots of changes and then commit those changes to a new image tag. Buildah can be described as a superset of commands related to creating and managing container images and, therefore, it has much finer-grained control over images. Podman’s build command contains a subset of the Buildah functionality. It uses the same code as Buildah for building.

Why move away from Docker?

According to Redhat:

Kubernetes 1.20, support of the Docker container engine is deprecated, but users will still be able to use Docker container images and registries, as well as create containers that look identical at runtime .… it will always support the OCI and Docker image formats.

Docker requires a daemon to be run on every host and by default it will run the daemon as root, which can potentially cause a security vulnerability

Development Effort

• 1 sprint = 2 weeks * 1 FTE

• Update core HySDS code (job_worker.py, container_utils.py, etc.) - 2 sprints minimum

  • Core code is due for an upgrade

  • To support all 3 container engines without separation of code would require a large refactor

  • Break up large functions into smaller more readable functions

    • Maybe use use object-oriented design to de-couple the container engines

    • Parent class branches off into Docker Podman and Singularity with a factory function

• Update the verdi code tarball - ~1 sprint

• Update the docker images (running the CircleCI job) - ~0.5 sprint

• Updating the PCM code (for NISAR, SWOT, etc.) and get them up to speed with the changes to HySDS core - 1-2 sprint minimum

• Integration of podman - 1-2 sprint minimum

  • Revisiting HySDS core code for integration of podman logic in the job execution

  • Testing for full run through of cluster deployment and end-to-end test with podman running all container jobs

Related documentation:

• https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/1972895745