Overview

With the resolution of HC-522 - Getting issue details... STATUS , this added the capability of HySDS Core to optionally be able to support the use of Podman instead of Docker, depending on project needs. The implementation was largely influenced by the analysis that was done on the use of Podman within HySDS core: https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/2008350721/Analysis+of+Podman+Integration+into+HySDS+core#Testing%2FExperimenting-with-podman-using-vagrant-(Oracle-Linux-8-VM)

This wiki is intended to provide a guide on how projects can set up their cluster so that Podman can be used.

Setup

Update SDS Config

In the SDS config file (~/.sds/config), add the following settings:

CONTAINER_ENGINE - set it to ‘podman’
HOST_VERDI_HOME
VERDI_HOME
VERDI_SHELL

The following table describes these settings in a bit more detail. Note that if you do not specify these in the SDS config, the default values specified in this table will be used.

SETTING	DESCRIPTION	DEFAULT VALUE
CONTAINER_ENGINE	Tells the PCM system which container engine to use. Valid values are either ‘docker’ or ‘podman’	docker
HOST_VERDI_HOME	Tells the PCM system the base directory of where the code bundle will be deployed to on the workers. Default is $HOME. This value gets used when the celery worker starts up via supervisord. See the ‘celery worker queue’ service configuration in the supervisord.conf in a worker. This is also used to properly set the volume mounts in podman as podman seems to only recognize volume mounts at the host level than the container level.	$HOME
VERDI_HOME	Tells the PCM system where the HySDS code resides inside the Verdi container. Default is ‘/home/ops’. This ultimately gets used when the --passwd-entry flag option is set in the podman run commands. This is also used to properly set the volume mounts in podman as podman seems to only recognize volume mounts at the host level than the container level.	/home/ops
VERDI_SHELL	Tells the PCM system which shell to default to when the container is in use. Default is ‘/bin/bash’. This ultimately gets used when the --passwd-entry flag option is set in the podman run commands.	/bin/bash

CeleryConfig and install.sh Updates

Make sure to pull in the latest CeleryConfig.py and install.sh updates into your project repository:

CeleryConfig - Podman Settings

There are settings in the celeryconfig specific to Podman:

PODMAN_CFG = {
  "set_uid_gid": True,
  "set_passwd_entry": True,
  "cmd_base": {
    "userns": "keep-id",
    "security-opt": "label=disable"
  },
  "environment": [
    "HOST_VERDI_HOME",
    "HOST_USER"
  ]
}

Basically, the settings here is used to build out the podman run commands that HySDS core uses when they need to run a container.

By default, if you leave this alone, the following podman run command gets built out as the following:

$ podman --remote --url unix:/run/user/<user id>/podman/podman.sock run --init --rm \
-e HOST_VERDI_HOME -e HOST_USER \
-u $UID:$(id -g) \
--passwd-entry=<HOST_USER>:*:$UID:$(id -g)::<VERDI_HOME>:<VERDI_SHELL> \
--userns=keep-id \
--security-opt label=disable

Below is a table that describes these settings in more detail:

Setting	Description	Default Value
set_uid_gid	Sets the -u flag	-u $UID:$(id -g)
set_passwd_entry	Sets the --passwd-entry flag	--passwd-entry=ops::$UID:$(id -g)::/home/ops:/bin/bash*
cmd_base	Contains a list of flag options that can optionally be additionally modified based on desired need	--userns=keep-id --security-opt label=disable
environment	Contains a list of environment variables to be persisted onto the container. Assumes that the environment exists on the host machine.	HOST_VERDI HOST_HOME

At the time of this writing, projects can just use the default settings and modify over time depending on circumstances.

Update PGE Wrapper Code

Note that this section may be project-specific, but documenting it here because this is what SWOT and NISAR have currently implemented in the adaptation layer when running PGE software.

The following should be added to the PGE wrapper code:

        engine_value = app.conf.get("CONTAINER_ENGINE", "docker")
        container_engine = container_engine_factory(engine=engine_value)
        if isinstance(container_engine, Podman):
            container_engine.set_passwd_entry(False)

        cmd = container_engine.get_container_cmd(docker_params[dep_img_name], [rc_file])

What this code snippet does is that when running a PGE container through podman, it is found that we do not need to run it with the --passwd-entry flag option. Therefore what this code snippet does is:

Checks that the celeryconfig has CONTAINER_ENGINE=podman
If true, then do not set the --passwd-entry flag option when building out the podman run container for the PGE

Adding Support for Podman