...

$ docker images | grep python3.9
hysds/cuda-pge-base                                      python3.9             ############        20 hours ago        3.84GB
hysds/pge-base                                           python3.9             ############        20 hours ago        3.41GB
hysds/verdi                                              python3.9             ############        20 hours ago        3.25GB
hysds/cuda-dev                                           python3.9             ############        21 hours ago        4.92GB
hysds/dev                                                python3.9             ############        21 hours ago        3.15GB
hysds/cuda-base                                          python3.9             ############        21 hours ago        3.44GB
hysds/base                                               python3.9             ############        21 hours ago        2.85GB

able to confirm python 3.9 running in all docker images

if you want to test your PGEs with the hysds/pge-base:python3.9 docker image you can SSH into CI and export the image with docker save and import it with docker load

You can build the docker images yourself if you don’t want to export/import image elsewhere, you would have to run https://github.com/hysds/puppet-verdi/blob/HC-385/build_docker.sh (w/ branch HC-385) in your CI instance

• warning: it can take a while to build the images & there are a lot of steps as well (some missing from here)

• example Circle CI build taking 2+ hrs

cd puppet-verdi/
$ ./build_docker.sh python3.9

will build the images for with Dockerfile, Dockerfile.pge-base, Dockerfile.cuda-pge-base

able to confirm python 3.9 running in all docker images

$ docker run -it hysds/base:python3.9
ops@3951c4870953:~$ python --version
Python 3.9.5

$ docker run -it hysds/dev:python3.9
ops@102228433e3b:~$ python --version
Python 3.9.5

$ docker run -it hysds/cuda-base:python3.9
ops@66324a20369b:~$ python --version
Python 3.9.5

$ docker run -it hysds/cuda-dev:python3.9
ops@22cecadebfe2:~$ python --version
Python 3.9.5

$ docker run -it hysds/verdi:python3.9 bash
ssh-keygen: generating new host keys: RSA1 RSA DSA ECDSA ED25519 
ops@0625c883075b:~$ python --version
Python 3.9.5

$ docker run -it hysds/cuda-pge-base:python3.9 bash
pythoops@666f74ab8432:~$ python --version
Python 3.9.5

$ docker run -it hysds/pge-base:python3.9 bash
ops@16a9019779f0:~$ python --version
Python 3.9.5

...

• https://github.jpl.nasa.gov/IEMS-SDS/pcm-releaser/compare/NSDS-1555

• https://github.jpl.nasa.gov/IEMS-SDS/nisar-pcm/compare/NSDS-1555

• https://github.jpl.nasa.gov/IEMS-SDS/CNM_product_delivery/tree/NSDS-1555

A kluge was added to both pcm-releaser and puppet-hysds_base to allow the provisioner to install the latest version of miniconda which uses python 3.9

• https://github.jpl.nasa.gov/IEMS-SDS/pcm-releaser/compare/NSDS-1555

• https://github.com/hysds/puppet-hysds_base/compare/HC-385

Testing

Was able to run a dev-e2e (both fwd and reprocessing for NISAR), the output logs are attached in the NSDS-1555 ticket

• all dataset checks passed

• no jobs failed

docker images built for nisar-pcm and CNM_product_delivery

• after fixing the versions for dependencies

No core functionality broke:

• Rest APIs working (mozart + GRQ2)

• celery workers are functioning

  • orchestrator

  • user rules processor (for datasets and jobs)

• lightweight-jobs was tested with the following jobs working:

  • purge dataset

  • purge job

  • echo

  • revoke job

  • retry job

  • AWS get