Page Navigation:

Table of Contents

Confidence Level High This article been formally reviewed and is signed off on by a relevant subject matter expert.

Agenda

develop -> develop-es1
run end-to-end smoke test to test develop-es1
create release v3.0.1 from develop-es1
run end-to-end smoke test to test v3.0.1
develop-es7 -> develop
run end-to-end smoke test to test new develop
create release v4.0.0-beta.0 from develop
run end-to-end smoke test to test v4.0.0-beta.0

develop -> develop-es1

hysds-dockerfiles
- create develop-es1 branch from develop and update CircleCI config
- follow what we did for develop-es7
- once these modifications have been made, the develop-es1 branch of hysds-dockerfiles will build the docker-es1 branches of the puppet modules
- next we need to create docker-es1 branches for the pertinent puppet modules
puppet modules
- puppet-verdi, puppet-mozart, puppet-metrics, puppet-grq, puppet-cont_int
  - develop → develop-es1
    - update develop-es1 branch, e.g.: https://github.com/hysds/puppet-verdi/compare/develop...develop-es1
  - docker → docker-es1
    - update docker-es1 branch, e.g.: https://github.com/hysds/puppet-verdi/compare/docker...docker-es1
    - this will install hysds framework based on develop-es1 branch which will build off of develop-es1 branch of core repos
hysds-framework
- develop → develop-es1
- update develop-es1, e.g.: https://github.com/hysds/hysds-framework/compare/develop...develop-es1
create develop-es1 branches for HySDS core repos (see “All repos updated” section below)
push CircleCI build to build the docker images for develop-es1 and deploy them up to dockerhub
- Since CircleCI is having issues this morning, we will be reusing the hysds/verdi:develop docker image, renaming it to hysds/verdi:develop-es1, and uploading it as a release asset so that the end-to-end smoke test downloads this docker image

Run end-to-end smoke test to test develop-es1

Push nisar-pcm jenkins job that tests end-to-end functionality

end-to-end succeeded

Code Block

null_resource.mozart (remote-exec): + cat /tmp/datasets.txt
null_resource.mozart (remote-exec): SUCCESS: Found 66 expected v1.0,2.0,d0001,1 datasets.
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/datasets.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/check_stamped_dataset_result.txt
null_resource.mozart (remote-exec): SUCCESS: Found daac_delivery_status in datasets ['hello_world-product-20200330T191839.415236163Z-0d8e3', 'hello_world-product-20200330T191556.097105247Z-62d17'].
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/check_stamped_dataset_result.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/cop_catalog.txt
null_resource.mozart (remote-exec): SUCCESS: Found 229 expected number of documents in the cop_catalog index
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/cop_catalog.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/rost_catalog.txt
null_resource.mozart (remote-exec): SUCCESS: Found 4 expected number of documents in the rost_catalog index
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/rost_catalog.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/check_empty_isl_result.txt
null_resource.mozart (remote-exec): SUCCESS: No files exist in the ISL Bucket: nisar-dev-isl-fwd-gmanipon++ grep -c '^SUCCESS' /tmp/check_empty_isl_result.txt
null_resource.mozart (remote-exec): + SUCCESS=1

create release v3.0.1 from develop-es1

https://github.com/hysds/hysds-framework/releases/tag/v3.0.1
awaiting creation and publishing of v3.0.` verdi docker image on circleci: https://app.circleci.com/pipelines/github/hysds/hysds-framework/102/workflows/84ca3118-9185-410c-b77d-20548e346905/jobs/198
- published: https://github.com/hysds/hysds-framework/releases/tag/v3.0.1
if a new release was created for lightweight-jobs, ensure that hysds-verdi-latest.tar.gz and docker-registry-2.tar.gz was published as assets

run end-to-end smoke test to test v3.0.1

https://nisar-pcm-ci.jpl.nasa.gov/job/force-branches/job/gmanipon-force_branch-E2E-test/24/console

Code Block

null_resource.mozart (remote-exec): SUCCESS: Found 66 expected v1.0,2.0,d0001,1 datasets.
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/datasets.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/check_stamped_dataset_result.txt
null_resource.mozart (remote-exec): SUCCESS: Found daac_delivery_status in datasets ['hello_world-product-20200330T233623.680399832Z-33ff7', 'hello_world-product-20200330T233854.951246808Z-64c08'].
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/check_stamped_dataset_result.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/cop_catalog.txt
null_resource.mozart (remote-exec): SUCCESS: Found 229 expected number of documents in the cop_catalog index
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/cop_catalog.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/rost_catalog.txt
null_resource.mozart (remote-exec): SUCCESS: Found 4 expected number of documents in the rost_catalog index
null_resource.mozart (remote-exec): ++ grep -c '^SUCCESS' /tmp/rost_catalog.txt
null_resource.mozart (remote-exec): + SUCCESS=1
null_resource.mozart (remote-exec): + [[ 1 -eq 0 ]]
null_resource.mozart (remote-exec): + cat /tmp/check_empty_isl_result.txt
null_resource.mozart (remote-exec): SUCCESS: No files exist in the ISL Bucket: nisar-dev-isl-fwd-gmanipon++ grep -c '^SUCCESS' /tmp/check_empty_isl_result.txt
null_resource.mozart (remote-exec): + SUCCESS=1

update nightly jobs for hysds_release v3.0.0 → v3.0.1
- https://nisar-pcm-ci.jpl.nasa.gov/job/nightlies/job/E2E-nisar-pcm-develop-baseline-pge-hysds-release/
- https://nisar-pcm-ci.jpl.nasa.gov/job/nightlies/job/E2E-nisar-pcm-develop-hysds-release/
- https://nisar-pcm-ci.jpl.nasa.gov/job/nightlies/job/E2E-nisar-pcm-develop-real-pge/

develop-es7 -> develop

puppet modules

create annotated tags to bookmark the last ES1.7 commit:

Code Block
git checkout develop-es1 git tag -am "Last ES1.7 commit." es1 git checkout docker-es1 git tag -am "Last ES1.7 commit." es1-docker git push --tags

update es7 branches so that explicit clones of develop-es7 and docker-es7 are reverted back to develop and docker, respectively.

merge develop-es7 into develop:

Code Block
git checkout develop git pull . develop-es7 git push git branch -D develop-es7 git push --delete origin develop-es7

Then:

Code Block
git checkout docker git pull . docker-es7 git push git branch -D docker-es7 git push --delete origin docker-es7

Finally pull develop into master:

Code Block
git checkout master git pull . develop git push

All Repos updated

hysds https://github.com/hysds/hysds.git
- PR: https://github.com/hysds/hysds/pull/52
hysds-framework
- PR: https://github.com/hysds/hysds-framework/pull/15
container-builder https://github.com/hysds/container-builder.git
- PR: https://github.com/hysds/container-builder/pull/12
Rest APIs
- grq2 https://github.com/hysds/grq2.git
  - PR: https://github.com/hysds/grq2/pull/18
- mozart https://github.com/hysds/mozart.git
  - PR: https://github.com/hysds/mozart/pull/13
Good for now, may move to hysds repo
- hysds_commons https://github.com/hysds/hysds_commons.git
  - https://github.com/hysds/hysds_commons/pull/21
Will remove because code and scripts ahve been moved to grq/mozart repo
- figaro https://github.com/hysds/figaro.git
  - https://github.com/hysds/figaro/pull/15
- tosca https://github.com/hysds/tosca.git
  - PR: https://github.com/hysds/tosca/pull/18
sdscli https://github.com/sdskit/sdscli.git
- PR: https://github.com/sdskit/sdscli/pull/46
lightweight-jobs https://github.com/hysds/lightweight-jobs.git
- PR: https://github.com/hysds/lightweight-jobs/pull/14
Other repos that are installed in every hysds component
- prov_es https://github.com/hysds/prov_es.git
- chimera https://github.com/hysds/chimera.git
  - PR: https://github.com/hysds/chimera/pull/2
- pele https://github.com/hysds/pele.git
  - PR: https://github.com/hysds/pele/pull/6

📖 Related Articles:

Filter by label (Content by label)

showLabels	false
max	12
sort	title
showSpace	false
cql	label = "job_managementdevelopment"

Have Questions? Ask a HySDS Developer:

Anyone can join our public Slack channelto learn more about HySDS. JPL employees can join #HySDS-Community

JPLers can also ask HySDS questions atStack Overflow Enterprise

Live Search

placeholder	Search HySDS Wiki

🚀 Page Information:

Was this page useful?

Yes No

Contribution History:

Contributors

mode	list
showLastTime	true
order	update

Subject Matter Expert:

Gerald Manipon

Find an Error?

Is this document outdated or inaccurate? Please contact the assigned Page Maintainer:

Gerald Manipon

Versions Compared

Old Version 17

New Version Current

Key

Agenda

develop -> develop-es1

Run end-to-end smoke test to test develop-es1

create release v3.0.1 from develop-es1

run end-to-end smoke test to test v3.0.1

develop-es7 -> develop

All Repos updated

Page Comparison

Versions Compared

Old Version 17

New Version Current

Key

Agenda

develop -> develop-es1

Run end-to-end smoke test to test develop-es1

create release v3.0.1 from develop-es1

run end-to-end smoke test to test v3.0.1

develop-es7 -> develop

All Repos updated