2019-04-08 Meeting notes

Date

Apr 8, 2019 at 12pm Pacific Time

WebEx info: https://jpl.webex.com/jpl/j.php?MTID=m47a5e7153b3354edcf4422d50aee463f

Call in: +1-510-210-8882 USA Toll (Access code: 906 603 518)

Collaborative meeting notes are at https://docs.google.com/document/d/1E_sw-_9IHMtAtqfbr0Dy_d8Ci2e3ehYxo5EtinGL710/edit#heading=h.sjbyvaasxmmz

Participants

• @Sujen Shah

• @Hook Hua

Goals

Discussion topics

Goals of these meetings ?

Notes:

Update from Hook

Notes:

Change control process ?

• Share/Summarize community process slides that was previously reviewed with NISAR, SWOT, and 398 section line

• Code reviews?

• Change Control Board process?

  • Do we need a defined CCB?

• Are we encouraging contributions from the public?

• Resources for CI/CD and “smoke testing”

  • Circle CI free-tier

Notes:

Configuration Management

• Consolidate hysds-core tickets (currently scattered)

  • If we’re settled on using JIRA, we should disable GitHub issues to reduce confusion

  • https://github.jpl.nasa.gov/hysds-org/general/issues

  • https://hysds-core.atlassian.net/secure/Dashboard.jspa

  • https://github.com/hysds/hysds-framework/issues

• Jenkins instance for CI/CD?

• “Smoke testing” for hysds-core?

• Workflow for bugs/feature requests?

• Git strategy

  • fork + PR, branch + PR, etc… ?

  • What is the meaning of master branch?

  • https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows#ch05-distributed-git

Notes:

Documentation

• Contribution Guidelines

• HySDS Community Wiki

  • Update intro page

    • HySDS Community Wiki landing page cannot be edited for some reason?

  • Access to comment on the wiki pages

    • https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/17989633/HySDS+Community+Wiki

  • Discuss use of wiki vs. versioned documentation (e.g. github pages, read the docs)

• Consolidate wikis (currently scattered)

  • https://wiki.jpl.nasa.gov/display/hysds/HySDS+Community

  • https://hysds-core.atlassian.net/wiki/spaces/HYS/pages/17989633/HySDS+Community+Wiki

  • https://github.com/hysds/hysds-framework/wiki

• Atlassian wiki markdown macro is broken

• Google Season of Docs -- similar to Summer of Code, but aimed at technical documentation. Google provides stipend to both open source project and technical writer

  • https://opensource.googleblog.com/2019/03/introducing-season-of-docs.html

  • Need 2 volunteers to act as mentors and handle registration if we want to pursue this

Notes

Communication

• Do we want a HySDS developer mailing list?

• Migrate use of #hysds-dev to be general channel?

Notes

Tutorials/Training?

• Aimee Barciauskas at devseed, working on the MAAP project, deployed hysds independently and also made a video tutorial for it. She posted it on Vimeo (now made private)

• How to develop in Containers?

• Hello World for live tutorial

• sdscli overview

Notes

Workflows

• Missions need to skip steps in a linear sciflo workflow for start/stop capability.

  • It may mean that the workflow needs first-class support for conditional and looping constructs. We can either add support for these features to sciflo or go with one out there that already has it.

  • Several workflows similar to sciflo that are lightweight and run in-memory and provide those features.

    • https://workflow.readthedocs.io/en/latest/

      • It also integrates well with celery.

        • The celery example is pretty straight forward and would integrate easily with hysds core:

        • https://workflow.readthedocs.io/en/latest/#celery-support

    • Airflow

    • Toil

      • supports WDL which is another standard

      • can be a bit more heavyweight but do come with some out-of-the-box web interfaces that would benefit operators.

• OGC Testbed 15 Kickoff for Common Workflow Language

  • https://www.commonwl.org/

Notes:

Use of docker registry to register/run jobs

Notes:

Should we participate in Google Season of Docs. (Deadline 4/23)

Notes:

Consolidation or organization of documentation

Notes:

SWH: CM issues when trying to move from DEV delivery to INT deployment. Current issue is with AutoScaling Groups, but reflects that we haven’t yet established a robust process to progress from DEV to INT to PROD

Notes:

On premise LOM and communication with it ?

Notes:

HySDS component sitting on premise to listen to job completion.

Notes:

Timers ?

Notes:

Action items

Decisions

1. Use Jira for issue tracking and disable github issues