Merge pull request #41 from QGreenland-Net/dev-standup-notes-2024-05-06

Add dev standup notes for 2024-05-06

interfaces/recipe-author-to-recipe-repo.qmd

@@ -55,7 +55,7 @@ linkStyle 0,1 stroke:#ff6666,stroke-width:2px;
 # WARNING: This is not a real shell script, although it is meant to look and
 # feel like one. Each line, unless it starts with #, is interpreted as a
 # workflow step. To test, use the `ogdc-workflow-test` program.
-gdalwarp -t_srs EPSG:3413 {input} {output} 
+gdalwarp -t_srs EPSG:3413 {input} {output}
 gdal_transform ... {output} {input}
 ```
 

notes/2024-03-25_dev-standup.md

@@ -12,7 +12,7 @@ date: "2024-03-25"
 
 ## Discussion
 
-### pangeo-forge 
+### pangeo-forge
 
 * Relationship to goals of OGDC - geoharmonization/transformation of data into standard
   formats.

notes/2024-03-26_ndcc-workshop.md

@@ -27,15 +27,15 @@ using cloud-computing resources (e.g., permafrost landscape analysis in the
 Arctic). Discussions ranged from long-term visions of a cloud-based
 computational environment that users interact with through AI natural-language
 interfaces to what we can do as a disparate group of individually-funded
-projects in the next 6-18 months. 
+projects in the next 6-18 months.
 
 The NDC-C is a 2-year pilot that aims to lay the foundations for an NDC that
 will cater to a wider science community.
 
 There were many discussions about AI, and comparisons between the NDC-C and the
 [National Artificial Intelligence Research Resource
 Pilot](https://new.nsf.gov/focus-areas/artificial-intelligence/nairr) were
-highlighted by the NSF reps in attendance. 
+highlighted by the NSF reps in attendance.
 
 Two NSF representatives were present at the meeting, [Marlon
 Pierce](https://www.nsf.gov/staff/staff_bio.jsp?lan=mpierce&org=OAC&from_org=OAC)
@@ -64,7 +64,7 @@ There were two guided discussions facilitated by the workshop organizers:
 * A conversation with Elizabeth Simmons, Executive Vice Chancellor, UC San
   Diego, about how the collective impact framework has been successfully
   utilized by UC San Diego.
-  
+
 * A panel discussion about equity and education in climate research. Panelists
   were:
   * Olivia Graeve, professor at UC San Diego

notes/2024-04-04_pangeo-forge.md

@@ -82,9 +82,9 @@ using PF, and all have their own use cases and infra. Needs.
       an “STP” which is a template which allows you to package all info about how to transform
       a legacy format into a new output format and generate STAC compliant metadata.
     * STP is a place like PF where people can coordinate on upstream data format issues and
-      share best practices. 
+      share best practices.
     * Can create an STP for an individual dataset to fix up issues.
-    * DevSeed orchestrates STP at scale in different ways. 
+    * DevSeed orchestrates STP at scale in different ways.
           * https://github.com/developmentseed/stactools-pipelines - for AWS
 
 ### Questions / needs from QGreenland-Net
@@ -110,17 +110,17 @@ using PF, and all have their own use cases and infra. Needs.
   provides those recipes optionally?
     * Geospatial-specialized PTransforms
     * Generic subprocess-calling PTransform
-    * E.g. https://pangeo-forge.readthedocs.io/en/latest/ecosystem.html 
+    * E.g. https://pangeo-forge.readthedocs.io/en/latest/ecosystem.html
 * QGreenland-Net is funding Matt to work on Openscapes. How can we take advantage of
   Openscapes specialization in cross-DAAC collaboration to help understand and meet
   NASA/DAAC needs and bring in collaborators to move faster?
 * If we do decide to integrate with PF, Aimee says “yes, we can integrate and can
   contribute!” Join Pangeo Forge coordination meetings! Might be on a hiatus now, in the
   air. Next months will be disruptive because the approach is more of a do-ocracy with
-  less centralized governance. 
+  less centralized governance.
 * Aimee’s roadmap for governance is a WIP. More work needs to be done to communicate the
   new vision for PF.
-* Look at Aimee’s cloud optimized formats guide: https://guide.cloudnativegeo.org/ 
+* Look at Aimee’s cloud optimized formats guide: https://guide.cloudnativegeo.org/
 
 
 

notes/2024-04-11_dev-standup.md

@@ -27,7 +27,7 @@ date: "2024-04-11"
 
 * Matt and Trey were exploring how to deploy/setup Apache Spark on k8s, so that we could test running Apache beam workflows.
   Got stuck on configuring Spark. Documentation is dense, a LOT to learn.
-    * Spark originally designed for JVM-based langs. Python, Go, others tacked on later. We want to focus on accessibility, we should pick a lang. Python makes sense as a focus. 
+    * Spark originally designed for JVM-based langs. Python, Go, others tacked on later. We want to focus on accessibility, we should pick a lang. Python makes sense as a focus.
 * Is ADC transitioning from Ray.io -> parsl? Why?
   Are they used for different scenarios?
   * [viz-workflow](https://github.com/PermafrostDiscoveryGateway/viz-workflow/tree/main) README:
@@ -46,5 +46,3 @@ date: "2024-04-11"
 
 - [ ] Rushiraj: Ask Juliet for opinions on Parsl vs. Ray
 - [ ] Continue exploring Parsl
-
-

notes/2024-04-17_tech-meeting.md

@@ -76,9 +76,4 @@ date: "2024-04-17"
 - [ ] Matt J add Matt, Trey, Rushiraj to PDG Slack
 - [ ] Schedule meeting to discuss quarterly goals. What are our milestones?
   - [ ] Matt J to share calendar w/ Matt F & Trey
-- [ ] 
-
-
-
-
-
+- [ ]

notes/2024-05-06_dev-standup.md

@@ -0,0 +1,41 @@
+---
+title: "2024-05-06: Dev standup"
+date: "2024-05-06"
+---
+
+## Participants
+
+* Trey
+* Rushiraj
+
+
+## Activities review
+
+* Rushiraj:
+    * Continuing work on Cesium map
+    * Took gpkg from QGreenland, using PDG pipeline code to get formatted into
+      web tiles for cesium.
+* Trey:
+    * Matt and I started focusing on developing our core set of functionality
+      for OGDC ([see this
+      issue](https://github.com/QGreenland-Net/qgreenland-net.github.io/issues/40)). PR
+      with our initial work
+      [here](https://github.com/QGreenland-Net/qgreenland-net.github.io/pull/39).
+    * Also began to refine criteria and do assessment of different workflow
+      technologies ([see this
+      issue](https://github.com/QGreenland-Net/.github/issues/26)). Evaluation
+      spreadsheet can be found
+      [here](https://docs.google.com/spreadsheets/d/1f8CgOu1wXnaoAfEODrCkxNfY-Xq_as1AJPrPVK74lM0/edit#gid=0).
+        * Did an initial assessment of parsl, which we feel has some issues that
+          other tools might already have solutions for (e.g., logging bash app
+          commands, pods not getting cleaned up in some error states, etc.)
+        * In last hour of last week, we explored use of
+          [argo](https://argo-workflows.readthedocs.io/en/latest/). We were able
+          to configure argo and reproduce our seal-tags workflow in that hour.
+          We were very impressed with argo's user-experience, which includes a
+          monitoring dashboard and a CLI that provides a straightforward way of
+          watching/monitoring jobs without needing to use `kubectl` commands
+          directly. See our spike repo
+          [here](https://github.com/QGreenland-Net/argo-spike)
+    * Trey now at 25% time for QGreenland-Net, which will go down to 0 at the
+      end of this month. Will be focusing on other project work this week.

presentations/ndcc-overview.md

@@ -20,7 +20,7 @@ author:
 
 ::: {.notes}
 Workshop focused on defining a vision for the NSF's activities supporting the
-creation of a more integrated cloud-based system for climate science. 
+creation of a more integrated cloud-based system for climate science.
 
 The NDC-C is a 2-year pilot that aims to lay the foundations for an NDC that
 will cater to a wider science community.
@@ -61,7 +61,7 @@ projects in the next 6-18 months.
 
 * Reccomendations for the NSF NDC-C via a workshop report
 * Slides describing each  project
-* Establishment of actual and potential collaborations 
+* Establishment of actual and potential collaborations
 
 
 #