diff --git a/interfaces/recipe-author-to-recipe-repo.qmd b/interfaces/recipe-author-to-recipe-repo.qmd index deb15a6..95d6d79 100644 --- a/interfaces/recipe-author-to-recipe-repo.qmd +++ b/interfaces/recipe-author-to-recipe-repo.qmd @@ -55,7 +55,7 @@ linkStyle 0,1 stroke:#ff6666,stroke-width:2px; # WARNING: This is not a real shell script, although it is meant to look and # feel like one. Each line, unless it starts with #, is interpreted as a # workflow step. To test, use the `ogdc-workflow-test` program. -gdalwarp -t_srs EPSG:3413 {input} {output} +gdalwarp -t_srs EPSG:3413 {input} {output} gdal_transform ... {output} {input} ``` diff --git a/notes/2024-03-25_dev-standup.md b/notes/2024-03-25_dev-standup.md index b8c914d..0b1c5cc 100644 --- a/notes/2024-03-25_dev-standup.md +++ b/notes/2024-03-25_dev-standup.md @@ -12,7 +12,7 @@ date: "2024-03-25" ## Discussion -### pangeo-forge +### pangeo-forge * Relationship to goals of OGDC - geoharmonization/transformation of data into standard formats. diff --git a/notes/2024-03-26_ndcc-workshop.md b/notes/2024-03-26_ndcc-workshop.md index d77c5ee..0056c0b 100644 --- a/notes/2024-03-26_ndcc-workshop.md +++ b/notes/2024-03-26_ndcc-workshop.md @@ -27,7 +27,7 @@ using cloud-computing resources (e.g., permafrost landscape analysis in the Arctic). Discussions ranged from long-term visions of a cloud-based computational environment that users interact with through AI natural-language interfaces to what we can do as a disparate group of individually-funded -projects in the next 6-18 months. +projects in the next 6-18 months. The NDC-C is a 2-year pilot that aims to lay the foundations for an NDC that will cater to a wider science community. @@ -35,7 +35,7 @@ will cater to a wider science community. There were many discussions about AI, and comparisons between the NDC-C and the [National Artificial Intelligence Research Resource Pilot](https://new.nsf.gov/focus-areas/artificial-intelligence/nairr) were -highlighted by the NSF reps in attendance. +highlighted by the NSF reps in attendance. Two NSF representatives were present at the meeting, [Marlon Pierce](https://www.nsf.gov/staff/staff_bio.jsp?lan=mpierce&org=OAC&from_org=OAC) @@ -64,7 +64,7 @@ There were two guided discussions facilitated by the workshop organizers: * A conversation with Elizabeth Simmons, Executive Vice Chancellor, UC San Diego, about how the collective impact framework has been successfully utilized by UC San Diego. - + * A panel discussion about equity and education in climate research. Panelists were: * Olivia Graeve, professor at UC San Diego diff --git a/notes/2024-04-04_pangeo-forge.md b/notes/2024-04-04_pangeo-forge.md index 7f64403..b7fa920 100644 --- a/notes/2024-04-04_pangeo-forge.md +++ b/notes/2024-04-04_pangeo-forge.md @@ -82,9 +82,9 @@ using PF, and all have their own use cases and infra. Needs. an “STP” which is a template which allows you to package all info about how to transform a legacy format into a new output format and generate STAC compliant metadata. * STP is a place like PF where people can coordinate on upstream data format issues and - share best practices. + share best practices. * Can create an STP for an individual dataset to fix up issues. - * DevSeed orchestrates STP at scale in different ways. + * DevSeed orchestrates STP at scale in different ways. * https://github.com/developmentseed/stactools-pipelines - for AWS ### Questions / needs from QGreenland-Net @@ -110,17 +110,17 @@ using PF, and all have their own use cases and infra. Needs. provides those recipes optionally? * Geospatial-specialized PTransforms * Generic subprocess-calling PTransform - * E.g. https://pangeo-forge.readthedocs.io/en/latest/ecosystem.html + * E.g. https://pangeo-forge.readthedocs.io/en/latest/ecosystem.html * QGreenland-Net is funding Matt to work on Openscapes. How can we take advantage of Openscapes specialization in cross-DAAC collaboration to help understand and meet NASA/DAAC needs and bring in collaborators to move faster? * If we do decide to integrate with PF, Aimee says “yes, we can integrate and can contribute!” Join Pangeo Forge coordination meetings! Might be on a hiatus now, in the air. Next months will be disruptive because the approach is more of a do-ocracy with - less centralized governance. + less centralized governance. * Aimee’s roadmap for governance is a WIP. More work needs to be done to communicate the new vision for PF. -* Look at Aimee’s cloud optimized formats guide: https://guide.cloudnativegeo.org/ +* Look at Aimee’s cloud optimized formats guide: https://guide.cloudnativegeo.org/ diff --git a/notes/2024-04-11_dev-standup.md b/notes/2024-04-11_dev-standup.md index 64cb59c..d6da8b9 100644 --- a/notes/2024-04-11_dev-standup.md +++ b/notes/2024-04-11_dev-standup.md @@ -27,7 +27,7 @@ date: "2024-04-11" * Matt and Trey were exploring how to deploy/setup Apache Spark on k8s, so that we could test running Apache beam workflows. Got stuck on configuring Spark. Documentation is dense, a LOT to learn. - * Spark originally designed for JVM-based langs. Python, Go, others tacked on later. We want to focus on accessibility, we should pick a lang. Python makes sense as a focus. + * Spark originally designed for JVM-based langs. Python, Go, others tacked on later. We want to focus on accessibility, we should pick a lang. Python makes sense as a focus. * Is ADC transitioning from Ray.io -> parsl? Why? Are they used for different scenarios? * [viz-workflow](https://github.com/PermafrostDiscoveryGateway/viz-workflow/tree/main) README: @@ -46,5 +46,3 @@ date: "2024-04-11" - [ ] Rushiraj: Ask Juliet for opinions on Parsl vs. Ray - [ ] Continue exploring Parsl - - diff --git a/notes/2024-04-17_tech-meeting.md b/notes/2024-04-17_tech-meeting.md index 3408018..d45a44c 100644 --- a/notes/2024-04-17_tech-meeting.md +++ b/notes/2024-04-17_tech-meeting.md @@ -76,9 +76,4 @@ date: "2024-04-17" - [ ] Matt J add Matt, Trey, Rushiraj to PDG Slack - [ ] Schedule meeting to discuss quarterly goals. What are our milestones? - [ ] Matt J to share calendar w/ Matt F & Trey -- [ ] - - - - - +- [ ] diff --git a/notes/2024-05-06_dev-standup.md b/notes/2024-05-06_dev-standup.md new file mode 100644 index 0000000..8597a12 --- /dev/null +++ b/notes/2024-05-06_dev-standup.md @@ -0,0 +1,41 @@ +--- +title: "2024-05-06: Dev standup" +date: "2024-05-06" +--- + +## Participants + +* Trey +* Rushiraj + + +## Activities review + +* Rushiraj: + * Continuing work on Cesium map + * Took gpkg from QGreenland, using PDG pipeline code to get formatted into + web tiles for cesium. +* Trey: + * Matt and I started focusing on developing our core set of functionality + for OGDC ([see this + issue](https://github.com/QGreenland-Net/qgreenland-net.github.io/issues/40)). PR + with our initial work + [here](https://github.com/QGreenland-Net/qgreenland-net.github.io/pull/39). + * Also began to refine criteria and do assessment of different workflow + technologies ([see this + issue](https://github.com/QGreenland-Net/.github/issues/26)). Evaluation + spreadsheet can be found + [here](https://docs.google.com/spreadsheets/d/1f8CgOu1wXnaoAfEODrCkxNfY-Xq_as1AJPrPVK74lM0/edit#gid=0). + * Did an initial assessment of parsl, which we feel has some issues that + other tools might already have solutions for (e.g., logging bash app + commands, pods not getting cleaned up in some error states, etc.) + * In last hour of last week, we explored use of + [argo](https://argo-workflows.readthedocs.io/en/latest/). We were able + to configure argo and reproduce our seal-tags workflow in that hour. + We were very impressed with argo's user-experience, which includes a + monitoring dashboard and a CLI that provides a straightforward way of + watching/monitoring jobs without needing to use `kubectl` commands + directly. See our spike repo + [here](https://github.com/QGreenland-Net/argo-spike) + * Trey now at 25% time for QGreenland-Net, which will go down to 0 at the + end of this month. Will be focusing on other project work this week. diff --git a/presentations/ndcc-overview.md b/presentations/ndcc-overview.md index 5d35148..dbce402 100644 --- a/presentations/ndcc-overview.md +++ b/presentations/ndcc-overview.md @@ -20,7 +20,7 @@ author: ::: {.notes} Workshop focused on defining a vision for the NSF's activities supporting the -creation of a more integrated cloud-based system for climate science. +creation of a more integrated cloud-based system for climate science. The NDC-C is a 2-year pilot that aims to lay the foundations for an NDC that will cater to a wider science community. @@ -61,7 +61,7 @@ projects in the next 6-18 months. * Reccomendations for the NSF NDC-C via a workshop report * Slides describing each project -* Establishment of actual and potential collaborations +* Establishment of actual and potential collaborations #