Skip to content

Commit

Permalink
add peakvalley notebook in repo instead of using fusets
Browse files Browse the repository at this point in the history
  • Loading branch information
jdries committed Nov 4, 2023
1 parent f091a49 commit e480c7f
Show file tree
Hide file tree
Showing 5 changed files with 2,861 additions and 1 deletion.
3 changes: 3 additions & 0 deletions tutorial/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,12 @@ parts:
title: How to exploit data on Pangeo
- file: part3/advanced_udf
title: Custom algorithms - UDF (OpenEO) and ufunc (Xarray)
- file: part3/peak_valley
title: Advanced OpenEO - Peak Valley Case
- file: part3/scaling_openeo
title: Scaling with OpenEO


- caption: Beyond the workshop
chapters:
- file: afterword/resources
Expand Down
26 changes: 26 additions & 0 deletions tutorial/part2/advanced_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,21 @@
- What are openEO platform 'credits'
- Learn about metadata generation in openEO

## Algorithm types

When writing an algorithm in openEO or Pangeo, the first thing to do is to determine the type of algorithm.
Once you know the type, you can fall back to well-known examples as a template to implement your algorithm.

Using the right method for your type of algorithm also ensures that it can be parallellized and executed
in the most efficient way. An algorithm that only requires a single observation for instance, does not need
to bother with retrieving a full timeseries of observations.

In part 2 of this tutorial, we'll cover a few common types:
- single observation, single pixel
- temporal analysis over a single pixel

In part 3, we have an example of an analysis that requires multiple values in both space and time.


## Case 1 Rank composites

Expand All @@ -20,6 +35,10 @@ Key takeaways:
- Use array_apply to 'loop' over data
- Create masks to selectively load data

Algorithm type:
- Spatial neighborhood for cloud proximity score/masking
- Per pixel temporal analysis for scoring

https://github.com/Open-EO/openeo-community-examples/blob/main/python/RankComposites/rank_composites.ipynb

## Case 2 Machine learning training and inference
Expand All @@ -35,6 +54,8 @@ Key takeaways:
- Transform a timeseries datacube into features for learning
- Use ML processes directly from openEO

Algorithm type: per-pixel timeseries analysis

Dynamic landcover: https://docs.openeo.cloud/usecases/landcover/

## Case 3 ESA WorldWater
Expand All @@ -45,6 +66,11 @@ by DHI-GRAS.
Key takeaways:
- Use of complex, rule-based classification

Algorithm type:

- temporal compositing into ARD
- single observation, single pixel, analysis

Note: the most recent code of this example is still under review in a pull request, so we include the relevant bits inline.

https://github.com/DHI-GRAS/worldwater-toolbox
Expand Down
11 changes: 10 additions & 1 deletion tutorial/part3/advanced_udf.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@
"\n",
"\n",
"The openEO user defined functions (UDF's), provide a way to run Pangeo code as part of a larger openEO workflow.\n",
"This allows you to get the best of both worlds.\n",
"This allows you to get the best of both worlds:\n",
"- You can use openEO to get access to a wide variety and full archives of EO data.\n",
"- Preprocessing to ARD such as Sentinel-1 backscatter is easily done in openEO\n",
"- For very fast interactive debugging of the core of your algorithm, local testing based on Pangeo is faster than basically any cloud based service\n",
"- Once your algorithm is ready, you can run it in the cloud without worries thanks to openEO\n",
"\n",
"\n",
"To understand user defined functions, it is recommended to start here:\n",
"\n",
Expand Down Expand Up @@ -43,6 +48,8 @@
"\n",
"## UDF with basic XArray code: peak-valley detection\n",
"\n",
"Algorithm type: single pixel timeseries analysis\n",
"\n",
"This example may require a `pip install fusets`\n",
"\n",
"The actual peak-valley algorithm can be found here:\n",
Expand All @@ -59,6 +66,8 @@
"\n",
"## UDF with machine learning\n",
"\n",
"Algorithm type: spatiotemporal data chunk\n",
"\n",
"Applying deep learning models is one of the most common reason to use UDF's.\n",
"\n",
"This example relies on understanding 'apply_neighborhood', which is explained here:\n",
Expand Down
Loading

0 comments on commit e480c7f

Please sign in to comment.