STAC Hackathon

🌏 Welcome to the DEA Notebooks ODC-STAC Hackathon!

Thanks for joining our two-day hackathon to help improve Digital Earth Australia’s (DEA) Notebooks!

🏆 What’s the goal of this hackathon?

The primary goal of the hackathon is to transition DEA Notebooks to use odc-stac. This will be the first full repository refresh of DEA Notebooks since 2019. A more detailed list of hackathon goals is outlined below.

• Convert DEA Notebooks repo to use odc-stac.load rather than datacube.load​
• Develop skills and knowledge on odc-stac (and STAC more generally) ​
• Further test, refine, and document alternate access patterns for DEA (e.g. CodeSpaces, Colab)
• Develop communications and KH articles on the update to DEA Notebooks
• Address backlog DEA Notebooks tasks (‘Issues’)
• Add/update documentation and guides to the DEA Notebooks wiki on accessing DEA data via sandbox, GitHub Codespaces, Google Colab and local environments

🗓️ Hackathon Structure

Day 1

9am

• Introduction
• Assign yourself (or small team) to tasks in the Hackathon project board
• Get started!

Day 2

9am

• Open discussion on any issues encountered on Day 1

3.30pm

• Showcase of work
• Discuss next steps and timeline for approval and deployment of updated DEA-Notebooks
• Publication of JOSS paper timeline?

🔧 Helpful Resources

Planning tools

• Hackathon project board: See what needs doing, and what others are working on
• Prioritised notebook list: Shows which notebooks are highest priority for updating (but feel free to pick any!)

Getting set up

• Set up instructions: How to get your environment running - see below
• develop_stac branch: The branch we're working in - make sure you're on it!
• DEA_notebooks_template: The latest notebook layout and style we are standardising on

Example notebooks already coverted to STAC and new notebooks template

• Introduction to loading data with STAC
• Introduction to DEA Surface Reflectance (Landsat, Collection 3)
• Using "load_ard" to load and cloud mask Landsat and Sentinel-2
• Generating satellite imagery animations
• Monitoring coastal erosion along Australia's coastline

Other resources

• Knowledge Hub article on STAC
• DEAfrica odc-stac conversion repository
• FrontierSI odc-stac example notebooks
• odc-stac docs page
• pystac-client docs page
• pystac docs page
• STAC spec
• Introduction to STAC (with the Brazilian DataCube)

✅ Generalised Notebook Conversion Checklist

• Notebook updated to follow updated DEA Notebooks template format
• For STAC loading:
  • odc-stac load() and configure_s3_access imported
  • pystac_client Client imported
  • Environment setup includes catalog, stac_client and configure_s3_access()
  • Cell that searched a collection for items with pystac-client
  • Cell that loads discovered items into an xarray with odc-stac
• Notebook no longer requires datacube
• Notebook no longer does sys.path.insert(1, '../Tools/')

⚙️ Set up instructions

DEA Sandbox

1. Start up an "unstable" Sandbox server
2. Navigate to where you have cloned your repo, e.g.:

   cd dev/dea-notebooks
   

3. Make sure you have the latest copy of the repo:

   git pull
   

4. Check out develop_stac branch:

   git checkout --track origin/develop_stac
   

5. Create a copy of develop_stac to do your work:

   git checkout -b my_new_branch develop_stac
   

6. Install latest version of DEA Tools:

   pip install -e .
   

Important

This step is important to make sure you can access any newly updated functions in DEA Tools.

7. Update your notebook to use STAC and the new DEA Notebooks template
8. When ready, commit and push your changes:

   git commit -am 'I made some changes woooo'
   git push
   

9. Raise a pull request in the DEA Notebooks repository, setting the "base branch" as develop_stac:

Github CodeSpaces

TBA!