STAC Hackathon

🌏 Welcome to the DEA Notebooks ODC-STAC Hackathon!

Thanks for joining our two-day hackathon to help improve Digital Earth Australia’s (DEA) Notebooks!

🏆 What’s the goal of this hackathon?

The primary goal of the hackathon is to transition DEA-Notebooks to use odc-stac. This will be the first full repository refresh of DEA-Notebooks since 2019. A more detailed list of hackathon goals is outlined below.

• Convert DEA-Notebooks repo to use odc-stac.load rather than datacube.load​
• Develop skills and knowledge on odc-stac (and STAC more generally) ​
• Further test, refine, and document alternate access patterns for DEA (e.g. CodeSpaces, Colab)
• Develop communications and KH articles on the update to DEA-Notebooks
• Address backlog DEA-Notebooks tasks (‘Issues’)
• Add/ update documentation and guides to the Dea-Notebooks wiki on accessing DEA data via sandbox, GitHub Codespaces, Google Collab and local environments

🗓️ Hackathon Structure

Day 1

9am

• Introduction
• Assign yourself (or small team) to tasks in the Hackathon project board
• Get started!

Day 2

9am

• Open discussion on any issues encountered on Day 1

3.30pm

• Showcase of work
• Discuss next steps and timeline for approval and deployment of updated DEA-Notebooks
• Publication of JOSS paper timeline?

🔧 Helpful Resources

Planning tools

• Hackathon project board: See what needs doing, and what others are working on
• Prioritised notebook list: Shows which notebooks are highest priority for updating

Getting set up

• Set up instructions: How to get your environment running - see below
• develop_stac branch: The branch we're working in - make sure you're on it!
• DEA_notebooks_template: The latest notebook layout and style we are standardising on

Example notebooks already coverted to STAC and new notebooks template

• Introduction to loading data with STAC
• Using "load_ard" to load and cloud mask Landsat and Sentinel-2
• Introduction to DEA Surface Reflectance (Landsat, Collection 3)
• Monitoring coastal erosion along Australia's coastline

Other resources

• Knowledge Hub article on STAC
• DEAfrica odc-stac conversion repository
• FrontierSI odc-stac example notebooks
• odc-stac docs page
• pystac-client docs page
• pystac docs page
• STAC spec
• Introduction to STAC (with the Brazilian DataCube)

✅ Generalised Notebook Conversion Checklist

• Notebook updated to follow updated DEA Notebooks template format
• For STAC loading:
  • odc-stac load() and configure_s3_access imported
  • pystac_client Client imported
  • Environment setup includes catalog, stac_client and configure_s3_access()
  • Cell that searched a collection for items with pystac-client
  • Cell that loads discovered items into an xarray with odc-stac
• Notebook no longer requires datacube
• Notebook no longer does sys.path.insert(1, '../Tools/')

⚙️ Set up instructions

DEA Sandbox

1. Start up an "unstable" Sandbox server
2. Navigate to where you have cloned your repo, e.g.:

   cd dev/dea-notebooks
   

3. Make sure you have the latest copy of the repo:

   git pull
   

4. Check out develop_stac branch:

   git checkout --track origin/develop_stac
   

5. Create a copy of develop_stac to do your work:

   git checkout -b my_new_branch develop_stac
   

6. Install latest version of DEA Tools:

   pip install -e .
   

7. Update your notebook to use STAC and the new DEA Notebooks template
8. When ready, commit and push your changes:

   git commit -am 'I made some changes woooo'
   git push
   

9. Raise a pull request in the DEA Notebooks repository, setting the "base branch" as develop_stac:

Github CodeSpaces

TBA!