Introduction to the Open Data Cube: GeoPython 2021

Welcome to this repo, here you will find all the material related to the talk.

1. Presetation
2. References
3. Deploy your own Data Cube

Presenation

Feel free to take at look at the Presentation.pdf file.

References

We cover from 1 to 5 during the talk. Reference 6 have more introductory notebooks from the DEA datacube. Finally, there are amazing Geoscience-BigData related projects that may be interesting to explore, references 7 and 8 cover one of these projects.

1. What is the Open Data Cube?
2. Imaging the Past
3. Open Data Cube Web Page
4. Open Data Cube Manual
5. Cube in a Box
6. Digital Earth Australia notebooks and tools repository
7. Pangeo: A community platform for Big Data geoscience
8. Pangeo History

Deploy your own Data Cube

Here I will show you how you can deploy your own instance of the open datacube using the configuration files I have prepared for you. So let's do it, I promise it won't be painful.

This configuration file were made to fulfil the purpose of this talk, then if you are considering to continue the exploration of the open data cube use the Cube in a Box alternative.

Requirements

1. Computer with at least 8 GB of RAM memory.
2. Windows or Linux OS

Preliminary configuration

1. Download and install Virtualbox 6.1.18
2. Download and install Vagrant 2.2.15
3. Reboot your computer

Deployment of a Virtual Machine with Vagrant

Vangrant is a tool that automates the deployment of virtual machines. All the details and configuration of the virtual machine is already set for me in the Vagrantfile, you can take a look on this. On this file I configured an ubuntu based virtual machine, this machine will use two host ports the 8080 for Jupyter Notebooks and 8081 for the open data cube explorer. Make sure this ports are not used by other application on your computer.

We use a virtual machine in order to easy the deployment of containers and avoid other tecnical details that can take place during the deployment, since this virtual machine is already prepared to do that. However, if you are familiar with containers you can follow up to the 4 step and go to the next section to deploy the open data cube containers directly on your system.

1. Download geopython-2021-main.zip repository.
2. Extract the ZIP file.
3. Get into the geopython-2021-main folder, make sure you can see the Vagrantfile on this folder.
4. Being located in the directory geopython-2021-main, open a PowerShell console (in Windows) or the Terminal (in Linux).
5. Use the following command in order to start a Virtual Machine (VM) automatically

vagrant up --provision

The vagrant up --provision command will perform the following steps: (1) look for the Vagrantfile that is located in your current directory, (2) it will read the specifications set on this file in order configure the Virtual Machine for you, (3) Download the ubuntu VM image from the Vagrant Cloud, and (4) run the ubuntu VM on your computer. The --provision flag will will tell vagrant to install docker and docker-compose as is specified in the Vagrantfile. Docker and docker-compose are the tools requiered to deploy the open data cube containers.

NOTE: The --provision flag is used only the first time you deploy the virtual machine.

Vagrant Usefull Commands

• stop the virtual machine use vagrant halt
• start the virtual machine use vagrant up
• enter the virtual machine console use vagrant ssh
• exit the virtual machine exit
• destroy the virtual machine use vagrant destroy, if you do this, the next time you need to use the virtual machine, use the vagrant up --provision command.

Deployment of Open Data Cube Containers

Assuming you did step 5 of the previous section, use the vagrant ssh to get into the virtual machine you created in the previuos section. Then, use the following commands to deploy the open data cube containers.

Get into the /vagrant directory

vagrant@vagrant:~$ cd /vagrant

Check if the docker-compose.yml is present in the current directory.

vagrant@vagrant:/vagrant$ ls -l

total 1780
-rwxrwxrwx 1 vagrant vagrant    1268 Apr 14 23:59 docker-compose.yml
...

Start postgis and jupyter containers

vagrant@vagrant:~$ sudo docker-compose up -d

Check that all containers are working properly

vagrant@vagrant:~$ sudo docker-compose ps

Initialize the data cube database

vagrant@vagrant:~$ sudo docker-compose exec jupyter datacube system init

Initialising database...
Created.
Checking indexes/views.
Done.

Open the browser at http://localhost:8080 to display the JupyterLab interface. Then, follow the instructions given in the notebooks. Enjoy your journey !!

1. Sentinel 2 - Product definition.ipynb
2. Sentinel 2 - Image Indexing.ipynb
3. Sentinel 2 - Normalized Vegetation Index (NDVI).ipynb