The Datacube UI from https://github.com/ceos-seo/data_cube_ui forked at commit c2ffec057f412c778716fcae5bf447969a121975
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
apps
config
data_cube_ui
db_backups
docs
requirements
scripts
static
templates
utils @ 7ec6d90
.gitattributes
.gitignore
.gitmodules
Dockerfile
LICENSE
README.md
docker-compose.yml
ingest-data.sh
initialize-datacube.sh
manage.py
run-django.sh

README.md

What is the purpose of this fork?

This fork tries to collect scripts and configuration files to run the Data Cube UI dockerized.

What has been changed in this fork?

This is a work in progress and there are still a few things to do, before everything works together as it should. To see, what needs to be done, look at the issues. Also, before using this in production, keep in mind to change credentials in the Docker configuration files and go through Django's settings.py to use another secret key, deactivate debug mode etc.

How to run this?

  • install Docker and Docker Compose on your machine (host)
  • clone this repository and cd into it
  • create a directory to host the data for the datacube on your host:
mkdir datacube
  • adjust host-side filepath for the datacube volume in docker-compose.yml (use the absolute path to the above created directory)
  • run sudo docker-compose build | tee -a docker-compose.build.log to create necessary containers
  • run sudo docker-compose up | tee -a docker-compose.up.log to spin up all containers
  • run initialize-datacube.sh to initialize the datacube and create some required directories inside the datacube volume
  • prepare your ingestion configuration as usual and place the original data unpacked into datacube/original_data
  • adjust the filepaths for the ingestion configuration in ingest-data.sh

Note: While technically it is not necessary to pipe the output into tee, it is helpful in debugging problems.

The Data Cube UI can be accessed via localhost:8080 or equivalent. 0.0.0.0:8080 works, too. At the moment, the following works:

  • ingesting data into the datacube
  • accessing the web interface via web browser
  • conduct analyses, if data is available in the data cube

Although sample data for Django is being inserted while running the initialize-datacube.sh step, no data is available within the data cube. Therefore, you need to ingest data into the datacube first and then configure the correct areas within Django.

CEOS Data Cube UI

The CEOS Data Cube UI is a full stack Python web application used to perform analysis on raster datasets using the Data Cube. Using common and widely accepted frameworks and libraries, our UI is a good tool for demonstrating the Data Cube capabilities and some possible applications and architectures. The UI's core technologies are:

  • Django: Web framework, ORM, template processor, entire MVC stack
  • Celery + Redis: Asynchronous task processing
  • Data Cube: API for data access and analysis
  • PostgreSQL: Database backend for both the Data Cube and our UI
  • Apache/Mod WSGI: Standard service based application running our Django application while still providing hosting for static files
  • Bootstrap3: Simple, standard, and easy front end styling

Using these common technologies provides a good starting platform for users who want to develop Data Cube applications. Using Celery allows for simple distributed task processing while still being performant. Our UI is designed for high level use of the Data Cube and allow users to:

  • Access various datasets that we have ingested
  • Run custom analysis cases over user defined areas and time ranges
  • Generate both visual (image) and data products (GeoTiff/NetCDF)
  • Provide easy access to metadata and previously run analysis cases

Installation

git clone https://github.com/ceos-seo/data_cube_ui.git -b master
cd ~/Datacube/data_cube_ui
git submodule init && git submodule update

Requirements

  • Full Data Cube installation with ingested data

  • apache2

  • libapache2-mod-wsgi-py3

  • redis-server

  • libfreeimage3

  • tmux

  • django

  • redis

  • celery

  • imageio

  • django-bootstrap3

  • matplotlib

  • stringcase

For more detailed instructions, please read the documentation. If you want to add a new algorithm to the UI, you can follow our adding a new algorithm documentation.