What is the purpose of this fork?
This fork tries to collect scripts and configuration files to run the Data Cube UI dockerized.
What has been changed in this fork?
- Git history has been removed to avoid problems with large files that had been added and removed from the original repository
- both patches from https://github.com/ikselven/datacube-setup-scripts have been applied
- Django migrations have been added to this repository
- configuration files for Docker, based on https://github.com/crc-si/datacube-ui-clean, have been added
- scripts for initializing the datacube and for ingesting data have been added
This is a work in progress and there are still a few things to do, before everything works together as it should. To see, what needs to be done, look at the issues. Also, before using this in production, keep in mind to change credentials in the Docker configuration files and go through Django's
settings.py to use another secret key, deactivate debug mode etc.
How to run this?
- install Docker and Docker Compose on your machine (host)
- clone this repository and cd into it
- create a directory to host the data for the datacube on your host:
- adjust host-side filepath for the datacube volume in
docker-compose.yml(use the absolute path to the above created directory)
sudo docker-compose build | tee -a docker-compose.build.logto create necessary containers
sudo docker-compose up | tee -a docker-compose.up.logto spin up all containers
initialize-datacube.shto initialize the datacube and create some required directories inside the
- prepare your ingestion configuration as usual and place the original data unpacked into
- adjust the filepaths for the ingestion configuration in
Note: While technically it is not necessary to pipe the output into tee, it is helpful in debugging problems.
The Data Cube UI can be accessed via
localhost:8080 or equivalent.
0.0.0.0:8080 works, too. At the moment, the following works:
- ingesting data into the datacube
- accessing the web interface via web browser
- conduct analyses, if data is available in the data cube
Although sample data for Django is being inserted while running the
initialize-datacube.sh step, no data is available within the data cube. Therefore, you need to ingest data into the datacube first and then configure the correct areas within Django.
CEOS Data Cube UI
The CEOS Data Cube UI is a full stack Python web application used to perform analysis on raster datasets using the Data Cube. Using common and widely accepted frameworks and libraries, our UI is a good tool for demonstrating the Data Cube capabilities and some possible applications and architectures. The UI's core technologies are:
- Django: Web framework, ORM, template processor, entire MVC stack
- Celery + Redis: Asynchronous task processing
- Data Cube: API for data access and analysis
- PostgreSQL: Database backend for both the Data Cube and our UI
- Apache/Mod WSGI: Standard service based application running our Django application while still providing hosting for static files
- Bootstrap3: Simple, standard, and easy front end styling
Using these common technologies provides a good starting platform for users who want to develop Data Cube applications. Using Celery allows for simple distributed task processing while still being performant. Our UI is designed for high level use of the Data Cube and allow users to:
- Access various datasets that we have ingested
- Run custom analysis cases over user defined areas and time ranges
- Generate both visual (image) and data products (GeoTiff/NetCDF)
- Provide easy access to metadata and previously run analysis cases
git clone https://github.com/ceos-seo/data_cube_ui.git -b master cd ~/Datacube/data_cube_ui git submodule init && git submodule update
Full Data Cube installation with ingested data