1. EWAIM: an Extensible Web App for Interactive Mapping

Cite this repository

@misc{Ewaim2017,
  author = {Muellerklein, OC and Miao, Z},
  title = {EWAIM: an Extensible Web App for Interactive Mapping},
  year = {2017},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Thru-Echoes/ewaim-webapp}},
  commit = {6507427e41f7bd9231d3759dbdd95cc516ae059c}
}

1.1 Overview

EWAIM-webapp is a web mapping app that uses Leaflet x D3 (Javascript) on the frontend and Flask x PostGreSQL on the backend. In addition, we include a directory titled EWAIM that contains a Python package for some simple data analysis and utility functionality - used in the backend. The EWAIM package has continuous testing through Travis-CL. Users of this app (for now) need to run the setup.py file to make the EWAIM module (EWAIM is imported and used in the main Flask app.py file). Our future development includes building further data processing and analysis functions as well as Species Distribution Model functions in the EWAIM package focused on spatial, movement, and time series data.

PEARL: using EWAIM to map projections of current and future distributions of species

Species: Abbreviata bancrofti

Showing predicted changes in native and global habitat due to climate trends using three different distribution models (green, yellow, and red). Current distribution shown in blue.

1.2 Research Application

EWAIM (the webapp as well as the package) will be extended for direct research applications with at least two large-scale research projects in the Wayne Getz lab at UC Berkeley. One will be for the Israel Migratory Bird project - a study that is collecting spatial data of bird movement (roughly 2 million points daily). The other direct application is for PEARL: Parasite Extinction Assessment and Red-Listing - a database of Species Distribution Models (predictions of presence / absence) of species of parasites currently and future projections based on climate change trends.

1.3 Breakdown of Development

This interactive database website is a collaborative project from Zhongqi Miao and Oliver Muellerklein from the Wayne Getz lab at UC Berkeley. Oliver primarily developed the frontend components of this project - focusing on routes to pages, GET / POST events, processing data from Flask (backend) in the HTML and Javascript, and interacting with D3 x Leaflet. In addition, Oliver contributed to building the underlying structure and continuous integration of testing of the EWAIM package. Miao developed the backend capabilities with SQL interactions with the database, raster data preparation with parallel tile functions (see raster_data_prep/ directory and associated README), and data analysis / GIS functionality for spatial and time series data.

2. Setup

2.1 CLI Setup of EWAIM Module

    $ python3 setup.py install
    # And test (used for Travis-CL)
    $ python3 setup.py test

2.2 Data Preparation

• See database_prep folder for database setup.
• See raster_data_prep folder for raster data preparation.

For the sake of this project submission we provide pre-made CSVs and database (database_prep/temperature.gz) for the data instead. This is because installing packages / modules GDAL can likely cause system-based errors. Normally we will use proper database connections (as our utility functions show) for connections to the data for layers and analysis.

NOTE: follow instructions in database_prep and raster_data_prep

NOTE: must install all database dependencies as described in database_prep/README.md for the non-example version of this app!

3. Future Development

• Make data table CSV downloadable
• Allow users to upload their own data

3.1 To Do (Oliver)

• Make nav_layout.html for web app skeleton
• Create Flask layout for maps
• Create Flask layout for data analysis results etc
• Pick data based on user event (local CSV Carbon vs CSV Twitter)
• Make front page to select data (have world map background)
• From selected data, render Carbon map
• Make clickable / selectable heatmap layer - users can select different variables / layers for mapping
• ... heatmap will be directly on map (all frontend), make it able to accept backend data

    # Turn tif into tiles for plotting (after test() turn grd into tif)
    $ gdal2tiles.py -z 1-6 -w none tif/ABBREVIATA\ BANCROFTI\ current.tif ../../../pearl-tiles/abba_curr

3.2 To Do (Miao)

• Assume frontend passes back 'use postgres' - make db connections
• Use Geo-Alch
• DATASHADER (Not going to be done now)
• Create GIS utility functions
• Create shapefile utility functions
  • Functions importing shapefiles into python
  • Functions importing imported shapefiles into database
  • Importing Pearl fliles
• Create raster utility functions
  • Importing rasters into databases
  • Creating rasters from databases
  • Make tiling maps from the rasters
• Data analysis utilities

---

4. Misc.

4.1 Data Analysis Methods

• Kerneley Density Estimation (KDE): A non-parameteric density estimator

• Gaussian Mixture Model (GMM): A parametric probability density function represented as a weighted sum of Gaussian component densities

More to come...