International Baccalaureate: Data Visualization -- Crawler
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
data
scripts
src
.gitignore
README.md
build.sbt
todo.md

README.md

Gapminder Crawler

This Scala project implements a simple Selenium-based crawler for Gapminder's geopolitical and demographic data sets (which have been included here for the reader's convenience).

Object com.alexorbit.ibdataviz.crawler.Crawler is a command-line Scala application that drives Firefox to Gapminder's download page and retrieves all data files from Google Drive in CSV format.

These files contain a variety of demographic statistics for all countries during the last two centuries on aspects such as population, health, income, etc. This data is collected to populate a relational database with purposes of data visualization and analysis.

I use the Saiku OLAP server to enable analytics on cubes created from Gapminder data.

This is part of my 2014 IB Personal Project for Suncoast High School.

Roadmap

  • 1.0: As many cubes as GapMinder files: the only dimensions are Geography (country) and Time (year)
  • 1.1: Coalesce GapMinder files to provide additional dimensions (e.g. age range)