Emission: Worldbank indicator data visualization [WIP]

About

This repository contains the components of Emission, a Heroku app that creates charts out of retrieved Worldbank data. The app is deployed at https://emission.herokuapp.com.

All of the server-side functionality - including the fetching, parsing, uploading, updating and formatting of Worldbank data - is written in C, while the graphs are produced client-side by the tui.chart JavaScript library.

Documentation

See API documentation. TODO: documentation is partly outdated.

Building

Run make in project root folder. This will create an executable file emiss in the /bin directory.

The following compilation options are provided. You can pass them to make as arguments: make -D[option-name](=[option-value]).

Option	Default value	Defined in	Description
HEROKU	undefined	emiss.h	Switch on Heroku-specific modifications

The application will look for a valid Postgres database URL in an environment variable (or a config var in Heroku context) DATABASE_URL.

No system-wide installation option is currently provided.

Project C files from include and src

• emiss.h: Main project header.
• wlcsv.h: A wrapper around libcsv, making the association of multiple callbacks per csv parsing instance possible.
• wlpq.h: Strives to provide asynchronous, nonblocking PostgreSQL database querying facilities around libpq.
• util_json.h/util_sql.h/util_curl.h: Auxiliary utility macros for formatting JSON, SQL and setting libcurl options, respectively.

Embedded C dependency files from include/dep and src/dep

Header files	Description	Source files
bstrlib.h	The Better String Library	bstrlib.c
civetweb.h	CivetWeb HTTP/S server	civetweb.c handle_form.inl md5.inl sha1.inl timer.inl
csv.h	CSV data parsing library	libcsv.h
miniz.h	Header-only zlib/Deflate implementation
uthash.h utlist.h	Hash table/linked list macro headers
zip.h	A (de)compression library built on miniz.h	zip.c

External C dependencies (system-wide availability assumed)

Header	Description
curl.h	libcurl
libpq-fe.h	PostgreSQL C library
pcre.h	PCRE regular expression library

Dependencies for Heroku app

Project	Used for	Delivery
Fontawesome	Icons	CDN
HTML5 Boilerplate CSS	App CSS incorporates some utility classes	Local, concatenated to project stylesheet
jQuery Slim	The chart paramer input functionality	CDN
Luxbar	CSS navbar implementation	Local, concatenated to project stylesheet
Normalize.css	Cross-browser style normalization.	Local, concatenated to project stylesheet
tui.chart	Client side data visualization	CDN
Verge	Viewport dimensions detection	Local

Future directions

• Replace the DIY event loops from wlpq.h with something like libev. Perhaps get rid of pthreads altogether.
• Create a more flexible update mechanism.
• Separate the data retrieval and parsing facilities from emiss.h into their own interface, opening avenues for general querying of indicator data from Worldbank.
• Replace the naive substring search in param.js with a suffix array or somesuch device.
• Add more chart types to the Heroku app.
• Get rid of the remaining jQuery.

License

(c) Joa Käis jiikai 2018-2019, MIT.