Interactive map of the origin-destination stocks by country
JavaScript Python
Permalink
Failed to load latest commit information.
.idea added the viz for the world bank indicators + a few fixes Sep 8, 2011
generated replacing NaNs by "no data" and moving Saint Helena to the north east Sep 24, 2011
jquery-ui-1.8.16.custom adding jquery + jquery ui Sep 5, 2011
univers-else-font adding missing changes (I'm still not that good at git....) Sep 2, 2011
CNAME First commit Sep 1, 2011
GDP.json adding data obtained through the world bank API Sep 8, 2011
HIV.json adding data obtained through the world bank API Sep 8, 2011
LICENSE facebook open graph integration Sep 2, 2011
POP.json adding data obtained through the world bank API Sep 8, 2011
README.md Added documentation for the data generation process in Readme.md Dec 17, 2011
TUBERCULOSIS.json adding data obtained through the world bank API Sep 8, 2011
UNDER-FIVE-MORTALITY.json adding data obtained through the world bank API Sep 8, 2011
about.css adding about.html Sep 15, 2011
about.html fixing the english of the about page Sep 18, 2011
arrowcomputer2.py added a progress indicator for the off-line mode Sep 5, 2011
associate_map_to_db.py First commit Sep 1, 2011
cache.manifest correcting the coordinates of Macedonia, Serbia and Saint Helena Sep 24, 2011
code_to_coordinates.json replacing NaNs by "no data" and moving Saint Helena to the north east Sep 24, 2011
code_to_name.json added the viz for the world bank indicators + a few fixes Sep 8, 2011
countrieslatlong.json First commit Sep 1, 2011
facebook-image.png modifying facebook image Sep 7, 2011
favicon.ico adding missing changes (I'm still not that good at git....) Sep 2, 2011
geojsontosvg.py First commit Sep 1, 2011
geoloc.json replacing NaNs by "no data" and moving Saint Helena to the north east Sep 24, 2011
global_migrant_origin_database_version_4.csv First commit Sep 1, 2011
gray_texture.jpg First commit Sep 1, 2011
hasher.js First commit Sep 1, 2011
index.html rising western sahara out of the water Sep 24, 2011
jquery.js First commit Sep 1, 2011
jsoncsvunifier.py First commit Sep 1, 2011
main.css adding about.html Sep 15, 2011
modernizr.geoloc.js First commit Sep 1, 2011
name_to_code.json added the viz for the world bank indicators + a few fixes Sep 8, 2011
parser.py rising burma out of the see Sep 24, 2011
raphael.js First commit Sep 1, 2011
reconciliation.json rising western sahara out of the water Sep 24, 2011
right_blue_arrow.png First commit Sep 1, 2011
server.py First commit Sep 1, 2011
signals.js First commit Sep 1, 2011
source.txt First commit Sep 1, 2011
target.png replacing where am I by a target icon Sep 3, 2011
world-bank-urls.txt forgotten modified files Sep 8, 2011
world.json rising burma out of the see Sep 24, 2011
worldBankDataGetter.py Added documentation for the data generation process in Readme.md Dec 17, 2011
world_svg_paths.json First commit Sep 1, 2011
world_svg_paths_by_code.json rising western sahara out of the water Sep 24, 2011

README.md

MigrationsMap.net

This is the code of the site http://migrationsmap.net, including the scripts used to generate the JSON data files.

The code is under MIT license.

Don't hesitate to contact me at madewulf@gmail.com, or via twitter: http://twitter.com/madewulf if you have any question.

Data Generation Process

The map has been generated using four data sources:

  • A csv file: global_migrant_origin_database_version_4.csv, containing all the migrations number from country to country. It is really just a matrix of country-to-country migrations number, addressable by country code.
  • world.json, a geojson file containing the borders data of all countries in the world, addressable by the countries full English name
  • The world bank data, which provide indicators, like AIDS prevalence, for every country in the world, addressable by year and country code.
  • The google geolocation API, to find out the "central" coordinates of a country (the point to which arrows will be pointing when pointing to a country)

Here is the process used to generate the json data files used by the map:

  1. Run functions (mainly, geolocalize_the_world and load_matrices) from parser.py, which, from global_migrant_origin_database_version_4.csv (provided), will generate two files:

    • code_to_name.json: a json object allowing to find the full English name of a country corresponding to a country code in ISO 639 format (ex: BEL -> Belgium)
    • geoloc.json: a json object containing the dump of all geolocalization requests sent to Google. Beware, since Google was not able to geolocalize correctly some name (for example, Georgia result is the US state, not the Country), the file has been edited manually a bit, and rerunning the function will reintroduce the errors.
  2. Run arrowcomputer2.py, which, from the matrixes of migrations data generated by parser.py from the csv file and the coordinates in geoloc.json, will generate two file per country in the 'generated' directory. Those file contains ten svg paths, one per "arrow" to be drawn on the map when clicking on the country, each one associated with the corresponding migrations number. There are two files per country: one for the incoming migrants and the other for the outgoing migrants, called for example for the USA: inUSA.json and outUSA.json. Arrowcomputer2.py also generates in the process the file code_to_coordinates.json, which is a json object/dictionary allowing to find for each country code, the corresponding central coordinates. (the map size specified in geojsontosvg.py is used to compute the size and positioning of the arrows).

  3. Run geojsontosvg.py, which, from world.json and a map size (specified directly in the file), will generate the file world_svg_paths.json, which is a json object allowing to get, for each full English country name , the borders in as svg paths. The projection is an equirectangular one (if I understood correctly).

  4. Run associate_map_to_db.py, which is a lousy interactive script allowing to solve the problem of the full English country names that are not matching exactly in the data coming from the csv files and the world.json file. For example, Myanmar is referred as Burma in one of the file and United States is not an exact match for United States of America. This script will generate the file reconciliation.json, which matches the problematic country names with the corresponding country code.

  5. Run jsoncsvunifier.py which, from world_sv_paths.json and reconciliation.json will generate the final borders-as-svg file : world_svg_paths_by_code.json, which is a json object allowing to get, for each country code, the borders in as svg paths.

  6. Run worlBankDataGetter.py (I know, I should definitely enhance my file naming conventions), to download the indicators data from the world bank api and store them in json files : GDP per capita (GDP.json), mortality under five (UNDER-FIVE-MORTALITY.json), tuberculosis prevalence (TUBERCULOSIS.json), HIV prevalence (HIV.json) and population size (POP.json)

To run those python scripts, you will need to install the python Requests library first.