Python Shell
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
fixtures
junk
.gitignore
activate
challenges.md
csv.sh
main_view-2012-12-01.html
msewage.sh
parse.py
pri.csv
readme.md
test.py
what-we-did.jpg

readme.md

We start at this page. That has a bunch of state names. Clicking one state gets to another table. That other table has a "Total" link at the end. We want that page. We want these columns in the final dataset

  • Year
  • State Name
  • District Name
  • Block Name
  • Panchayat Name

We downloaded the pages manually, and we have a thingy to convert them to the msewage format. Install the msewage importer.

gem install msewage-importer

Then run something like this.

. activate
cd data
find . -exec msewage.sh {} \;

Or to generate a csv

. activate
cd data
echo Year,State Name,District Name,Block Name,Panchayat Name > ../pri.csv
find . -name *.html -exec csv.sh {} \; >> ../pri.csv

This string is useful for geocoding in CartoDB.

{panchayat_name}, {block_name}, {district_name}, {state_name}, India

Here's the map.