Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Failed to load latest commit information.
Betashapes ========== created by Melissa Santos and Schuyler Erle (c) 2011 SimpleGeo, Inc. What is this? ------------- It's the code used by SimpleGeo to generate its international neighborhood dataset. See the blog post for an explanation: http://blog.simplegeo.com/2011/08/05/its-a-beautiful-day-in-the-neighborhood/ (The blog post content is now in the BLOG_POST.mkd file of this repo) Why's it here? -------------- We had fun writing it. We like giving stuff away. Maybe you'll find it useful. Maybe you'll improve it and send us a pull request! We provide no warranty, and no support. If it breaks, you get to keep the pieces. How's it work? -------------- Well, it helps if you download Yahoo's GeoPlanet dump, and load both it and all or some subset of Planet.osm into PostGIS. You'll need to create a data/ directory, and dump a mapping of WoE ID -> Name into a file called `data/names.txt`, and another mapping of Parent ID, Name, Type -> WoE ID into another file called `data/suburbs.txt`. This is stupid and could be done a lot more cleanly. Here is a sample of the names.txt we're using: 29372661 San Francisco Javier 772864 San Francisco de Paula 108040 Villa de San Francisco 142610 San Francisco Culhuacán 349422 San Francisco de Limache 12521721 San Francisco International Airport Here's a sample of the suburbs.txt: 44418 Streatham Common Suburb 20089509 44418 Upper Walthamstow Suburb 20089365 44418 Castelnau Suburb 20089570 44418 Harold Hill Suburb 22483 44418 Blackfriars Road Suburb 20094299 44418 Lampton Suburb 44314 44418 Lower Place Suburb 20089447 44418 Furzedown Suburb 20089510 44418 Crofton Suburb 20089334 44418 Collier's Wood Suburb 20089517 Running build_neighborhood.sh takes over from there. What's in it? ------------- build_neighborhood.sh <city> <woeid> This shell script makes the magic happen. Depends on PostgreSQL and GRASS, in addition to all the other stuff in here. blockr.py <names.txt> <blocks.json> <points.txt> The main neighborhood generation script. Takes a name file (tab-separated, mapping WoE ID to name), a GeoJSON FeatureCollection containing the block polygons to be assigned, and a points file (as generated by geocrawlr.py). Requires Shapely. outliers.py <points.txt> A module for reading points.txt files and discarding outlying points based on median absolute distance. If run as a script, prints the bounding box of the points after outliers are discarded. geocrawlr.py <woe_id> [<woe_id> ...] A script that crawls the Flickr API looking for geotagged photo records associated with the given woe_ids. Writes line-by-line, tab-separated values to stdout consisting of: Photo ID, WoE ID, Longitude, Latitude. Uses Flickr.API. You must have your FLICKR_KEY and FLICKR_SECRET set in the environment. geoplanet.py A utility script to query Y! GeoPlanet. Takes names, one per line, on stdin, queries GeoPlanet, and outputs the first WoE ID and name returned on stdout. Set YAHOO_APPID in your environment. mapnik_render.py A Mapnik script to visualize the neighborhood.json and blocks.json data together. leaves_from_woeid.py Walks a table of GeoPlanet data in PostgreSQL and fetches all the leaves descending from a given WoE ID. What's a "betashape"? --------------------- See: http://code.flickr.com/blog/2008/10/30/the-shape-of-alpha/ also see: http://code.flickr.com/blog/2009/01/12/living-in-the-donut-hole/ and for good measure: http://code.flickr.com/blog/2011/01/08/flickr-shapefiles-public-dataset-2-0/ Propers to Aaron Straup Cope for his ideas and encouragement. License ------- Copyright (c) 2011, SimpleGeo, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the SimpleGeo, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SIMPLEGEO, INC. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.