Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 151 lines (107 sloc) 5.255 kB
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
1 Betashapes
2 ==========
3 created by Melissa Santos and Schuyler Erle
4 (c) 2011 SimpleGeo, Inc.
af994ff You know, stuff.
Schuyler Erle authored
5
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
6 What is this?
7 -------------
8
8f7fff0 Add a link to the blog post.
Schuyler Erle authored
9 It's the code used by SimpleGeo to generate its international neighborhood
10 dataset.
11
12 See the blog post for an explanation:
13
14 http://blog.simplegeo.com/2011/08/05/its-a-beautiful-day-in-the-neighborhood/
25d964c @aguynamedben Update README
aguynamedben authored
15 (The blog post content is now in the BLOG_POST.mkd file of this repo)
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
16
17 Why's it here?
18 --------------
19
20 We had fun writing it. We like giving stuff away. Maybe you'll find it useful.
21 Maybe you'll improve it and send us a pull request! We provide no warranty, and
22 no support. If it breaks, you get to keep the pieces.
23
24 How's it work?
25 --------------
26
27 Well, it helps if you download Yahoo's GeoPlanet dump, and load both it and all
28 or some subset of Planet.osm into PostGIS.
29
8f7fff0 Add a link to the blog post.
Schuyler Erle authored
30 You'll need to create a data/ directory, and dump a mapping of WoE ID -> Name
31 into a file called `data/names.txt`, and another mapping of Parent ID, Name,
32 Type -> WoE ID into another file called `data/suburbs.txt`. This is stupid and
33 could be done a lot more cleanly.
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
34
35 Here is a sample of the names.txt we're using:
36
37 29372661 San Francisco Javier
38 772864 San Francisco de Paula
39 108040 Villa de San Francisco
40 142610 San Francisco Culhuacán
41 349422 San Francisco de Limache
42 12521721 San Francisco International Airport
43
44 Here's a sample of the suburbs.txt:
45
46 44418 Streatham Common Suburb 20089509
47 44418 Upper Walthamstow Suburb 20089365
48 44418 Castelnau Suburb 20089570
49 44418 Harold Hill Suburb 22483
50 44418 Blackfriars Road Suburb 20094299
51 44418 Lampton Suburb 44314
52 44418 Lower Place Suburb 20089447
53 44418 Furzedown Suburb 20089510
54 44418 Crofton Suburb 20089334
55 44418 Collier's Wood Suburb 20089517
56
57 Running build_neighborhood.sh takes over from there.
58
59 What's in it?
60 -------------
61
62 build_neighborhood.sh <city> <woeid>
63
64 This shell script makes the magic happen. Depends on PostgreSQL and GRASS,
65 in addition to all the other stuff in here.
66
67 blockr.py <names.txt> <blocks.json> <points.txt>
68
69 The main neighborhood generation script. Takes a name file
70 (tab-separated, mapping WoE ID to name), a GeoJSON FeatureCollection
71 containing the block polygons to be assigned, and a points file (as
72 generated by geocrawlr.py).
73
74 Requires Shapely.
75
76 outliers.py <points.txt>
77
78 A module for reading points.txt files and discarding outlying points based
79 on median absolute distance. If run as a script, prints the bounding box of
80 the points after outliers are discarded.
81
82 geocrawlr.py <woe_id> [<woe_id> ...]
83
84 A script that crawls the Flickr API looking for geotagged photo records
85 associated with the given woe_ids. Writes line-by-line, tab-separated
86 values to stdout consisting of: Photo ID, WoE ID, Longitude, Latitude.
87 Uses Flickr.API. You must have your FLICKR_KEY and FLICKR_SECRET set in the
88 environment.
89
90 geoplanet.py
91
92 A utility script to query Y! GeoPlanet. Takes names, one per line, on stdin,
93 queries GeoPlanet, and outputs the first WoE ID and name returned on stdout.
94 Set YAHOO_APPID in your environment.
95
96 mapnik_render.py
97
98 A Mapnik script to visualize the neighborhood.json and blocks.json data
99 together.
100
101 leaves_from_woeid.py
102
103 Walks a table of GeoPlanet data in PostgreSQL and fetches all the leaves
104 descending from a given WoE ID.
105
106 What's a "betashape"?
107 ---------------------
108
2f099db More references.
Schuyler Erle authored
109 See:
110
111 http://code.flickr.com/blog/2008/10/30/the-shape-of-alpha/
112
113 also see:
114
115 http://code.flickr.com/blog/2009/01/12/living-in-the-donut-hole/
116
117 and for good measure:
118
119 http://code.flickr.com/blog/2011/01/08/flickr-shapefiles-public-dataset-2-0/
120
121 Propers to Aaron Straup Cope for his ideas and encouragement.
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
122
123 License
124 -------
125
126 Copyright (c) 2011, SimpleGeo, Inc.
127 All rights reserved.
128
129 Redistribution and use in source and binary forms, with or without
130 modification, are permitted provided that the following conditions are met:
8f7fff0 Add a link to the blog post.
Schuyler Erle authored
131
4b38907 Move stuff around and add DOCUMENTATION.
Schuyler Erle authored
132 * Redistributions of source code must retain the above copyright
133 notice, this list of conditions and the following disclaimer.
134 * Redistributions in binary form must reproduce the above copyright
135 notice, this list of conditions and the following disclaimer in the
136 documentation and/or other materials provided with the distribution.
137 * Neither the name of the SimpleGeo, Inc. nor the
138 names of its contributors may be used to endorse or promote products
139 derived from this software without specific prior written permission.
140
141 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
142 ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
143 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
144 DISCLAIMED. IN NO EVENT SHALL SIMPLEGEO, INC. BE LIABLE FOR ANY
145 DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
146 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
147 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
148 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
149 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
150 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Something went wrong with that request. Please try again.