This is requested by Andrew Perry in issue #50 and seems like a more useful interpretation of "Covered by or overlaps".
This fixes issue #49. There are two changes introduced in this commit: - In the customized area lookup for country 'gb', pass on the query string if returning a redirect response. - In the JSONPMiddleware, only deal with the callback parameter if the response isn't a 302 redirect.
It's often useful to be able to easily visualize a particular subset of areas from MapIt on a map. This command (mapit_make_fusion_csv) lets you specify a particular area type and generate a CSV file that can be imported into Google Fusion Tables to show all areas of the that type randomly coloured. You can also restrict this to only show areas of that type that are covered by another area. Some instructions for importing the CSV into Google Fusion Tables is given in the comments at the top of the file and I will create a similar page on code.mapit.mysociety.org
It is useful to be able to export an area in these different formats in situations other than a view, e.g. from a django-admin command. This refactoring moves the code that generates KML, GeoJSON and WKT into a method of Area, which doesn't depend on having a request object. This refactoring also makes it easier to detect when your simplify_tolerance is high enough that all the polygons disappear, in which case a SimplifiedAway exception is raised.
This change also makes it easier to specify other boundary types in the future, based on combinations of tags.
... which is mapit_global_find_differences, to match mapit_global_analyse_differences.
The latest version of get-boundaries-by-admin-level.py also fetches boundaries with boundary="political", and changes the layout of the directory that the KML files are output to; now the top level directory names are just the 3-letter MapIt types that the KML files are intended for, rather than al02 to al11 administrative boundaries.
This script had previously only been tested with the initial data import of MapIt Global, and it was undecided at that stage how to deal with the next import (e.g. whether to preserve the boundaries from the previous generation or not). Our decision was to preserve the previous generation; we test whether the new boundary is the same as that in the current generation in the database - if so, then the generation_high for that area is extended (and the names are updated) or otherwise a new generation is created. The changes to the import script in this commit puts this decision into effect.
…t Global This script, when given a directory that contains a newly generated set of KML files, will check every KML file against the database to see whether the boundary has changed, just appeared, is invalid, etc. and produce a CSV file with a row for each KML file. This is useful for working out how many of the boundaries have changed since the last import, and so how much of an increase in database size one might expect on importing it.
… GSS ones.
Thanks to Jenny Duckett for pointing this error. In fact, this corrected version of the test is too brutal - we ignore the file if any Polygon in the MultiPolygon has too few points. In a subsequent commit I will change this behaviour.
Thanks to Geoff Stockham for pointing out this problem. This fixes issue #48. When we see a second polygon with the same unit_id and this is a new import, a dry-run import will fail because the name can't be found in the database from the unit_id - it was never saved the first time around. As with finding the name from ons_code we should ignore this error if we're not saving to the database. (The example in the bug report is the second polygon in Waterbeach ED in county_electoral_division_region.shp.)
This script generates random-enough locations in the UK and uses the timeit module to time Area.objects.by_location lookups for those locations. This is useful to assess how the performance of MapIt UK would be affected by adding polygons of various sizes and complexities to an existing instance. This script was written with Jenny Duckett.
This error would cause KML files to be generated with invalid polygons (e.g. A -> B -> A)
The calls to self.normalize_whitespace fail at the moment with the error: TypeError: normalize_whitespace() takes exactly 1 argument (2 given) We could just add self as the first argument to self.normalize_whitespace, but since this method isn't dependent on self, we might as well make it a static method.
It turns out that iteratively parsing the planet.osm file is too slow to be workable, and using a local Overpass server is much preferable.
Previously exceptions would be thrown if the API return an empty XML file when trying to fetch an element. This does sometimes happen, so this commit changes the behaviour so that such missing elements are ignored.
Sometimes the API returns an element which is missing - the code marks such a way as missing, but it will still be passed to join_way_soup. Ignore such elements.
The KML generation scripts for MapIt Global used to make a huge number of queries to http://www.overpass-api.de/api/interpreter - this is slow and hits their service rather hard. Setting up a local server and running queries using the osm3s_query binary is obviously a great deal faster, and doesn't put large load on an external service.