Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Remove docs moved to gh-pages branch, update README to point to it.

  • Loading branch information...
commit 8126830ad84fa645159a7267db2c2ed07909bd2d 1 parent b4c44af
@dracos dracos authored
View
122 INSTALL.rst
@@ -1,122 +0,0 @@
-Installation
-============
-
-General notes
--------------
-
-MapIt currently uses Postgres/PostGIS as its database backend - there's no
-reason why e.g. SpatiaLite could not be used successfully, but it has never
-been tried.
-
-To install GeoDjango and PostGIS, please follow all the standard instructions
-(including creating the template) at:
-
- http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#ubuntudebian
-
-And anything else you need to set up Django as normal.
-
-[ Note for UK/Ireland: Not only is the PostGIS that is installed missing SRID
-900913, as the GeoDjango docs tell you, but both SRID 27700 (British National
-Grid) and SRID 29902 (Irish National Grid) can be incorrect (and they're quite
-important for this application!). After you've installed and got a PostGIS
-template, log in to it and update the proj4text column of SRID 27700 to include
-+datum=OSGB36, and update SRID 29902 to have +datum=ire65. This may not be
-necessary, depending on your version of PostGIS, but do check. ]
-
-You will also need a couple of other Debian packages, so install them:
-
-::
-
- sudo apt-get install python-yaml memcached python-memcache git-core
-
-You will also need to be using South to manage schema migrations.
-
-Installation as a Django app
-----------------------------
-
-As mapit is a Django app, if you are making a GeoDjango project you can simply
-use mapit from within your project like any normal Django app:
-
- * Make sure it is on ``sys.path`` (a packaged install e.g. with ``pip
- install django-mapit`` does this for you);
- * Add ``mapit`` and ``django.contrib.gis`` to your ``INSTALLED_APPS``;
- * Add the following settings to ``settings.py``:
- - ``MAPIT_AREA_SRID`` - the SRID of your area data (if in doubt, set to
- ``4326``)
- - ``MAPIT_COUNTRY`` - used to supply country specific functions (such
- as postcode validation). If you look at the country files in
- ``mapit/countries/`` you can see how to add specialised
- country-specific functions.
- - ``MAPIT_RATE_LIMIT`` - a list of IP addresses or User Agents excluded
- from rate limiting
- * Set up a path in your main ``urls.py`` to point at ``mapit.urls``.
- * run ``./manage.py syncdb`` and ``./manage.py migrate`` to ensure the db
- is set up
-
-This approach is new, so please let us know if something goes wrong or could be
-improved.
-
-Installation standalone with the example project
-------------------------------------------------
-
-A standard git clone will get you the repository:
-
-::
-
- git clone git://github.com/mysociety/mapit.git
-
-Now in a terminal, navigate to the mapit directory you've just cloned.
-
-Copy ``conf/general.yml-example`` to ``conf/general.yml`` and edit it to point
-to your local postgresql database, and edit the other settings as per the
-documentation given in that file. If you don't know what ``SRID`` to use,
-delete that line or set it to ``4326``. ``COUNTRY`` is used for e.g. differing
-sorts of postcode (zipcode) validation - if you look at the country files in
-``mapit/countries/`` you can see how to add specialised country-specific
-functions to validate postcodes etc.
-
-If you're going to be importing big datasets, make sure that ``DEBUG`` is
-``False``; otherwise, you'll run out of memory as it tries to remember all the
-SQL queries made.
-
-Optionally, also turn off ``escape_string_warning`` in Postgres' config (unless
-you want your server to run out of disc space logging all the errors, as ours
-did the first time I imported overnight and didn't realise).
-
-At this stage, you should be able to set up the database and run the
-development server. Do add an admin user when prompted:
-
-::
-
- cd project
-
- # Optionally set up a virtual environment and install requirements
- virtualenv .venv
- source .venv/bin/activate
- pip install -r requirements.txt
-
- # generate the css from the sass
- ../bin/make_css
-
- # Setup django install
- ./manage.py syncdb
- ./manage.py migrate mapit
-
- # run dev server
- ./manage.py runserver
-
-(Alternatively, set up a live web server however you wish - see the Deployment
-Django documentation for details beyond the scope of this document. Remember to
-run ``./manage.py collectstatic`` too if running Django >= 1.3.)
-
-You can then visit http://localhost:8000/ and hopefully see the default MapIt
-homepage. http://localhost:8000/admin/ should show the admin interface.
-
-This is enough to have a working site. You can create areas and postcodes using
-the admin interface and they will automatically work in the API-based front
-end.
-
-However, if you have some bulk data you wish to import (which will make
-easier), you will need to import this data into MapIt. See the country specific
-READMEs section in ``README.rst`` for instructions on their bulk imports, which
-should show the general format for how it's done.
View
55 README.rst
@@ -1,55 +1,32 @@
MapIt
=====
-MapIt is a service that maps geographical points to administrative areas. It is
-useful for anyone who has the co-ordinates of a point on Earth, and who needs
-to find out what country, region, city, constituency, or state it lies within.
-It’s also great for looking up the shapes of all those boundaries.
+MapIt is an open source project to help people run a web service that maps
+geographical points to administrative areas. It is useful for anyone who has
+the co-ordinates of a point on Earth, and who needs to find out what country,
+region, city, constituency, or state it lies within. It’s also great for
+looking up the shapes of all those boundaries.
+
+It was created in 2003 by [mySociety](http://www.mysociety.org/) for use by
+their various tools needing admin area lookup.
+
+Examples
+--------
`mySociety <http://www.mysociety.org>`_ (the UK charity that wrote this code)
-runs several public MapIts that you can use:
+runs public installations of MapIt that you might be able to use:
* `MapIt Global <http://global.mapit.mysociety.org/>`_ - global boundaries
- from the `OSM <http://www.openstreetmap.org/>`_ project.
+ from the `OpenStreetMap <http://www.openstreetmap.org/>`_ project.
* `MapIt UK <http://mapit.mysociety.org/>`_ - UK boundaries from various
sources.
The above are free for non-commercial, low-volume use.
-Technical Overview
-------------------
-
-The ``mapit`` directory contains a standard `GeoDjango
-<http://geodjango.org/>`_ app, and the ``project`` directory contains an
-example GeoDjango project using that app. Other top-level things are mostly to
-fit within mySociety's standard deployment, or country specific data, one-off
-scripts and so on.
-
-MapIt has been installed on Debian lenny and squeeze, and on Ubuntu from 10.04
-onwards. If GeoDjango and `PostGIS <http://postgis.refractions.net/>`_ run on
-it, it should theoretically be okay; do let us know of any problems.
-
Installation
------------
-MapIt can be installed as a Django app, or as a standalone server. Full details
-are in ``INSTALL.rst``.
-
-Country specific READMEs
-------------------------
-
-These are extensive notes for some of the countries that MapIt has been set up
-to serve. These are in the ``docs`` folder:
-
- * UK: ``docs/README-UK.rst``
- * Norway: ``docs/README-NORWAY.rst``
-
-Improvements / patches
-----------------------
-
-Are welcome :)
-
-The code is hosted on GitHub and we use their bug tracker too:
+MapIt can be installed as a Django app, or as a standalone server. For full
+details, please see our site at <http://code.mapit.mysociety.org/> for help
+and documentation.
- * https://github.com/mysociety/mapit
- * https://github.com/mysociety/mapit/issues
View
73 docs/README-NORWAY.rst
@@ -1,73 +0,0 @@
-Norway
-======
-
-Here are the basic instructions to install data from OSM:
-
-1. Set AREA_SRID in conf/general.yml to 4326 (as OSM shapes are in WGS84).
-2. cd bin and run "python osm_to_kml" to fetch OSM XML and convert it to KML.
-3. Change to the project directory, and create database tables:
-
-::
-
- ./manage.py syncdb
- ./manage.py migrate mapit
-
-4. Run the following (you can run anything without --commit to do a dry run):
-
-::
-
- ./manage.py mapit_generation_create --commit --desc "Initial import."
- ./manage.py loaddata norway
- ./manage.py mapit_NO_import_osm --commit ../../data/cache/*.kml
- ./manage.py mapit_import_area_unions --commit data/norway/regions.csv
- ./manage.py mapit_generation_activate --commit
-
-Please see below for information on where osm_to_kml gets its OSM data from.
-
-Alternatively, here are the basic instructions to install the N5000 data:
-
-1. Set AREA_SRID in conf/general.yml to 4326 (as we'll put N5000 shapes into
- the WGS84 co-ordinate system).
-2. Download `N5000
- <http://www.statkart.no/nor/Land/Kart_og_produkter/N5000_-_gratis_oversiktskart/>`_
- and save/unzip in the data directory.
-3. Change to the project directory, and create database tables:
-
-::
-
- ./manage.py syncdb
- ./manage.py migrate mapit
-
-4. Run the following (you can run anything without --commit to do a dry run):
-
-::
-
- ./manage.py mapit_generation_create --commit --desc "Initial import."
- ./manage.py loaddata norway
- ./manage.py mapit_NO_import_n5000 --commit ../../data/N5000\ shape/N5000_AdministrativFlate.shp
- ./manage.py mapit_import_area_unions --commit data/norway/regions.csv
- ./manage.py mapit_generation_activate --commit
-
-You should now be able to go to /point/4326/10.756389,59.949444 and have Oslo
-returned as a result.
-
-Norway OSM data
----------------
-
-The way osm_to_kml works is to fetch a number of fixed and pre-defined relation
-IDs from OSM - one (412437) containing all fylke, and then one for each fylke
-containing all the kommune inside. These relations should stay and (now they're
-correct) not change within OpenStreetmap, though of course the underlying
-relations can have their boundaries improved and so on. See the relation_ids
-list in the source of bin/osm_to_kml if you'd like to see the other relation
-IDs.
-
-The OSM tags 'name', and 'name:no' if 'name' is not set, are used to find the
-primary name of the fylke and kommune. In addition, the values of the tags
-'name:smi', 'name:fi' are imported into mapit. Only the primary name is shown
-in the mapit web pages and JSON data, while the other names are stored in the
-database.
-
-The kommune and fylke number (ID) is fetched from a the tag 'ref' in OSM, and
-if it is missing a static list of such IDS in mapit/data/norway/ids.csv is
-consulted using the name (for fylke) or name+fylke (for kommune) as the key.
View
94 docs/README-UK.rst
@@ -1,94 +0,0 @@
-United Kingdom
-==============
-
-Here are the basic instructions to install OS OpenData and ONSPD:
-
-1. AREA_SRID in conf/general.yml should be 27700 (as Boundary-Line shapes are
- in OSGB).
-2. Download the latest Code-Point Open, Boundary-Line and ONSPD from
- <http://parlvid.mysociety.org:81/os/>, and save/unzip in the data directory.
-3. Change to the project directory, and create database tables (if you haven't
- already done this):
-
-::
-
- ./manage.py syncdb
- ./manage.py migrate mapit
-
-4. Run the following in order:
-
-::
-
- ./manage.py mapit_generation_create --commit --desc "Initial import."
- ./manage.py loaddata uk
- ./manage.py mapit_UK_import_boundary_line --control=mapit.controls.first-gss --commit `ls ../data/Boundary-Line/Data/*.shp|grep -v high_water`
- # (You can run without --commit to do a dry run.)
- # first-gss in the above assumes the Boundary Line you're importing is
- # October 2010 or later, and uses the new GSS codes.
- ./manage.py mapit_UK_find_parents
- ./manage.py mapit_UK_import_codepoint ../data/Code-Point-Open/*.csv
- ./manage.py mapit_UK_scilly ../data/Code-Point-Open/tr.csv
- ./manage.py mapit_UK_import_nspd_ni_areas
- ./manage.py mapit_UK_import_nspd_ni ../data/ONSPD.csv
- ./manage.py mapit_UK_import_nspd_crown_dependencies ../data/ONSPD.csv
- ./manage.py mapit_generation_activate --commit
-
-For notes on what was done to create generations as you can see on
-http://mapit.mysociety.org/ , see the end of this file.
-
-Notes on future releases
-------------------------
-
-When a new Code-Point is released, you should just be able to run
-mapit_UK_import_codepoint and mapit_UK_scilly; when new ONSPD is out,
-mapit_UK_import_nspd_ni if it's only postcodes that have changed, or
-mapit_UK_import_nspd_ni_areas first if boundary changes too (this is
-incomplete, it doesn't use a control file like mapit_UK_import_boundary_line
-does); when new Boundary-Line, mapit_UK_import_boundary_line and
-mapit_UK_find_parents.
-
-In May 2011, the Northern Ireland Assembly boundaries move to match the current
-Parliamentary boundaries - mapit_UK_import_nspd_ni_areas needs changing to cope
-with that, it currently only creates the current (pre May 2011) boundaries.
-
-You can manually increase the generation_high_id when something is new and
-something else isn't (most commonly, a new Boundary-Line means a new generation
-for Great Britain, and you can then increase the Northern Ireland boundaries
-manually to be in the new generation).
-
-
-Notes on creating what's live
------------------------------
-
-When creating what you see at mapit.mysociety.org, to enable it to have
-pre-2010 election boundaries, I ran the above (or rather, what existed at the
-time, which is not identical) twice, once with 2009-10 Boundary-Line and then
-the 2010-05 edition. I had to write the 2010-05 control file you can see, did
-not re-run mapit_UK_import_codepoint (as no postcodes had changed), and only
-ran the NI stuff the second generation (as we only had current data). The
-commands I basically ran are below.
-
-Even worse, as I had to maintain IDs between our old and new versions of mapit,
-I then matched up all IDs and names using the scripts in bin, manually inserted
-some generation 10 areas (in data) for FixMyStreet and some generation 12 NI
-WMC areas for WriteToThem, and manually added our test/fake areas that used to
-be in code but can now happily sit in the database along with everything else.
-You probably don't need any of that for your own install.
-
-::
-
- # Create inactive generation.
- ./manage.py mapit_UK_import_boundary_line --control=mapit.controls.2009-10 `ls ../../data/Boundary-Line/2009-10/*.shp|grep -v high_water`
- ./manage.py mapit_UK_import_codepoint ../../data/Code-Point-Open-2010-05/*.csv
- ./manage.py mapit_UK_find_parents
- # Not importing NI here, as that only has the current boundaries.
- ./manage.py mapit_UK_scilly ../../data/Code-Point-Open-2010-05/tr.csv
- # Make generation active, add another inactive generation
- ./manage.py mapit_UK_import_boundary_line --control=mapit.controls.2010-05 `ls ../../data/Boundary-Line/2010-05/*.shp|grep -v high_water`
- # mapit_UK_import_codepoint not needed as it's the same and there's no P-in-P tests!
- ./manage.py mapit_UK_find_parents
- ./manage.py mapit_UK_scilly ../../data/Code-Point-Open-2010-05/tr.csv # I doubt the boundaries change! But updates the generation.
- ./manage.py mapit_UK_import_nspd ../../data/NSPD-2010-05.csv # This is now split into two scripts, see below.
- # Make generation active.
-
-
View
108 docs/how-data-is-stored.rst
@@ -1,108 +0,0 @@
-How Data is stored in MapIt
-===========================
-
-MapIt is designed to be very flexible and to be able to represent real world
-situations. This means that geographic areas can have several names and
-identifiers, and can change over time.
-
-When you initially create a MapIt instance most of the complexity can be
-ignored, but as you add and maintain the data you might need to take advantage
-of it.
-
-Areas
------
-
-``Area``s are the central part of MapIt. "California" is an area. The areas you
-store depend on what you want to use MapIt for, but they'll most likely be
-countries, states and other administrative boundaries.
-
-It is possible for an area to have a parent and children, if you need to reflect a
-hierarchy. So a 'county' can contain 'constituencies' and so on. Note that this
-is different to one area covering another - it is only intended for defined
-hierarchies. It is possible to search for overlapping, covering and touching
-areas based only on their geometries.
-
-
-Geometries
---------
-
-An ``Area`` is actually made up of one or more ``Geometries`` - which define a
-boundary. Most areas will be defined by a single geometry, but others may
-require several. For example a country boundary may have a geometry for the
-mainland, and then several smaller geometries for outlying islands. Geometries
-are optional; you can store Areas with no associated geometry.
-
-
-Area Types
----------
-
-Each area has an associated type - this allows you to search just for states, or
-local authorities. You'll create the area types before importing you data.
-
-Each type has a name (eg 'Country') and a code (eg 'CTR'). You can choose both,
-there is no set list.
-
-
-Names
------
-
-An area can have several names, for example "United Kingdom", "United Kingdom of
-Great Britain and Northern Ireland" and "UK" are all names for the same thing.
-You could also add in "Royaume-Uni" which is the French.
-
-Each name you add has a ``NameType`` - For the above examples you might use
-'common', 'full', 'abbreviated' and 'french'.
-
-To begin with you'll probably only use one ``NameType``.
-
-
-Codes
------
-
-As well as having names areas often have codes. For example the UK has the `ISO
-3166 alpha 2 <http://en.wikipedia.org/wiki/ISO_3166>'_ code ``GB``. It is also
-common for various bodies to assign codes.
-
-To begin with you can ignore the codes, but like the names they'll probably come
-in handy after a while.
-
-
-Generations
------------
-
-Areas change over time, and by using ``generations`` you can store historic
-areas for reference without having them appear in contemporary searches. You
-can, if you want, specifically search particular generations.
-
-For example say you have the boundaries of electoral areas - constituencies in
-the UK. Between elections these are often adjusted to account for population
-movements. You could store the 2007 boundaries in generation 1, the 2010
-boundaries in generation 2 etc. If an area has not changed then it is in both
-the generations.
-
-There is a single ``current`` generation which is the one that is used by
-default when searching - it is the most recent active one. It is also possible
-to have ``inactive`` generations that don't appear in search results - this
-allows you to add them to the system in advance but only activate them when
-you're ready.
-
-
-Postcodes
----------
-
-Postcodes are named items, that optionally point to a location. In the UK we have
-a large set of postcodes such as 'SW1A 1AA' that map to a latitude and longitude
-(the centroid of all the addresses that use that postcode to be precise); but we
-also hold postcodes of Crown Dependencies for which we do not know the location -
-but we store them anyway, so we can at least say whether an entered postcode is
-e.g. a valid Jersey postcode or not.
-
-The Postcodes model could be used to store any sort of named points, not
-necessarily postcodes or zip codes.
-
-
-There is also a many-to-many table between Postcodes and Areas. This is for the
-case where you have an Area without a geometry, but have some sort of list that
-maps which postcodes are in which area. This is the case in the UK for Northern
-Ireland Parliamentary constituencies, for example.
-
View
159 docs/import-from-gadm.rst
@@ -1,159 +0,0 @@
-Importing boundaries from GADM.org
-==================================
-
-`GADM <http://www.gadm.org/>`_ is a database of administrative areas from
-around the world. If you are setting up a MapIt for your country then it is
-likely that this is a good starting point. At mySociety it is what we used for
-Kenya and Nigeria.
-
-Note that the GADM data is freely available for academic and other
-non-commercial use. Redistribution, or commercial use, is not allowed without
-prior permission from them.
-
-These are step by step instructions for loading all the areas for a country
-(we'll use Nigeria) into MapIt:
-
-
-Find and download the boundaries
---------------------------------
-
-1) Go to the download page: http://www.gadm.org/country
-2) Select the country (in our case Nigeria)
-3) Select the "Google Earth .kmz" format
-4) Click "OK" to show the available downloads
-5) Download all the "levels" to your computer - for Nigeria there are levels 0,
- 1 and 2.
-
-[ TODO: Talk about shapefile downloads here too - some have more data than KML. ]
-
-Preview the downloaded files
-----------------------------
-
-Once the data files have been downloaded you can use the free Google Earth
-software to view the boundaries. It is available for download from:
-http://www.google.com/earth/explore/products/desktop.html
-
-
-Prepare the files for import
-----------------------------
-
-1) Collect all your files in one folder - we'll use ``/tmp/gadm_data``
-2) Convert the files to kml (kmz is just kml gzipped):
-
-::
-
- cd /tmp/gadm_data
-
- # decompress all the kmz files
- unzip '*.kmz'
-
-
-Work out the area types you'll need
------------------------------------
-
-There will be several different types of area you'll want to store. For example
-there will be the country boundary, and there may also be states or counties.
-Look in the kml files to find the different areas represented there.
-
-This command will list them all:
-
-::
-
- grep '<description>' *.kml | sort | uniq -c | grep -v 'href='
-
-For the Nigerian kml files we get this output:
-
-::
-
- 1 NGA_adm0.kml:<description><![CDATA[NGA]]></description>
- 37 NGA_adm1.kml:<description><![CDATA[State]]></description>
- 1 NGA_adm1.kml:<description><![CDATA[Water body]]></description>
- 775 NGA_adm2.kml:<description><![CDATA[Local Authority]]></description>
-
-The bit we're looking for is in the ``CDATA[[...]]`` block. The number at the
-start of the line in the number of occurrences.
-
-Note that there is a single 'Water Body' in one of the kml files - using a text
-editor we'll remove that manually as it is not relevant to us. This is done by
-finding the 'Water body' text and then deleting everything from and including
-the '<Placemark>' before it and up to and including the closing '</Placemark>'
-after it.
-
-Alternatively you could import using the steps below and then find it in the
-admin interface and delete it.
-
-
-Prepare the MapIt install for the import
-----------------------------------------
-
-We'll assume that you are using a locally running instance of MapIt. If not
-please change the URLs to match your setup.
-
-1) Install MapIt as per the instructions in ``INSTALL.rst``
-2) Start the dev server.
-3) We're assuming that your database is empty. If this is not the case you may
- have some conflicts.
-4) Either visit the admin at http://127.0.0.1:8000/admin and add a generation
- there, with description 'GADM version 2.0', or run the following (which does
- the same thing):
-
-::
-
- ./manage.py mapit_generation_create --desc='GADM version 2.0' --commit
-
-5) (optional) You can add the various Types in the admin interface now, or be
- prompted for them when you run the import script. If you want to use the
- admin interface, we'll want a country - Nigeria with code N; a name type -
- 'gadm' with description 'GADM English Name'; and appropriate area types, CTR
- "Country", STA "State", and LGA "Local Government Area".
-
-Import the kml
---------------
-
-There is an import script that will look at the kml file and create entries from
-it. We'll call it as follows:
-
-::
-
- ./manage.py mapit_import \
- --country_code N \
- --generation_id 1 \
- --area_type_code CTR \
- --name_type_code gadm \
- --commit \
- /tmp/gadm_data/NGA_adm0.kml
-
-You will be prompted to provide descriptions for the codes if you haven't
-created them; we can use "GADM English Name" for gadm, "Country" for CTR,
-"State" for STA, "Local Government Area" for LGA, and "Nigeria" for 'N'.
-
-Repeat the above line for all your data files changing the path to the file and
-the ``--area_type_code`` to suit your import. You may also need to use a
-different value for ``--generation_id`` if this is not a fresh MapIt install.
-
-If you want to try the import without committing to the database don't specify
-the ``--commit`` switch.
-
-Activate the generation
------------------------
-
-Once you are happy that the data is correct activate the generation in the
-admin interface, or run:
-
-::
-
- ./manage.py mapit_generation_activate --commit
-
-
-Shapefiles
-----------
-
-The same import script can import shapefiles too. You might need the extra
-command line parameter of --encoding if the encoding of the shapefile is not
-UTF-8 (GADM is sometimes ISO-8859-1, for example).
-
-You will also need to know which field in the shapefile contains the name of
-the area, and specify it with the --name_field parameter. If you run without
-this parameter and 'Name' doesn't work, the program will output a list of
-possible choices that it could be.
-
Please sign in to comment.
Something went wrong with that request. Please try again.