High quality mapping for the outdoors
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is even with grough-outdoors:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
archive
bin
product
source
viewer
volatile
.gitattributes
.gitignore
GPL
LICENCE
README.md
Vagrantfile

README.md

grough-map

High quality mapping for the outdoors. Map tiles are generated using composite data from a number of sources (Ordnance Survey, OpenStreetMap, Environment Agency, Natural England, Natural Resources Wales, etc.). Data and tools are used within Ubuntu LTS through Vagrant.

See the grough map site for less technical information.

We're making the code available straight away, but it's probably worth considering that documentation wasn't high on our list of priorities before the first full tileset was made available. We'll improve this over time, and feel free to ask questions.

Be warned that running the build process on a single computer for the whole of Great Britain, is likely to take several days. Processing the transport network and buildings are the most intensive steps. We use Amazon EC2 to distribute processing; the Vagrantfile is configured for the AWS provider, if you also specify some environment variables. See the Vagrantfile and AWS provider documentation for details.

Preparation...

  1. Install Oracle VirtualBox.
  2. Install Vagrant.
  3. Clone from GitHub.
  4. Provision the machine
    1. Provision the machine with 'vagrant up'.
    2. Log into the machine with 'vagrant ssh'.
    3. Run each build step.

The SSH command will require access to an ssh executable. If you're using Git BASH, you should be fine.

Basic build process

Load in the basic schema required...

  • gm-restore-schema

Fetch the required data from the web...

  • gm-download-grid
  • gm-download-zone (zones include national parks, CRoW boundaries, and highway authorities, etc.)
  • gm-download-os (requires text of an OS OpenData download email, which you'll be prompted for)
  • gm-download-prow (TODO: This command doesn't yet download anything!)

The British National Grid is required and should be imported straight away...

  • gm-import-grid

Output tiles are sliced to provide levels of details, using the pyramid generated by...

  • gm-build-pyramids

The OSM database is required in full for almost all commands, and so is kept within the database constantly, unlike the different layers from other sources...

  • gm-import-osm

Build each composite map layer, for which order is important where dependencies are shown...

  • gm-build-zone
  • gm-build-transport (invokes gm-import-prow, requires zones)
  • gm-build-buildings
  • gm-build-surface
  • gm-build-watercourses (requires surfaces to be built first)
  • gm-build-features
  • gm-build-places (requires watercourses to be built first)
  • gm-build-cartography

For each tile to be generated...

  • gm-tile <tile>

More advanced commands

LiDAR processing

LiDAR data can be processed using these commands for each tile...

  • gm-build-obstructions <tile> (for each LiDAR tile, highly intensive process)
  • gm-build-terrain <tile> (for each tile, but will be automatic during gm-tile when missing)

LiDAR tiles can be downloaded manually, or will be pulled when required...

  • gm-download-eagg <tile>
  • gm-download-nrw <tile>

Importing data

Data is imported using these commands, but each build process should call these automatically if the data is not already within the build database.

  • gm-import-zone
  • gm-import-os
  • gm-import-osm
  • gm-import-prow (requires zones to be built first)

Each build process contains a list of required data, which calls the relevant import command. You can call the require function yourself to import a data provider, data product, or a subset. New tables added by the import command are tracked, and will then remain in the database until the clean command is issued. All data source tables are prefixed with _src when they are considered volatile.

  • gm-require-db <provider> <product> <subset>
  • gm-clean-sources
vagrant@map-system:~$ gm-require-db os opmplc railway_station
This tool requires data from 'os' product 'opmplc' subset 'railway_station'... Testing for data...
 + Attempting to satisfy requirements...
    Import script found. Attempting to run...
Preparing to import OS OpenData products...
-----------------------------------
--> Extracting archives...
-----------------------------------

   ...lots of files...
   
55 archives were successfully processed.
--> Import complete.
 + Found _src_os_opmplc_railway_station. Testing for data...
    Data found. Requirement satisfied.
I am now happy with the data found.

Source attribution

A table of source attributions is built using JSON files detailing the licence constraints and copyright statements. Dates for data are determined automatically if possible. An attribution JSON file can be added to the source table, and the full list of sources can be output using the following commands:

  • gm-require-attribution <json path>
  • gm-build-sources [HTML|TXT]

The full list will be placed in the product directory, alongside mapping tiles generated. The list should regenerate itself each time a source is updated, so you'll likely never need to invoke gm-sources yourself.

  ORDNANCE SURVEY (http://www.ordnancesurvey.co.uk/)
  National Mapping Agency

   - OS Open Roads
     Contains OS data © Crown copyright and database right 2015.

     Licensed under Open Government Licence v3 (OGL).
     http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

     Data archived on 2015-10-29.
	 
  ...

An example output when adding data from OS OpenMap Local shows the date determined from the archive ZIP files.

--------------
  Dataset:           OS OpenMap Local
  Source attribution
    Organisation:    Ordnance Survey
    Category:        National Mapping Agency
    URL:             http://www.ordnancesurvey.co.uk/
    Licence type:    OGLv3
    Statement:       Contains OS data © Crown copyright and database right 2015.
    Date source:     archive
      using:         opmplc/*.zip
      returning:     to_date('20150316', 'YYYYMMDD')
    Disclaimers:     ARRAY[]
    Constraints:     ARRAY[]
--------------

Licence

All the code in this repository is licensed under the GNU General Public License v3. You are free to use and distribute it, including for commercial use, but any changes you make must be made available under the GPL with build and install instructions.

OpenStreetMap data is used by these tools, and as such many of the map layers (but not all) are covered by the Open Database License, which may require you to share improvements you make to the database.