Sceptred

This royal throne of kings, this sceptred isle,
This earth of majesty, this seat of Mars,
This other Eden, demi-paradise,
This fortress built by Nature for her self
  — William Shakespeare, Richard II

Sceptred is a project to model the island of Great Britain in 3D, using open-source data.

Overview

Sceptred is a web application, split between a browser-based client and a server.

• The client is written in Typescript, and uses Three.js to render the terrain in a WebGL context.
• The server is written in Go, and is ultimately backed by an adaptation of the Ordnance Survey's Terrain 50 dataset in a MongoDB database.

The project is designed to use modern web development tools & practices. The client is compiled into JavaScript with Webpack and Babel, and has automated testing in Jest. The server is packaged up & deployed with Docker, and the entire repo has integrations with Travis, Codecov and Sentry.

Developing

Before you start

Prerequisites: Go, Node + npm, Glide, a MongoDB instance

To install Webpack and associated build & test functionality, run:

$ npm install

To install the vendor files, it is recommended you use Glide:

$ glide install

To install the database, download the database dump (a copy is here, about 200MB in size). Unzip and cd into the dump folder. Then install using mongorestore:

$ mongorestore --db sceptred ./sceptred

You may need to set the --host, --username and --password flags to correctly connect to your MongoDB instance.

All of these only needs to be done once, before you first run.

Development server & watcher

To run the dev server, run:

$ npm run serve

To actively develop on the TypeScript, i.e. run Webpack in watch mode, run:

$ npm run watch

Testing

Client-side testing is done with Jest, and can be run as:

$ npm run test:js

Server-side testing is done with Go's testing package, and can be run as:

$ npm run test:go

All tests can be run with:

$ npm run test

Deploying

Building a Docker container is straightforward:

$ docker build -t sceptred .

And to run locally:

$ docker run -it -p 8000:8000 sceptred

A GCloud deploy (if you have Google App Engine set up) is done with:

$ gcloud app deploy

Architecture

Client

The client consists of:

• React Components for the form and the map renderer (app.tsx and map.tsx)
• A supporting library of modules, the most important of which represents the world (world.ts). Also paying attention to are the libraries for handling the OS Grid (grid.ts) and for parsing data into correct geometry objects (data.ts)

These are located in client/src/js. The biggest dependency is Three.js but we also rely on React and ReactDOM for the UI. The colouring of the terrain is handled by chroma-js.

The CSS is currently handled by Spectre.css (no relation).

Server

The server is a simple API server written in Go with the help of Echo. This handles the homepage, and delivering each tile via a JSON API.

Database

The database is a simple MongoDB database. There is one collection of ~2,800 Documents, each covering a 10x10 grid square in the UK. The format for each is:

{
    "_id" : "NT27",
    "meta" : {
        "squareSize" : 50,
        "gridReference" : "NT27"
    },
    "heights" : [
        [8, 79, 81, 76, 79, 79, ...],
        ...
    ]
}

To rebuild the database, you will need two sets of source files:

• The Ordnance Survey's OS Terrain 50 dataset, which can be ordered here. Make sure the ASCII Grid format is selected. After downloading, unzip the single zip file and place all of its contents in the directory terrain/asc.
• A shapefile for the United Kingdom's coastline - I have used the GBR_adm0.shp shapefile from http://www.diva-gis.org/gdata, which can be reached by selecting United Kingdom / Administrative areas. I have not tested the script with other shapefiles. Download and unzip the shapefiles into terrain/shp.

Once installed and unzipped into the above location, cd into terrain/scripts. Then install the Python libraries (virtualenv is recommended):

$ pip install -r requirements.txt

And then run the script:

$ ./process_data.py

On my MacBook Pro, converting the entire UK into the MongoDB database takes about 5-6 minutes.

Copyright

The source code for this project is licensed under the MIT License.

Mapping data contains Ordnance Survey data © Crown copyright and database right 2017.