NYPL Map Warper
Ruby HTML JavaScript CSS Shell
Latest commit 5f2b967 Feb 14, 2017 @mgiraldo mgiraldo 👾 added CORS
Permalink
Failed to load latest commit information.
app 👾 added CORS Feb 14, 2017
bin Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015
config Fix for ssl verification error with mail setup, and in case of a mail… Jul 8, 2015
db 👾 started with saving rectify data to db May 12, 2016
lib vagrant uses 8GB ram default / readme changes / setting rails_env in … Mar 9, 2016
log Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015
public Modify robots.txt for the warper relative url root Jul 6, 2015
test Fix for bug where inset maps were getting the same status as the orig… Jul 23, 2015
vendor/assets Timeslider for maps and layer search. Tied in with sorting table head… Jun 21, 2015
.gitignore Ignore any compiled assets Jun 30, 2015
CHANGES.md links to main authors May 19, 2016
Capfile Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015
Gemfile Upgrade to 3.0.0 release Activerecord postis adapter gem Jun 26, 2015
Gemfile.lock 👾 started with saving rectify data to db May 12, 2016
MIT-LICENSE re-add MIT license Jul 7, 2015
README.md Update README.md May 20, 2016
README_API.md Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015
Rakefile Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015
Vagrantfile vagrant uses 8GB ram default / readme changes / setting rails_env in … Mar 9, 2016
config.ru Mapwarper master overwrite. Import and overwrite with main mapwarper … Apr 29, 2015

README.md

NYPL Map Warper

This software has been developed by Topomancy LLC with the support and collaboration of the New York Public Library.

You can see this code in action at http://maps.nypl.org/.


Consult the wiki for for more documentation including a report detailing the changes in this version compared with the previous version. https://github.com/NYPL/nypl-warper/wiki


The NYPL Map Warper is an open source map geo-rectification, warping and georeferencing application. It enables a user to browse and explore the NYPL's historical map collection, and by placing control points on a reference map and the image, to warp it, to stretch it to fit.

Map Warper screenshot of main interface

The application can be seen in use at http://maps.nypl.org

The application is a web based crowdsourced geospatial project that enables people and organisations to collaboratively publish images of maps online and digitize and extract vector information from them.

Users rectify, warp or stretch images of historical maps with a reference basemap, assigning locations on image and map that line up with each other. Often these historical maps were in big paper books, and so for the first time they can be stitched together and shown as a whole, in digital format.

Users can crop around the maps, and join them together into layers.

By georeferencing the images, they can be warped or georectified to match the locations in space, and used in GIS software and other services. One such use of these warped maps is an application that that helps people digitize, that is, trace over the maps to extract information from them. For example, buildings in 18th Century Manhattan, details changing land use, building type etc. This application is called the Digitizer.

The application runs as a Ruby on Rails application using a number of open source geospatial libraries and technologies, including PostGIS, Mapserver, Geoserver, and GDAL tools.

The resulting maps can be exported as a PNG, GeoTIFF, WMS, Tiles, and KML for use in many different applications.

Features

  • Find and search maps by geography
  • Adding control points to maps side by side
  • Crop maps
  • User commenting on maps
  • Align maps from similar
  • Login via Github / Twitter / etc
  • OR signup with email and password
  • Export as GeoTiff, PNG, WMS, Tile, KML etc
  • Preview in Google Earth and Google Maps
  • Map Favourites
  • Social media sharing
  • Bibliographic metatadata creation and export support
  • Multiple georectfication options
  • Control point from files import
  • API
  • Admin tools include
    • User statistics
    • Activity monitoring
    • User administration, disabling
    • Roles management (editor, developer, admin etc)
    • Batch Imports
    • User Flags
    • User Statistics
    • Revisions and Rollbacks

Note on code and branches

Unmaintained branches exist for older systems and setups

  • Rails 2.3 and Ruby 1.8.4 - See the rails2 branch

Ruby & Rails

  • Rails 4.2
  • Ruby 1.9+

Database

  • Postgresql 9.1 (or 9.1, 8.4)
  • Postgis 2 (may work with 1.5)

Installation Dependencies

Check out the Vagrant section lower down in the readme if you want to get started quickly.

on Ubuntu 14.04 LTS

apt-get install -y ruby ruby-dev postgresql-9.3-postgis-2.1 postgresql-server-dev-all postgresql-contrib build-essential git-core libxml2-dev libxslt-dev imagemagick libmapserver1 gdal-bin libgdal-dev ruby-mapscript nodejs

Due to a bug with the gdal gem, you may need to disable a few flags from your ruby rbconfig.rb see https://github.com/zhm/gdal-ruby/issues/4 for more information

Then install the gem files using bundler

bundle install

Configuration

Create and configure the following files

  • config/secrets.yml
  • config/database.yml
  • config/application.yml

In addition have a look in config/initializers/application_config.rbfor some other paths and variables, and config/initializers/devise.rb for devise and omniauth and also config/environments/development.rb if you rather not use the /warper relative path.

Database creation

Create a postgis database

sudo -u postgres createdb mapwarper_development
psql mapwarper_development -c "create extension postgis;"
RAILS_ENV=development bundle exec rake db:migrate

Database initialization

Creating a new user

user = User.new
user.login = "super"
user.email = "super@superxyz123.com"
user.password = "your_password"
user.password_confirmation = "your_password"
user.confirmed_at = Time.now
user.save

role = Role.find_by_name('super user')

permission  = Permission.new
permission.role = role
permission.user = user
permission.save

role = Role.find_by_name('administrator')
permission = Permission.new
permission.role = role
permission.user = user
permission.save

Development

Via Vagrant, there is a Vagrantfile you can use which uses the /lib/vagrant/provision.sh provision script. Note: the default Vagrantfile is configured to take 8GB RAM. To use this file type:

vagrant up

to get and install the virtual machine - this will also install the libraries and depencies and ruby gems for mapwarper into the virtual machine. See /lib/vagrant/provision.sh for more details about this process.

After that runs, type vagrant ssh to login and then you can:

cd /srv/mapwarper
rails c

Create a user in the console, as shown above and then exit. Then start the server in a relative url root use thin:

RAILS_ENV=development bundle exec thin --prefix=/warper start

and then browse to http://localhost:3000/warper/

In non-Vagrant circumstances you may want to run it just using rails s.

Import a test map

This codebase only supports maps from NYPL via the Digital Collections API (use that link to get an account and token). Fortunately, many of our maps are Public Domain so you can test the warper locally (non PD maps are not downloadable in high-res from the API). The warper imports maps (subject to throttling limits) based on the UUID visible in the item page (98b7f36a-3d7a-6c65-e040-e00a18064b00 in this case):

  1. add the generated API token to /config/nypl_repo.yml
  2. start the server and log in with a super user
  3. go to Admin / Imports
  4. Click Add New Import
  5. Select Import Type: Map and type in the UUID: a66456f6-3860-95d5-e040-e00a18065ecc and click Create
  6. Click Start Import

This will take a few minutes to pull the map. You now have a map you can work with and test functionality!

Deployment instructions

The system can use capistrano for deployment.

Making Code Changes and Deployment

There are two main changes to the system. Changes to checked in code, and changes to configuration files. Some configuration files (/config/*) are not stored in the repository. To change these you need to ssh into the server and change the file. Some .example files are provided for reference.

Changes to checked in code.

Develop, patch the code locally and get it running. Probably the easiest way to start developing is via Vagrant, see above for more details. Once you have Vagrant running, these are some useful commands to make sure updates are reflected:

If assets had been changed (images, JS, CSS etc.) or if you just want to be safe:

RAILS_ENV=production rake assets:clobber

RAILS_ENV=production rake assets:precompile

If necessary to run rails console commands:

RAILS_ENV=production rails console

To restart the server:

touch tmp/restart.txt

To restart the Apache server (not needed in almost all cases):

sudo service apache2 restart

Changes to config file:
  1. ssh into the server
  2. Find the desired config file
  3. Edit the file
  4. Restart the warper: touch tmp/restart.txt

API

See README_API.md for API details