CGImap

Note: this is a copy of https://github.com/zerebubuth/openstreetmap-cgimap which has been slightly customized to work with the Kartta Labs suite of applications.

Overview

CGImap is a C++ implementation of some parts of the OpenStreetMap API as an FCGI process. The rails implementation of the "map" call had a few problems with memory - it uses a lot of it and there is a leak which makes it annoying to use in long-running environments, such as the main OSM server.

CGImap attempts to address these memory problems and makes it easier to optimise the queries, something which is also a bit of a pain to do in Rails.

Currently, CGImap implements:

• the "map" API call,
• single node, way and relation fetches,
• multiple node, way and relation fetches,
• the "full" way and relation calls and
• changset metadata downloads, including discussions,
• single node, way and relation history calls,
• single node, way and relation specific version fetches,
• multiple node, way and relation specific version fetches,
• relations for element (node, way, relation),
• ways for node,
• create changeset,
• update changeset,
• close changeset,
• changeset downloads,
• changeset uploads.

Note that changeset metadata isn't the same thing as the changeset download (i.e: osmChange data).

Requirements

CGImap uses a PostgreSQL server for the APIDB backend. A minimum server version of 9.4 is required.

CGImap depends on the following libraries. Versions used during development are in brackets. Other versions may work, but YMMV.

• libxml2 (2.9.3+dfsg1-1ubuntu0.2)
• libpqxx-4.0 (4.0.1+dfsg-3ubuntu2)
• libfcgi (2.4.0-8.3)
• libboost (1.58.0+dfsg-5ubuntu3.1)
• libcrypto++ (5.6.1-9)

If you're running a Debian or Ubuntu system these can be installed using the following command:

sudo apt-get install libxml2-dev libpqxx-dev libfcgi-dev \
  libboost-dev libboost-program-options-dev libboost-filesystem-dev \
  libboost-system-dev libboost-locale-dev libmemcached-dev \
  libcrypto++-dev

Note that C++11 is required to build CGImap.

The build system used is GNU Make, using pkg-config to provide some of the flags.

Note that the full set of packages needed from a fresh install (tested with Ubuntu 18.04) - you may already have many or all of these - is:

sudo apt-get install git build-essential automake autoconf libtool

If you want to enable the OSM API JSON format, then you'll also need:

sudo apt-get install libyajl-dev

To build the system from scratch, first check out the source code (skip this step if you've already got the source):

git clone git://github.com/zerebubuth/openstreetmap-cgimap.git

Then change to the source code directory to configure and build:

cd openstreetmap-cgimap/
./autogen.sh
./configure [--enable-yajl]
make

You should now have a "./openstreetmap-cgimap" executable in the current directory.

Setup

To run CGImap binary, use the command:

./openstreetmap-cgimap --socket :54321 --backend apidb \
  --dbname openstreetmap

Note: CGImap has to be used with a fastcgi enabled server like lighttpd, apache2 etc. See the instructions below to use CGImap with lighttpd or apache2.

A sample lighttd.conf file is provided, which can be used to test. To test CGImap with lighttpd you will need to install lighttpd:

sudo apt-get install lighttpd

Edit the supplied lighttpd.config file to include your CGImap path and run it with the lighttpd like

/usr/sbin/lighttpd -f lighttpd.conf

You can then access the running instance at http://localhost:31337/api/0.6/map?bbox=...

The api.osm.org instance runs CGImap as a daemon and Apache with mod_fastcgi_handler. An init.d script to run CGImap as a daemon is supplied in scripts/cgimap.init. To use it modify the paths and environment variables to suit your installation, copy it to /etc/init.d/cgibin and change the mode to 755, and owner:group to root:root.

An example of this can be found in OSM Chef.

Typically you will need to modify the database connection parameters and path to the executable. See ./openstreetmap-cgimap --help for a list of options. To convert a command line option to an environment variable append CGIMAP_ to the option and capatalize it. For example, the option --dbname becomes the environment variable CGIMAP_DBNAME.

Fcgi programs can be deployed with Apache using mod_fastcgi_handler, mod_fcgid, mod_fastcgi, and on recent versions mod_proxy_fcgi. A sample Apache configuration file that will work in conjunction with CGImap as a daemon is supplied in scripts/cgibin.conf. To use this on a Ubuntu-based system you need to copy the cofiguration to where Apache will read it and create an api directory:

sudo cp scripts/cgimap.conf /etc/apache2/sites-available/cgimap
sudo chmod 644 /etc/apache2/sites-available/cgimap
sudo chown root:root /etc/apache2/sites-available/cgimap
sudo mkdir /var/www/api
sudo a2ensite cgimap
sudo service apache2 restart

The apache modules mod_proxy and mod_fastcgi_handler must also be enabled.

Database Permissions

The read only apidb backend requires permissions to SELECT on the Postgres server. OAuth and update database connections require additional permissions to INSERT/UPDATE/DELETE data, as well as creating temporary tables.

It is recommended that a separate account is created for CGImap to avoid any possibility of data corruption. Care has been taken programming CGImap but, as with most C++ applications, there is the chance of an exploitable flaw leading to complete pwnage.

Testing

To run the test suite using make check you will need additional packages installed:

sudo apt-get install postgresql postgresql-contrib postgis

And you will need to be able to create databases as your user:

sudo -u postgres createuser -s $USER

Formatting

The CGImap code is formatted using clang-format in a style which is based on the "LLVM style" which ships with clang-format. Note that version 3.6 or later is needed to avoid introducing problems with formatting try-catch blocks. To enable an automatic reformatting option, provide the --with-clang-format option to configure and then reformatting can be done across the whole set of source files by running:

make clang-format

Ideally, this should be done before committing each set of changes.

Acknowledgements

CGImap contains code from and is partly based on the following:

• modosmapi by d40cht and japplebyalis.
• quad_tile.c by TomH.
• GNU CGICC by Stephen F. Booth and Sebastien Diaz.
• libxml++ by The libxml++ development team