Part of osmose that runs the analysis, and send the results to the frontend.
Clone or download
Latest commit c033d02 Oct 16, 2018
Permalink
Failed to load latest commit information.
.github Add github issue_template.md Feb 17, 2018
.tx Update po Makefile to pull translations from transifex May 2, 2014
analysers Amend desc in analyser_merge_healthcare_FR_finess Oct 14, 2018
dictionaries Update INSEE data #241 Nov 9, 2017
docker add dependencies of modules later installed by pip Oct 9, 2018
mapcss mapcss: capture a tag key in all case #328 Sep 30, 2018
merge_data Fix overlapping class in healthcare_FR_finess.mapping.csv #347 Oct 10, 2018
modules Support plain file in SourceVersion computer Oct 5, 2018
osmconvert Re-compile osmconvert on Debian stable (squeeze) Dec 16, 2012
osmosis Setup resume for Analyser_Osmosis #166 Jan 9, 2018
plugins Remove rule way[tracktype>=2][surface=asphalt] in Bicycle.mapcss 3032… Oct 10, 2018
po Update translations Oct 16, 2018
tests Update xml results for sax tests Oct 16, 2018
tools fix pgsql version detection Oct 9, 2018
.gitignore add docker work directory to gitignore Sep 2, 2018
LICENSE Add LICENSE file Jan 29, 2013
README.md package build-essential is required for pip later Oct 9, 2018
cron-launcher Fix cron-launcher to read file line per line, allowing options to be … Aug 20, 2013
launcher Fix osmose_run.py path in launcher Sep 18, 2012
local-launcher Add launcher to launch on local machine, and wait for the end of the … Apr 13, 2013
osmose_config.py Set country code for western_sahara Oct 7, 2018
osmose_config_password-example.py Add docker-compose Jan 27, 2018
osmose_config_password.py Define function set_password(), needed by osmose_config.py Aug 27, 2011
osmose_run.py Add option --skip-analyser and --skip-upload to osmose_run Oct 7, 2018
requirements-dev-python3.txt Add support of MapCSS Jan 18, 2018
requirements-dev.txt Improve installation detail in README and use pip for python deps Jul 5, 2015
requirements.txt Add new dependency python-requests to requirements.txt and README Oct 4, 2018
setup_bdd.sh Remove old references to osm2pgsql Sep 10, 2014

README.md

Backend part of Osmose QA tool

This is the part of osmose [http://osmose.openstreetmap.fr] which analyses OSM and send results to frontend. This works as following:

  • an .osm.bz2 or .osm.pbf extract is downloaded from a path
  • downloaded file is converted to .osm, with bunzip2 or osmconvert
  • if necessary, an osmosis dump is generated in a local database
  • analyses are run directly on .osm file, or on the database
  • analyses are stored on a local webserver, and a link is sent to the frontend so that it can download the results
  • temporary extract files and database are purged

Installation Python

Osmose QA backend requires python > 2.6.

Setup system dependencies (Debian Stretch)

apt install python

You can install python dependencies in the system or in a virtualenv.

In the system install the folowing packages:

apt install python-dateutil python-imposm-parser python-lockfile python-polib python-poster python-psycopg2 python-shapely python-regex python-requests

Alternatively instal python-virtualenv and create a new virtualenv.

Setup system dependencies (Debian Stretch)

apt install build-essential git python-dev python-virtualenv libpq-dev protobuf-compiler libprotobuf-dev

Create a python virtualenv, active it and install python dependencies

virtualenv --python=python2.7 osmose-backend-venv
source osmose-backend-venv/bin/activate
pip install -r requirements.txt

To run tests, additional packages are needed.

pip install -r requirements-dev.txt

Installation Database

Setup system dependencies (Debian Stretch)

apt install postgresql-9.6 postgresql-contrib-9.6 postgresql-9.6-postgis-2.1

As postgres user:

createuser osmose
# Set your own password
psql -c "ALTER ROLE osmose WITH PASSWORD '-osmose-';"
createdb -E UTF8 -T template0 -O osmose osmose
# Enable extensions
psql -c "CREATE extension hstore; CREATE extension fuzzystrmatch; CREATE extension unaccent; CREATE extension postgis;" osmose
psql -c "GRANT SELECT,UPDATE,DELETE ON TABLE spatial_ref_sys TO osmose;" osmose
psql -c "GRANT SELECT,UPDATE,DELETE,INSERT ON TABLE geometry_columns TO osmose;" osmose

Dependencies

Java JRE for osmosis (Ubuntu Server 14.04):

apt install openjdk-7-jre-headless

osmosis is installed in osmosis/osmosis-0.44/. osmconvert is installed in osmconvert/.

Configuration

A few paths are hardcoded in modules/config.py, and should be adapted or created.

  • dir_osmose is the path of where osmose is installed
  • dir_work is where extracts are stored, and results generated.
  • url_frontend_update is the url used to send results generated by analyses

The local postgresql database should be configured in osmose_config.py:

  • db_base = osmose # database name
  • db_user = osmose # database user
  • db_password = # database password if needed
  • db_host = # database hostname if needed

You may want to include this info in ~/.pgpass to avoid entering the database password while processing the files.

See https://wiki.postgresql.org/wiki/Pgpass for more info.

Fetching josm translations

JOSM translations are used by some MapCSS plugins, and can be retrieved by bzr:

apt install bzr
cd po/josm
bzr checkout --lightweight lp:~openstreetmap/josm/josm_trans

Run

Look at the osmose_run.py help for options

osmose_run.py -h

Connection to the "official" frontend at http://osmose.openstreetmap.fr

When you have configured the backend for the country you want to add, please send an email to osmose-contact@openstreetmap.fr. We will then send you the password to use to connect to the frontend.

Run Tests

Setup a ~/.pgpass file to allow pgsql to connect to the test database without asking for password:

hostname:port:database:username:password

Create a test database osmose_test and initialize it:

createdb -O fred osmose_test
psql -c "CREATE extension hstore; CREATE extension fuzzystrmatch; CREATE extension unaccent; CREATE extension postgis;" osmose_test
psql -c "GRANT SELECT,UPDATE,DELETE ON TABLE spatial_ref_sys TO osmose;" osmose_test
psql -c "GRANT SELECT,UPDATE,DELETE,INSERT ON TABLE geometry_columns TO osmose;" osmose_test

Finally run the tests:

nosetests analysers/Analyser_Osmosis.py
nosetests analysers/analyser_sax.py