Instructions for running with own OSM dataset

[GPistre]

I followed the instructions for building with docker & own osm dataset at README.md and docker/README.md,

• added my own osm file to docker/data
• created a new app.config file from the sample at https://github.com/GIScience/openrouteservice/blob/master/openrouteservice/WebContent/WEB-INF/app.config.sample, replacing heidelberg.osm.gz with my own file
• edited docker-compose.yml to reference the new app.config
• ran docker-compose up

running docker logs ors-core results in this output :

22-Dec-2017 14:04:31.084 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.5.24
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Nov 27 2017 13:05:30 UTC
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.5.24.0
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            4.10.0-40-generic
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/lib/jvm/java-8-openjdk-amd64/jre
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_151-8u151-b12-1~deb9u1-b12
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /usr/local/tomcat
22-Dec-2017 14:04:31.087 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /usr/local/tomcat
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:TargetSurvivorRatio=75
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:SurvivorRatio=64
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxTenuringThreshold=3
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseConcMarkSweepGC
22-Dec-2017 14:04:31.088 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseParNewGC
22-Dec-2017 14:04:31.089 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ParallelGCThreads=4
22-Dec-2017 14:04:31.089 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms1g
22-Dec-2017 14:04:31.089 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx2g
22-Dec-2017 14:04:31.089 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
22-Dec-2017 14:04:31.089 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based Apache Tomcat Native library [1.2.16] using APR version [1.5.2].
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
22-Dec-2017 14:04:31.090 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
22-Dec-2017 14:04:31.094 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.0f  25 May 2017]
22-Dec-2017 14:04:31.178 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
22-Dec-2017 14:04:31.189 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
22-Dec-2017 14:04:31.194 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
22-Dec-2017 14:04:31.195 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
22-Dec-2017 14:04:31.196 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 459 ms
22-Dec-2017 14:04:31.211 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
22-Dec-2017 14:04:31.212 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.24
22-Dec-2017 14:04:31.219 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
22-Dec-2017 14:04:31.245 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
22-Dec-2017 14:04:31.266 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 70 ms
22-Dec-2017 14:04:41.236 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/usr/local/tomcat/webapps/ors.war]
22-Dec-2017 14:04:42.568 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] -  Total - 1019 MB, Free - 770.22 MB, Max: 2.00 GB, Used - 248.78 MB
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] -       
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] - ====> Initializing profiles from 'data/heidelberg.osm.gz' (2 threads) ...
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] -                               
22-Dec-2017 14:04:42.746 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/local/tomcat/webapps/ors.war] has finished in [1,509] ms
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] -                
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] - [1] Profiles: 'cycling-regular, cycling-mountain, cycling-road', location: 'data/graphs/bike'.
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] - [2] Profiles: 'driving-car,driving-hgv', location: 'data/graphs/vehicles'.
22 Dec 14:04:42 ERROR [routing.RoutingProfileManager] - java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Your specified OSM file does not exist:/usr/local/tomcat/data/heidelberg.osm.gz
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] - [3] Profiles: 'foot-walking,foot-hiking', location: 'data/graphs/pedestrian'.
22 Dec 14:04:42 INFO [routing.RoutingProfileManager] - [4] Profiles: 'cycling-safe,cycling-tour, cycling-electric', location: 'data/graphs/bike2'.
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Your specified OSM file does not exist:/usr/local/tomcat/data/heidelberg.osm.gz
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at heigit.ors.routing.RoutingProfileManager.initialize(RoutingProfileManager.java:237)
        at heigit.ors.routing.RoutingProfileManager.getInstance(RoutingProfileManager.java:83)
        at heigit.ors.servlet.listeners.ORSInitContextListener.lambda$contextInitialized$0(ORSInitContextListener.java:43)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Your specified OSM file does not exist:/usr/local/tomcat/data/heidelberg.osm.gz
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:151)
        at com.graphhopper.GraphHopper.importData(GraphHopper.java:704)
        at com.graphhopper.GraphHopper.process(GraphHopper.java:673)
        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:648)
        at heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:91)
        at heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:194)
        at heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:137)
        at heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:44)
        at heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

He's still looking, for some reason I couldn't determine, for heidelberg.osm.gz, so the instructions must be missing some part to be complete.

[TimMcCauley]

Greetings - can you show me your app.config file and docker-compose.yml ?

[GPistre]

I managed to get it to work using those exact commands (for a specific osm file)

# With heidelberg.osm.gz
git clone https://github.com/GIScience/openrouteservice.git
cd openrouteservice/
cd docker/
docker-compose up -d
# Wait a moment until app ready with http://localhost:8080/ors/health
# Test with http://localhost:8080/ors/routes?profile=foot-walking&coordinates=8.676581,49.418204|8.692803,49.409465


# With Ile De France
cd data/
wget http://download.geofabrik.de/europe/france/ile-de-france-latest.osm.pbf
cd ..
sudo rm -rf logs/ build/ data/graphs/ rm data/cgiar_provider/

nano conf/app.config.sample # change 'heidelberg.osm.gz' to 'ile-de-france-latest.osm.pbf'
docker-compose up --build --force-recreate -d

[TimMcCauley]

Great - can I consider this closed? p.s. and sorry for the late reply..!

[GPistre]

Yes.
No worries, as long as it works :)

[eduardoandino]

Sorry to bother again with this. But I did exactly what @GPistre said and the service stays in {"status":"not ready"}.
It works with the dafult OSM dataset, but when I use "ile-de-france-latest.osm.pbf" it does not work. I modified the docker-compose.yml file to point to this new file. What am I missing?

The log in logs/ors/ors.log is empty, and the onw of tomcat shows no error.
Thanks in advance for your help

[Jaszkowic]

@eduardoandino Did you try to check the docker container output with docker container logs <container_id> ? I had a similar problem where status was stuck on "not ready" with no log output in logs/ors/ors.log but some Java out of memory problem that was logged in the docker container. However, I did use a pretty large OSM dataset (~around 4GB), with smaller datasets everything worked fine.