✨ Create an OTP docker container to use with the simulation code

[shankari]

It would be good if we could have an OTP docker container and launch it to both:

• test the integration, and
• actually generate the trips in one easy swoop.

This issue documents the design decisions related to this feature

[shankari]

The URL for the OTP server is currently stored in emission/net/ext_service/otp/planner.json. This is clearly terrible since it mixes code and config. There are two possible fixes:

• move it to conf/net/ext_service, or
• since it only has one variable, use an environment variable OTP_SERVER instead

Using an environment variable makes it easier to use in a docker-compose file since we can create an internal network and then use the name of the container directly.

[shankari]

We also need to decide which docker image to use.
@alvinalexander created a docker container for the SF bay area, which he published at
https://hub.docker.com/r/alvinghouas/otp-sfbay

This currently has 23 downloads and 1 star.
However, it is also 800MB+ and I can't run it either on my laptop or on my local linux server.
Trying on one of the millennium servers...

[alvinalexander]

@shankari If I remember correctly, the large size is because all the map-data was included in the build. A better approach would probably be to create an image that can fetch map data from an external source (such as an s3 bucket)

[shankari]

@alvinalexander good point! However, the problem is not just the disk space, but the memory required. On my laptop, the docker image launches, but then the java process is killed.

Trying on one of the millennium servers...

This works. I'm going to go with this right now to unblock @njriasan

[njriasan]

I just wanted to update this with some details on the memory required. I was able to launch the OTP server on my laptop and in the process it consumed 8.4 GB of RAM. It was also much faster than 10 minutes for me to start up, probably closer to 2.

[shankari]

I guess I need a better laptop 😄

I am surprised that it took less than 10 minutes because the millennium machines are fairly beefy and it definitely took ~ 10 minutes to read the graph.

09:43:37.061 INFO (InputStreamGraphSource.java:207) Loading graph...
09:43:47.997 INFO (Graph.java:789) Graph version: MavenVersion(1, 3, 0, , 23d319183a7d6fa9b3caef5412fc3ad8fcc54290)
09:43:47.998 INFO (Graph.java:790) OTP version:   MavenVersion(1, 3, 0, , 23d319183a7d6fa9b3caef5412fc3ad8fcc54290)
09:43:47.998 INFO (Graph.java:807) This graph was built with the currently running version and commit of OTP.

..... 10 minutes ....

09:51:41.121 INFO (Graph.java:759) Main graph read. |V|=1043836 |E|=2857766
09:51:50.294 INFO (GraphIndex.java:124) Indexing graph...
09:51:53.943 INFO (GraphIndex.java:583) Clustering stops by geographic proximity and name...
09:51:56.410 INFO (GraphIndex.java:198) Creating a spatial index for stop clusters.
09:51:56.482 INFO (GraphIndex.java:187) Done indexing graph.
09:51:56.722 INFO (Router.java:100) No default routing parameters were found in the router config JSON. Using built-in OTP defaults.
09:52:04.910 INFO (Router.java:127) Timeouts for router 'sfbay': [5.0, 4.0, 2.0]
09:52:04.911 INFO (Router.java:134) Incoming requests will not be logged.
09:52:04.918 INFO (GraphUpdaterConfigurator.java:55) Using configurations: [main]
09:52:05.190 INFO (Router.java:166) Computed ellipsoid/geoid offset at (37.74047035, -122.1951963) as -32.28489758259721
09:52:05.198 INFO (GrizzlyServer.java:72) Starting OTP Grizzly server on ports 8080 (HTTP) and 8081 (HTTPS) of interface 0.0.0.0
09:52:05.198 INFO (GrizzlyServer.java:74) OTP server base path is ./
09:52:05.262 INFO (GrizzlyServer.java:51) Java reports that this machine has 48 available processors.

I wonder if this is an I/O issue - maybe the docker is configured to store the containers on non-local storage.