Rapid, subsequent requests will return unexpected 500's

[clinct]

I did some stress testing of OTP master branch and found out that if I rapidly fire different requests (1 per 0.5 seconds, around 20), some of them that normally return a 200 will suddenly start returning 500’s.

The performed requests were all send with similar parameters with only varying fromPlaces:

?maxTransfers=25&_dc=1358423838102&arriveBy=false&mode=TRANSIT,WALK&optimize=QUICK&maxWalkDistance=3000&walkSpeed=1.3888&hst=true&date=2014-11-11&time=16:30&toPlace=52.368104267594056,4.856208655327167&fromPlace=52.3681107,4.8562698

[hannesj]

I've also seen this. It seems that the current usage of having runState as a field in GenericAStar is not thread-safe. I solved this by having a new GenericAStar per request and haven't had issues since.

See https://github.com/hannesj/OpenTripPlanner/blob/master/src/main/java/org/opentripplanner/routing/impl/SimpleAStarPathServiceImpl.java

[abyrd]

Thanks @clinct for the report, and thank you @hannesj for pointing out the source of the problem. These state variables were originally function-scoped, but were pulled out to fields recently to make it possible to pause the routing process and observe its behavior. It's not immediately obvious to someone jumping into OTP that GenericAStar is not instantiated on a per-request basis, so yes we ended up with a non-thread-safe situation. OTP behavior is currently undefined under concurrent requests, so we'll make it a priority to fix this.

[abyrd]

By the way @hannesj, if you spot and diagnose things like this, definitely feel free to ticket/report them in the future.

[hannesj]

@abyrd I've always tried to report them when they come up, but this one slipped my mind, as I was working on a separate path service implementation at that time. (even tough it is actually a workaround for another bug in OTP)

[bmander]

PathService implementations no longer take SPTServices in the constructor; rather take an SPTServiceFactory and instantiate their own SPTService instances as needed.

84c9e0a

@abyrd Could you sign off on this before closing it?

[clinct]

Repeated the same tests and the issue is no longer there; no unexpected 500's were returned. Thanks!

[abyrd]

Some line notes added directly in 84c9e0a, but as that is not a pull request they did not carry over to this ticket. Most of that commentary is now ticket #1618.