Found issue with delays

[xivk]

Using the routeplanner when travelling I discovered our API doesn't show the correct delays while iRail and NMBS routeplanners do. Not sure where the bug is to be found.

The departure and arrival times match perfectly. I already check the code and this commit may have fixed things: ab0d166

[pietervdvn]

I have looked into this, but not yet found why it happens. An unit test and functional test did not indicate something amiss too, although I am able to reproduce it on the live server.

It seems quite doable to reproduce on the live version (2019-03-06):

• Take the live board of a busy station on irail, e.g. Brussels-Central
• Some train is always delayed. Take it and plan a route using it.

[pietervdvn]

Still occurs with In storage we trust (Itinero-transit 0.8.2-pre61)

[pietervdvn]

I've noticed that, when this bugs occurs,gtfs:dropOffType "gtfs:NotAvailable" often appears in the upstream data.

Also, if a connection is cancelled, the departure time (with delay included) is still given in the JSON - but the delay is not. This could be a partial explanation, although I'm not sure about that.

[pietervdvn]

Found in the upstream Linked Connections. Notice the BXL-Congres --> Noord AND Centraal-->Noord transfer of what is seemingly the same trip.

9	
@id	"http://irail.be/connections/8813045/20190306/IC3216"
@type	"Connection"
departureStop	"http://irail.be/stations/NMBS/008813045" // Congres
arrivalStop	"http://irail.be/stations/NMBS/008812005" // Noord
departureTime	"2019-03-06T17:25:00.000Z"
arrivalTime	"2019-03-06T17:27:00.000Z"
gtfs:trip	"http://irail.be/vehicle/IC3216/20190306"
gtfs:route	"http://irail.be/vehicle/IC3216"
direction	"Saint-Nicolas"
gtfs:pickupType	"gtfs:NotAvailable"
gtfs:dropOffType	"gtfs:Regular"


208	
@id	"http://irail.be/connections/8813003/20190306/IC3216"
@type	"Connection"
departureStop	"http://irail.be/stations/NMBS/008813003" // centr
arrivalStop	"http://irail.be/stations/NMBS/008812005" // Noord
departureTime	"2019-03-06T17:27:00.000Z"
arrivalTime	"2019-03-06T17:30:00.000Z"
departureDelay	240
arrivalDelay	180
direction	"Saint-Nicolas"
gtfs:trip	"http://irail.be/vehicle/IC3216/20190306"
gtfs:route	"http://irail.be/vehicle/IC3216"


562	
@id	"http://irail.be/connections/8812005/20190306/IC3216"
@type	"Connection"
departureStop	"http://irail.be/stations/NMBS/008812005"// Noord
arrivalStop	"http://irail.be/stations/NMBS/008812021"  // Bockstael 
departureTime	"2019-03-06T17:32:00.000Z"
arrivalTime	"2019-03-06T17:37:00.000Z"
gtfs:trip	"http://irail.be/vehicle/IC3216/20190306"
gtfs:route	"http://irail.be/vehicle/IC3216"
direction	"Saint-Nicolas"
gtfs:pickupType	"gtfs:Regular"
gtfs:dropOffType	"gtfs:NotAvailable"
departureDelay	180
arrivalDelay	120

[pietervdvn]

Deployed the version which drops cancelled trains upon loading - bug is still there.

[pietervdvn]

I dissected a live transitdb, the delay data is missing in there too

[pietervdvn]

I wrote a small test where I fetched a 'connections.json', loaded it and counted the number of delayed entries. No delayed entries were missing in the output.

I suspect that this is a caching issue. It takes a few minutes for the delays to get from SNCB to irail, it takes another few minutes to get them loaded and a few extra minutes are added by the time that we get them in our server - causing the delays to be missing. As the most important delays are 'right now', this is quite inconvenient, so some more discussion is needed on this.

[pietervdvn]

The refreshing time has been lowered to 5 seconds, caching should avoid the biggest load. This should fix most of it.