NMBS endpoint changed: Error 500 when querying connections

[Bertware]

PHP message: PHP Warning:  SimpleXMLElement::__construct(): Entity: line 9: parser error : Opening and ending tag mismatch: hr line 7 and body in /var/www/api.irail.be/api/data/NMBS/connections.php on line 198
PHP message: PHP Warning:  SimpleXMLElement::__construct():

Cause by HTML

Not Found

The requested URL /jp/sncb-nmbs-routeplanner/extxml.exe was not found on this server.
Apache/2.2.15 (Red Hat) Server at www.belgianrail.be Port 80

[Bertware]

Example alternative: using the mobile API enpoint. We'd need to decipher it though.
This is a route Halle -> Gent Sint Pieters:

POST http://www.belgianrail.be/jp/sncb-nmbs-routeplanner/mgate.exe HTTP/1.1
User-Agent: SNCB/302132 (Android_5.0.2) Dalvik/2.1.0 (Linux; U; Android 5.0.2; HTC One Build/LRX22G)
Content-Type: text/plain;charset=UTF-8
Host: www.belgianrail.be
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 807


{"auth":{"aid":"sncb-mobi","type":"AID"},"client":{"id":"SNCB","name":"NMBS","os":"Android 5.0.2","type":"AND","ua":"SNCB/302132 (Android_5.0.2) Dalvik/2.1.0 (Linux; U; Android 5.0.2; HTC One Build/LRX22G)","v":302132},"lang":"nld","svcReqL":[{"cfg":{"polyEnc":"GPA"},"meth":"TripSearch","req":{"arrLocL":[{"lid":"A=1@O=Gent-Sint-Pieters@X=3710675@Y=51035896@U=80@L=008892007@B=1@p=1429490515@","name":"Gent-Sint-Pieters","type":"S"}],"depLocL":[{"lid":"A=1@O=Halle@X=4240634@Y=50733931@U=80@L=008814308@B=1@p=1481329402@n=ac.1=GA@","name":"Halle","type":"S"}],"jnyFltrL":[{"mode":"BIT","type":"PROD","value":"11101111111111"}],"outDate":"20170315","outTime":"192345","economic":false,"extChgTime":-1,"getIST":false,"getPasslist":true,"getPolyline":true,"liveSearch":false}}],"ver":"1.11","formatted":false}
HTTP/1.1 200 OK
Date: Wed, 15 Mar 2017 18:23:54 GMT
Server: Apache/2.2.15 (Red Hat)
Content-Length: 12276
Connection: close
Content-Type: application/json; charset=utf-8

{
  "ver": "1.11",
  "lang": "nld",
  "id": "29k64uyqwwmjxic4",
  "svcResL": [
    {
      "id": "",
      "meth": "TripSearch",
      "err": "OK",
      "res": {
        "common": {
          "locL": [
            {
              "lid": "A=1@O=Halle@X=4240634@Y=50733931@U=80@L=8814308@",
              "type": "S",
              "name": "Halle",
              "icoX": 1,
              "extId": "8814308",
              "crd": {
                "x": 4240634,
                "y": 50733931,
                "z": 0
              },
              "pCls": 68,
              "rRefL": [
                0,
                1,
                2
              ]
            },
            {
              "lid": "A=1@O=Gent-Sint-Pieters@X=3710675@Y=51035897@U=80@L=8892007@",
              "type": "S",
              "name": "Gent-Sint-Pieters",
              "icoX": 1,
              "extId": "8892007",
              "crd": {
                "x": 3710675,
                "y": 51035897,
                "z": 0
              },
              "pCls": 100,
              "rRefL": [
                0,
                1,
                2
              ]
            },
            {
              "lid": "A=1@O=Brussel-Zuid@X=4336531@Y=50835707@U=80@L=8814001@",
              "type": "S",
              "name": "Brussel-Zuid",
              "icoX": 1,
              "extId": "8814001",
              "crd": {
                "x": 4336531,
                "y": 50835707,
                "z": 0
              },
              "pCls": 69,
              "rRefL": [
                0,
                1,
                2
              ]
            }
          ],
          "prodL": [
...
          ],
          "opL": [
            
          ],
          "remL": [
            {
              "type": "A",
              "code": "GA",
              "prio": 999,
              "icoX": 0,
              "txtN": "Belgium stations"
            },
            {
              "type": "A",
              "code": "GH",
              "icoX": 0,
              "txtN": ""
            },
            {
              "type": "A",
              "code": "LI",
              "icoX": 0,
              "txtN": ""
            }
          ],
          "icoL": [
            {
              "res": "INFO"
            },
            {
              "res": "STA_SNCB"
            },
            {
              "res": "IC"
            }
          ]
        },
        "outConL": [
          {
            "cid": "C-0",
            "date": "20170315",
            "dur": "005300",
            "chg": 1,
            "sDays": {
              "sDaysR": "Ma - Vr",
              "sDaysI": "niet 17. Apr, 1., 25. Mei, 5. Juni, 21. Juli, 15. Aug, 1. Nov",
              "sDaysB": "7CF9F3E7CF9F387CF9A3E7CF9F3E7CF8F3E3CF9F3A7C79F3E7CF9F3C7CF9F2E7CF9F3E7CF9F3E7CF9B3E7CF9F3E0"
            },
            "dep": {
              "locX": 0,
              "dPlatfS": "4",
              "dTimeS": "192700",
              "dTimeR": "192700",
              "dProgType": "PROGNOSED"
            },
            "arr": {
              "locX": 1,
              "aPlatfS": "12",
              "aTimeS": "202000",
              "aTimeR": "202100",
              "aProgType": "PROGNOSED"
            },
            "secL": [
              {
                "type": "JNY",
                "icoX": 2,
                "dep": {
                  "locX": 0,
                  "dPlatfS": "4",
                  "dTimeS": "192700",
                  "dTimeR": "192700",
                  "dProgType": "PROGNOSED"
                },
                "arr": {
                  "locX": 2,
                  "aPlatfS": "15",
                  "aTimeS": "193600",
                  "aTimeR": "193700",
                  "aProgType": "PROGNOSED"
                },
                "jny": {
                  "jid": "1|869|0|80|15032017",
                  "prodX": 0,
                  "dirTxt": "Luik-Guillemins",
                  "isRchbl": true,
                  "stopL": [
                    {
                      "locX": 0,
                      "idx": 18,
                      "aTimeS": "192600",
                      "aProgType": "PROGNOSED",
                      "dProdX": 0,
                      "dTimeS": "192700",
                      "dTimeR": "192700",
                      "dProgType": "PROGNOSED"
                    },
                    {
                      "locX": 2,
                      "idx": 23,
                      "aProdX": 0,
                      "aTimeS": "193600",
                      "aTimeR": "193700",
                      "aProgType": "PROGNOSED",
                      "dTimeS": "193900",
                      "dProgType": "PROGNOSED"
                    }
                  ],
               
           ... 

[Bertware]

A temporary implementation is in place. Following functions are not available at this moment:

• Changed platform indication
• Service alerts
• Arrival/Departure selection

Anyone who wants to help out debugging the new implementation is welcome to do so!

[derhuerst]

FYI I wrote two mobile HAFAS API clients in JavaScript, so maybe the parsing logic is helpful to you. The client doesn't parse everything the API provides, but most of it, and has been running fine for the past 2 years.

Specifically, look at the following files:

• hafas-client/parse.js – contains functions to parse everything VBB and DB have in common.
• vbb-hafas/parse.js – contains functions to parse VBB-specific stuff.
• db-hafas/parse.js – contains functions to parse DB-specific stuff.

Let me know if I can assist you with HAFAS-mobile-API-related problems, as long as they're not specific to Belgian Rail.

[Bertware]

@derhuerst Thanks! I've looked around for a bit, and it seems really helpful (especially the implementations for vbb/db give me some more insights on how to use it)

Do you happen to know how we can detect a platform change? Platforms are in aPlatfS, but is there also something like aPlatfR for 'real' data in case it changed? PlatfR stands for realtime data!

Have you seen the search parameter liveSearch before? Or do you happen to know how to search journeys by time of arrival (instead of departure time)

[Bertware]

According to traffic by the official NMBS app:

• The vehicle field should be 01101111000111 for only trains
• When searching by "arrive at", the following field should be added: "outFrwd": false
• Example output when there are 'alerts' : https://pastebin.com/bfq2trTz

This should be enough to resolve all our issues with the new parser

• Show all trains, but no buses etc
• Allow searching by arrival time
• Show alerts when requested using the &alerts=true parameter

[derhuerst]

Have you seen the search parameter liveSearch before? Or do you happen to know how to search journeys by time of arrival (instead of departure time)

I haven't, sorry. What you can do though is use e.g. mitmproxy to redirect traffic of the common apps through your machine and record all requests. By doing this, you can reverse-engineer the parameters.

[Bertware]

@derhuerst Just figured it out how to search on arrival time using fiddler :) Still wondering what that liveSearch parameter is, it's always included, always false, and doesn't seem to affect anything. I've set it to true atm for our application in the hopes that it will return fresher data or anything.