National Rail Open Data Python Example

This repository contains an Python 3 example of how to use the Darwin v16 messages from the National Rail Open Data platform, located at the following URL:

• https://opendata.nationalrail.co.uk/

To use this service, you will need to sign up for a free account and subscribe to the 'Darwin' feed.

Configuration

Create a .env file and set the USERNAME, PASSWORD, HOSTNAME and HOSTPORT variables to the values shown in 'Username', 'Password', 'Messaging host' and 'STOMP Port' on the 'My Feeds' page on https://opendata.nationalrail.co.uk/feeds. An example .env file is .env.example.

Leave CLIENT_ID set to socket.getfqdn() - this will use the hostname of your client for the durable subscription. You may need to change this to something else if you want to fail over the subscription to a different client. In this case, you could use your username.

The HEARTBEAT_INTERVAL_MS is set by default to 15 seconds (15,000ms) - this should be sufficient for almost every application. Don't change it unless you have a good reason to do so.

RECONNECT_DELAY_SECS will enforce a 15-second delay before exiting. This will let you run the client in a loop, such as through a shell script, and protect against you accidentally degrading the service for everyone by reconnecting far too frequently.

Finally, install the required dependencies by running pip install -r requirements.txt, and run

python opendata-nationalrail-client.py

Generating classes

The messages are produced in XML format, and a good way to consume them is by using generated classes with PyXB.

pyxbgen --schema-root=ppv16 --module PPv16 rttiPPTSchema_v16.xsd

Running the code

The opendata-nationalrail-client.py script will show the raw message body and print the timestamp from the parsed XML through the classes generated by PyXB.

The stomp library is quite verbose in its logging, but you should see output similar to the following:

2022-08-22 10:54:00,778 INFO	attempt reconnection (True, None, 0)
2022-08-22 10:54:00,778 INFO	Attempting connection to host darwin-dist-44ae45.nationalrail.co.uk, port 61613
2022-08-22 10:54:00,803 INFO	Established connection to host darwin-dist-44ae45.nationalrail.co.uk, port 61613
2022-08-22 10:54:00,804 INFO	Starting receiver loop (<Thread(Thread-1, started daemon 140347258906368)>)
2022-08-22 10:54:00,804 INFO	Created thread <Thread(Thread-1, started daemon 140347258906368)> using func <function default_create_thread at 0x7fa52555c4c0>
2022-08-22 10:54:00,804 INFO	Connecting to darwin-dist-44ae45.nationalrail.co.uk
2022-08-22 10:54:00,804 INFO	Sending frame: 'STOMP'
2022-08-22 10:54:00,822 INFO	Received frame: 'CONNECTED', len(body)=0
2022-08-22 10:54:00,823 INFO	Sending frame: 'SUBSCRIBE'
2022-08-22 10:54:00,823 INFO	Starting heartbeat loop
2022-08-22 10:54:00,846 INFO	Received frame: 'MESSAGE', len(body)=467
2022-08-22 10:54:00,846 INFO	{cmd=MESSAGE,headers=[{'content-length': '467', 'expires': '1661162040963', 'destination': '/topic/darwin.pushport-v16', 'CamelJmsDeliveryMode': '1', 'subscription': '1', 'priority': '4', 'breadcrumbId': 'ID-nrdp-prod-01-dsg-caci-co-uk-1661027611761-0-14496036', 'Content_HYPHEN_Type': 'application/xml', 'Username': 'thales', 'SequenceNumber': '702508', 'message-id': 'ID:nrdp-prod-01.dsg.caci.co.uk-43581-1661027612860-6:238:1:1:704479', 'PushPortSequence': '5502748', 'MessageType': 'TS', 'timestamp': '1661161640963'}],body=b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00\x9d\x94Qo\x9b0\x10\xc7\xbf\x8a\xe5\xd7)\x01;!\x01\x14\xa8\xb2F\x95"m*\x02\xa6\xf6\xad\xb2\xc0J\x90\x003\xfb\x9cl\xdf~GHK\xd9\x1bE\xc2\xba;\xdf\xff\xc7\x9d\xe5c\xf7\xf0\xa7\xa9\xc9EjS\xa96\xa2l\xe9R"\xdbB\x95U{\x8a\xe8\xaf\xfci\xe1Sb@\xb4\xa5\xa8U+#\xfaW\x1a\xfa\x10\xef\x92Ni \xa8mMD\xcf\x00]\xe88\xd7\xebu\tgQKs\xd2\xcav\xcbB5\x8e\x06\xa8\x9c\xc4\x9as\x82\xf9\xce\x85m\xe8 \n[\xc3g\x08\xb3\xe2,K\x8b\t\xcee5\x12V_#\xf0\x91\xb0\x9eAxR\xba\x11\x80\xc74Ex\xf3\x10\xb2\x10\x06\xa6ml\xbeX\x04\x1b\x11\xdb9\'\x017\xc0Oi\x8c8\xc9)\xc7\x9f\xc1\xc9\xb5\xa8\xda}-5L\x19\xc1\\\xc6\xb3.\xa5\x9e \x98;\x87q8\x08\x10S=\x9b\xa1\xdf\xd7B7\xd3\x16\xd8x7\xff\xd7\xa6y~\xfc\xd0\xf6Gi\x8d\xa3\x95\x827\x94\x03\xce\x02w9_\xb8\xfe\x82\xf3\x9c\xb9\xe1z\x1brw\xe9\xf9\x81\xbf\n\xb6\xdf\\\x16\xba8_\xe3\xb4mp\xdc\xe2\x9dM\x89\xedJ\x01\xf2YW\xa7\n\xe3\x8f\xc7\x8c\x12-\x7f[i SV\x178w\xafi\xfa\xf8\x11<\x1e\xf0C\xdc\xf5\xf1\xbd=\x01[\x05\x08\xca3\xa2\xabr\xa8\xa1\xdf\xf4]\x9fq/\xa0\xc4\xf6\xd1\xe4\xee\x18S~.\x13ux\x89\xc3\x1f\xaa\xb8]\x0b\x02]\x1d\xd1\xfd\xf7\x97\xc3kJ\xc9\x15\x04\x96\xe9\x85lKI7\xda\x83DhM$`\xcc\x0f=\xdc\x97\xd0\xf4\xc5\xdf=\xa3\x8b{#h\x1d[\x03\xf7\x16\x9cA\xdb\xd5\x02H\xbf\x18\xdbE\x14\xb4\x95\x94\x14\x95I&\x91x\xbds\xde\x93\xe3\xc1|/\x13\xdd<\xc3\xc5\xa6\xb8\xdc~F\xf1?g\x8dwp\xcb\x04\x00\x00'}
2022-08-22 10:54:00,846 INFO	Message sequence=702508, type=TS received
2022-08-22 10:54:00,894 INFO	Successfully received a Darwin Push Port message from 2022-08-22 09:47:20.589840+00:00

To adapt this code for your own purposes, extend the on_message() method and process the contents of the obj created.

Docker

To run with Docker:

docker build -t nationalrail-stomp-client .
docker run -it --rm --name nationalrail-stomp-listener nationalrail-stomp-client

Support

For support and questions with using Darwin, please use the forum at the following URL:

• https://groups.google.com/group/openraildata-talk