waltti-apc-journey-matcher

Match APC messages from the vehicles with GTFS Realtime messages to augment the APC messages with vehicle journey metadata.

To do the matching, one needs a few mappings between IDs. During the pilot phase, the mappings can be encoded in an environment variables. After the pilot phase, the mappings could be managed in a separate system from this microservice, e.g. in the fleet registry.

This repository has been created as part of the Waltti APC project.

Development

1. Create a suitable .env file for configuration. Check below for the configuration reference.

2. Create any necessary secrets that the .env file points to.

3. Install dependencies:

   npm install

4. Run linters and tests and build:

   npm run check-and-build

5. Load the environment variables:

   set -a
   source .env
   set +a

6. Run the application:

   npm start

Docker

You can use the Docker image tvvlmj/waltti-apc-journey-matcher:edge. Check out the available tags.

Configuration

Environment variable	Required?	Default value	Description
APC_WAIT_IN_SECONDS	❌ No	6	A float describing how long to wait accumulating more APC data after a GTFS Realtime event has triggered the sending process for this vehicle and stop but before sending the summed values onwards for this vehicle and stop.
COUNTING_SYSTEM_MAP	✅ Yes		A map from the counting system IDs to vehicles and vendor names. The map is given in the form of a stringified JSON array of strings in the shape [[systemId1, [uniqueVehicleId1, vendorA]], [systemId2, [uniqueVehicleId2, vendorB]], ...] like the output of Map.prototype.entries(). An example could be [[\"c5e96843-e820-4837-8eef-6176be4b4c4e\",[\"fi:jyvaskyla:6714_503\",\"Acme\"]],[\"6dd41f2e-841f-44a0-b5f8-a108847dc4a2\",[\"fi:jyvaskyla:6714_529\",\"Corpcorp\"]]].
FEED_MAP	✅ Yes		A map from Pulsar topics to feed publisher IDs, Waltti OpenData Authority IDs and timezone names. The map is given in the form of a stringified JSON array of strings in the shape [[pulsarTopic1, [feedPublisher1, walttiOpenDataAuthority1, timezone1]], [pulsarTopic2, [feedPublisher2, walttiOpenDataAuthority2, timezone2]], ...] like the output of Map.prototype.entries(). The feed publisher ID is a unique ID for the authority or the GTFS feed publisher whose APC data will be handled. The format is <country-code>:<name> where <country-code> follows a lowercase version of ISO 3166-1 alpha-2 and <name> is unique within the country. An example could be fi:jyvaskyla. The corresponding Waltti OpenData Authority IDs are listed here: https://opendata.waltti.fi/docs#gtfs-static-packages . The timezone name refers to the timezone used by the authority or the GTFS feed publisher for local time. It is given in the format of an IANA timezone (tz database), e.g. Europe/Helsinki. An example of the whole list could be [[\"persistent://apc-dev/source/gtfs-realtime-vp-fi-kuopio\",[\"fi:kuopio\",\"221\",\"Europe/Helsinki\"]],[\"persistent://apc-dev/source/gtfs-realtime-vp-fi-jyvaskyla\",[\"fi:jyvaskyla\",\"209\",\"Europe/Helsinki\"]]].
HEALTH_CHECK_PORT	❌ No	8080	Which port to use to respond to health checks.
PINO_LOG_LEVEL	❌ No	info	The level of logging to use. One of "fatal", "error", "warn", "info", "debug", "trace" or "silent".
PULSAR_APC_CONSUMER_TOPICS_PATTERN	✅ Yes		The topic pattern to consume APC vehicle messages from.
PULSAR_APC_SUBSCRIPTION	✅ Yes		The name of the subscription for reading messages from PULSAR_APC_CONSUMER_TOPICS_PATTERN.
PULSAR_BLOCK_IF_QUEUE_FULL	❌ No	true	Whether the send operations of the producer should block when the outgoing message queue is full. If false, send operations will immediately fail when the queue is full.
PULSAR_COMPRESSION_TYPE	❌ No	ZSTD	The compression type to use in the topic where messages are sent. Must be one of Zlib, LZ4, ZSTD or SNAPPY.
PULSAR_GTFSRT_CONSUMER_TOPICS_PATTERN	✅ Yes		The topic pattern to consume GTFS Realtime messages from.
PULSAR_GTFSRT_SUBSCRIPTION	✅ Yes		The name of the subscription for reading messages from PULSAR_GTFSRT_CONSUMER_TOPICS_PATTERN.
PULSAR_OAUTH2_AUDIENCE	✅ Yes		The OAuth 2.0 audience.
PULSAR_OAUTH2_ISSUER_URL	✅ Yes		The OAuth 2.0 issuer URL.
PULSAR_OAUTH2_KEY_PATH	✅ Yes		The path to the OAuth 2.0 private key JSON file.
PULSAR_PRODUCER_TOPIC	✅ Yes		The topic to send messages to.
PULSAR_SERVICE_URL	✅ Yes		The service URL.
PULSAR_TLS_VALIDATE_HOSTNAME	❌ No	true	Whether to validate the hostname on its TLS certificate. This option exists because some Apache Pulsar hosting providers cannot handle Apache Pulsar clients setting this to true.