PostgreSql events proxy

Send NOTIFY and WAL events from PostgreSQL to upstream services

But Why?

This tool enables a decoupled architecture, think sending emails when a user signs up. Instead of having explicit code in your signup function that does the work (and slows down your response), you just have to worry about inserting the row into the database. This event (either the WAL or a NOTIFY generated by a trigger) will be proxied across to an upstream service. From there, you can have multiple consumers reacting to that event (send signup email, send sms notification). Those consumers tend to be very short, self contained scripts.

The larger goal is to enable the development of backends around PostgREST/subZero philosophy. Check out the PostgREST Starter Kit/subZero Starter Kit to see how pg-event-proxy fits in a larger project.

Currently the supported upstreams

• amqp 0.9 (RabbitMQ)
• mqtt (Apache ActiveMQ, Cassandana, HiveMQ, Mosquitto, RabbitMQ, AWS IoT, Amazon MQ, ...)
• redis pubsub (Redis)
• SNS (Amazon Simple Notification Service)
• SQS (Amazon Simple Queue Service)
• Lambda (AWS Lambda)

This gives you a wide range of "destinations" for the database events

Configuration

Configuration is done through environment variables (useful in docker context) or a ini configuration file See docker-compose.yml and config_example.ini for details

Note: It's recommended to always use the same name for postgresql channel and exchange/queue/topic in BRIDGE_CHANNELS, for example app_events->topic:app_events, table_changes->topic:tables_changes to make it easy to determine where the event originated

Running as docker container

See the docker-compose.yml file in this repository for a sample

Running from command line

pg-event-proxy /path/to/config.ini (the executable is not available in this repository, see subzero.cloud )

Sending messages

A message can be sent either by running a query like NOTIFY pgchannel1, 'My message' or by simply running an INSERT/UPDATE/DELETE query if you configured WAL streaming by using wal2json as the name of the channel.

When using using amqp or mqtt as upstreams it's possible to specify the routing key/topic for the message. For example, having the BRIDGE_CHANNELS configured as pgchannel->exchange:events (rabbitmq) or pgchannel->topic:events (mqtt), by running the following query NOTIFY pgchannel, 'signup|A new user signup', a message will be published to the events exchange with the routing key signup in rabbitmq and respectively, a message published to the mqtt messaging server with the topic being events.signup. This allows for a wide range of routing topologies implemented in your messaging server.

Helper Functions

To make it bit more user friendly it's recommended to use events schema functions to send events. This module provides some durability features in case of intermittent disconnects. With this lib you can send events with a query like this

select events.send_message('pgchannel1', 'My message', 'routing-key', true)

See it in action

Bring up the stack

docker-compose up -d

In console 1

./generate_events.sh

In console 2

./listen_events.sh

Support

• Slack — Watch announcements, share ideas and feedback
• GitHub Issues — Check open issues, send feature requests

License

pg-event-proxy is a commercial product. Licenses can be purchase from subzero.cloud

subzerocloud/pg-event-proxy-development docker image is provided for testing/evaluation purposes