Simple Pub/Sub messaging

[alexweininger]

I know that the team is hard at work on the Realtime feature

I believe that Pub/Sub messaging would be a somewhat simple feature to add into Appwrite, yet it would provide a ton of value to users.

Even a basic implementation with only support for subscribing to channels and publishing messages would be powerful. Additional features such as getting a list of channels would be a nice to have.

Example client SDK usage:

// subscribe to channel
sdk.pubsub.subscribe('messages', (msg) => {
    console.log('Got message:', msg.text);
});

// publish message to channel
sdk.pubsub.publish('messages', { text: 'Hello, world' });

Since Appwrite already uses Redis, we could leverage it's Pub/Sub capabilities. Meaning Appwrite only has to provide a kind of 'wrapper' around the Redis Pub/Sub that enables client interaction.

[eldadfux]

@alexweininger I would love to know what main use cases you had in mind?
Is this mainly around having custom channels and the possibility to also send messages and not only to listen for them?

[Ghostw4lk]

For me, I have use cases in mind, which can be executed asynchronous.

For example: A user wants to get a copy of all his personal data. This can be a long running task, but it can be executed async. If the task is done, the program can send an e-Mail with all collected personal data to the user. For use cases like this, it can be helpful, that e.g. Apache Kafka with message persistence is integrated in appwrite.

[eldadfux]

This could be easily solved by Appwrite async function. Triggering the function execution with the REST API will send a pub/sub message to a worker who will execute the function in the background. From version 0.13 Appwrite will also support aysnc or sync function execution to your choice.

[aminry]

I think there are many other use cases for pubsub. You don't want to always use function to handle an event. Pubsub can be a mechanism used to extend AppWrite to interoperate with other services externally. For example I could use the pubsub to connect AppWrite with an existing function in AWS Lambda or Google Cloud Function, or even an existing legacy service that we have in cloud.

These services could send/receive pubsub message and easily interoperate with AppWrite. One argument could be we can make an API call to interoperate with other service. But that is not as good as pubsub. Since pubsub is fully asynchronous and enables load balancing. The publisher doesn't need to know anything about the subscriber and where it is and no formal connection needs to be established. Particularly the most important use cases is receiving pubsub from appwrite in these external services without the need to expose an API for AppWrite function to call.

I know internally AppWrite is using pubsub to handle scheduling. So maybe the same mechanism can be used to expose an API for external usage.

I suggest an API like this:
`
// subscribe to channel
sdk.pubsub.subscribe('messages', 'client-group', (msg) => {
console.log('Got message:', msg.text);
});

// publish message to channel
sdk.pubsub.publish('messages', { text: 'Hello, world' });
`

I added a 'client-group' to the @alexweininger suggested API so that we could load balance the messages on a channel across all subscribers with the same client-group name.