Publish data to channel by socketCluster object

[fanweixiao]

after var sc = new SocketCluster(opts);, I want to use sc object to publish data to channel, but can not find a way.

while I can use worker or socket to complete this job, but I want to use redis pub/sub to trigger sending data action, I don't want to do subscribing to redis on worker because I have to sub only once. So the sc creation is the best place to do redis pub/sub.

[jondubois]

The reason why you can't publish events directly from the SocketCluster master is because it is an anti-pattern - We should avoid putting any runtime logic in master - The master process isn't as resilient as worker processes so it's better to keep it lightweight (mostly good for configuration/bootstrap logic).

The best place to publish to an SC channel from outside is in the worker (worker.js) - The worker object which is passed inside the run() method has an id property which is the index of the current worker - There will always be at least one worker with id 0. So inside worker.js, you can have this code:

// By checking worker.id we can make sure that only one worker is handling a particular event
// So we don't get duplicates
if (worker.id == 0) {
  // ... Here we can do anything we want like listen for events from outside Node.js
  // For example, we can use the Node.js redis client https://github.com/mranney/node_redis 
  // to subscribe to specific channels from redis and publish data to 
  // SocketCluster channels by calling worker.global.publish('myChannel', dataFromRedis);
}

If you have a LOT of external data coming in from many different Redis channels (too much for a single worker process to handle), you could split responsibilities between multiple workers and make it so that each worker handles a specific subset of all Redis channels - This approach is more scalable.

[jondubois]

EDIT I edited the post above since the information provided earlier about publishing from stores is not ideal - You should actually publish from workers - That way channel data will automatically get routed to the appropriate store which is responsible for that channel.

Note that in SC, if you have multiple store processes, each store will only be responsible for some of the channels - That way they share the work evenly.

[jondubois]

I just started working on a Redis adapter which will allow hooking SC into Redis automatically to share channels.

[jondubois]

I'm closing this issue. The optimal solution will be the Redis adapter - I opened a separate issue for it here: #38

In the meantime, the best way to interact with SC from external systems is by passing data to workers. You can use IDs to uniquely identify them.

[slidenerd]

@jondubois thank you for the detailed explanation, I have a similar case and a small doubt about the framework which I couldnt find in the documentation.

My workers are asymmetric and the way I am currently separating them is by doing what you did above

if (worker.id === 0){
//get data from 1st external websocket API
}
else if(worker.id === 1){
//get data from 2nd external websocket API
}

so if worker 0 and worker 1 are getting data from different APis. how do I let all the clients subscribe to this data

if I do worker.exchange.publish() from worker 0 and if a client is connected to worker 1 will they still get this data? My case happens to be where I want individual workers fetching data from different API sources but connecting clients to be able to get access to whichever data source they want regardless of which worker they are connected with

[jondubois]

@slidenerd

if I do worker.exchange.publish() from worker 0 and if a client is connected to worker 1 will they still get this data?

Yes, channels in SC are global so they are shared between all processes. If you use SCC, channels are also shared between all machines so it doesn't matter what process or machine the client is connected to.