refactor(session): socket.io scalability

[andreialecu]

This PR allows deployd to properly scale across multiple nodes when using socket.io. the emitToUsers function is no longer tied to internal state, but instead sockets join specific rooms according to the user id.

This can be used together with the socket.io-redis adapter to properly keep socket state in sync across multiple processes, or separate machines.

A memory leak is also fixed by the refactoring made in this PR.

Also, it adds new functionality in the form of additional configuration options:

Example:

var redisAdapter = require('socket.io-redis');

var server = deployd({
  port: process.env.PORT || 1337,
  ...
  socketIo: {
    options: {
      transports: ['websocket'], // only websocket transport allowed (no xhr-polling)
    },
    adapter: redisAdapter({ host: 'localhost', port: 6379 }),
  },

});

I tested it in my own production project for a bit and everything seems fine.

[andreialecu]

I have added another commit which adds some features to this PR:

Expose join/leave room functionality for sessions. Rooms are persisted in the session, so reconnecting the session will rejoin the same rooms.

This exposes the session directly and the following additional methods:

session.joinRoom('roomname');
session.joinRooms(['roomname', 'roomname2']);
session.leaveRoom('roomname');
session.leaveRooms(['roomname', 'roomname2']);
session.leaveAllRooms();

Usage example:

UserCollection/On Login event:

if (me.admin) session.joinRoom('administrators');

Other script:

emit('administrators', 'eventname', { ... });

This should make it much more efficient to emit to certain groups of users by not needing to query the full user collection each time an emit is needed.

[andreialecu]

A few more updates here in the last commit:

Use pubsub via node_redis compatible client to sync up channels that sessions join across multiple nodes (cluster). Also syncs up session removal.

Requires redis, but is optional - you will only need this if you want to run deployd in a cluster and are using socket.io functionality.

Needs configuration like the following:

var redis = require('redis');
var sioPub = redis.createClient(process.env.REDIS_URL);
var sioSub = redis.createClient(process.env.REDIS_URL, { 'return_buffers': true });

var dpdPub = redis.createClient(process.env.REDIS_URL);
var dpdSub = redis.createClient(process.env.REDIS_URL);

var server = deployd({
  port: process.env.PORT || 1337,
  ...
  db: {
    connectionString: process.env.MONGO_URI,
  },
  sessions: {
    pubClient: dpdPub,
    subClient: dpdSub,
  },
  socketIo: {
    adapter: redisAdapter({ pubClient: sioPub, subClient: sioSub }),
  },

});

Also includes a bunch of tests.

[andreialecu]

This should be ready for master now, I've been using it successfully for at least a week.

[NicolasRitouet]

Landed as ff5f840

Thanks a lot for this awesome PR ! 🎁

[NicolasRitouet]

I'll let people play with this on master for a few days and I'll publish a new release on npm next week.