Improve SignalR Backplane for small number of servers

[tomap]

Summary

When using SignalR Backplane, in particular Redis Backplane, there is a cost in terms of bandwidth used and latency added by the backplane. The wasted bandwidth & latency is 1/N (N = Number of servers)
When the number of server is small, the cost is higher

Motivation and goals

This is my understanding of Redis SignalR Backplane. I'm 90% confident that I understood how it works, but I'm not 100% sure. Sorry if I got it totally wrong :)

1. Whenever a message is sent to a group or all on SignalR with a backplane, the message is sent to Redis via PUBLISH.
2. Redis sends this message to both the sender and the other server indiscriminately
3. If you have 2 servers, the message (Ex: 1kB) is sent to Redis once (1kB) and Redis sends it to 2 servers (2kB). So 50% bandwidth wasted. This ratio decreases with the number of servers (1/N)
4. In the same scenario, if a user is connected to the sender server, then he has to wait for the PUB/SUB roundtrip before receiving the message.

The proposal is the following:

1. Exclude the sender from receiving the message
2. Send the message directly to users connected to the current server without paying the cost of a Redis PUB/SUB round trip

Benefits / Limits

Benefits are obvious in terms of wasted bandwidth & latency

However, it does not improve the "worst possible latency".
It only improves the average latency.

There is an added benefit that messages that needs to be broadcasted might benefit from the freed bandwidth

Design proposal

1. to exclude the sender from receiving the message, in a scenario where the server don't know the names of the other servers, we could use PSUBSCRIBE/PPUBLISH
   Let's say you have a group named "TOY"
   Servers could subscribe to this channel using ^TOY(?!MACHINENAME1).*$ where MACHINENAME1 is the name of the current machine (there is probably some escaping to do to)
   Fortunately, the current implementation of Redis backplane automatically switches to Pattern based PUB/SUB
   See Add precision about wildcard (*) + Redis SignalR Backplane = Pattern based subscriptions AspNetCore.Docs#29435
   So this part is already "possible"

2. Then we must publish each message both to the Redis AND to the users connected to the current server
   This is done by adapting the publish code. Probably around here:
   https://github.com/dotnet/aspnetcore/blob/main/src/SignalR/server/StackExchangeRedis/src/RedisHubLifetimeManager.cs#L298

[BrennanConroy]

That's an interesting idea!

I'm not sure if it works with sharding though, which means it might need to be an opt-in feature so we don't regress sharding scenarios.

[tomap]

That's an interesting idea!

I'm not sure if it works with sharding though, which means it might need to be an opt-in feature so we don't regress sharding scenarios.

Indeed, I don't think it works with sharidng, but anyway sharding is not implemented yet in Redis backplane and in Redis client (stackexchange)
(Maybe Redis will add a PSPUUBLISH or SPPUBLISH in the future)

[BrennanConroy]

Oh right, I got that mixed up with clustering.

[mgravell]

As a footnote, SPUBLISH is indeed currently not present in SE.Redis; actually SPUBLISH is easy - the problem is there are some things we'd need to consider re reconnects and slot migrations - where we'd need to consider each time "does this connection still serve this slot?"

Not massively hard, just: something we haven't yet done.

[BrennanConroy]

This wouldn't work as proposed. Redis pattern matching is extremely limited in what it will do. See https://redis.io/commands/keys/ for the glob patterns it supports.

[BrennanConroy]

Closing as I don't think there is anything actionable here.