how to improve perf with multiple topics per socket

[rpgehlot]

Hello, I have been using uwebsocket in production environment under two different use cases.

Use case:1 Number of clients are larger but topics are much less, so using pub-sub works well here. So this use case is successful with uws since the underlying idea of maintaining topictree in uws and batching updates together for a event loop iteration yields good results.

Use case:2 Number of clients are smaller( ~20 ) but they have little high number of unique topics per socket they subscribe to which can be around ~10 as a rough number. But here i see huge lag in updates like minutes which basically seems to be happening becuase of larger number of overall topics.

the data input is same for both use cases. Cpu uasge is constantly 100% for second use case
topics are something like protocol/machine/team/account/positions
Is there any workaround through which we can improve perf in second use case ? would iterating over sockets for a topic and doing .send() call a better option here ?
Thanks.

[rpgehlot]

can anyone give any hints/ideas regarding this ?

[ghost]

The rule of thumb for publish is basically the same rule of thumb as for send; publish/send to topics/with sockets in ways to maximize locality - if you have 10 messages to send, send them to one socket a time, then move on to next. Same for publish; publish to topics in order of locality / relationship - use the sockets/topics that are the most related, in groups, don't just randomly publish.

[Shonke]

There is no upper limit of 64 topics, you can have millions of topics if you want to but the gain for using pub/sub over regular send is minimal if not negative in those cases. What the 64 limit is, is more an ideal number of simultaneously active topics. So ideal cases would have a handful of topics being shared among many sockets, rather than a buttload of topics.

Sorry to reply to closed discussions.
In your opinion, I should use fewer topics and broadcast to more clients.

Now I want to use multiple processes. I want to send the same topic to the same process for processing. That's why I want it.
#649

Do you have any suggestions about this?

[Shonke]

Or do I not need multiple processe to handle this? Will uWebSockets use all CPU cores?

[e3dio]

This discussion is outdated, v20 Publish is updated to do 65,536 unique messages per round instead of 64 unique topic combinations, topics are lightweight and using many is fine

[Shonke]

@e3dio Thank you for your answer.
Number of processes and CPU cores, is there any suggestion?

[ghost]

Repeating the conclusion:

The rule of thumb for publish is basically the same rule of thumb as for send; publish/send to topics/with sockets in ways to maximize locality - if you have 10 messages to send, send them to one socket a time, then move on to next. Same for publish; publish to topics in order of locality / relationship - use the sockets/topics that are the most related, in groups, don't just randomly publish.

[ghost]

LOL! My benchmark was run with ASAN and without optimization - with optimization the numbers are:

TestPerformance
Testing best case
41 ms
59 ms
68 ms
68 ms
67 ms
68 ms
69 ms
69 ms
69 ms
70 ms
1000000
Testing worst case
130 ms
134 ms
131 ms
128 ms
134 ms
134 ms
135 ms
131 ms
134 ms
133 ms

In other words; there is absolutely nothing wrong with the current algorithm; it beats my expectations by a lot.

[ghost]

The best addition would be to simply add isolations. Then you can do multi tenant stuff like said, where it is not really possible to organize publishes by locality, since they are isolated in nature. And you don't have to use it for only multi tenant but can even use it in many layers like tentant -> group -> topic by having the isolation be "tenant/group". That gives a lot of power to structure things really efficiently.

[e3dio]

The selected answer should be updated for v20 Publish, see #553 (reply in thread)

[Shonke]

The selected answer should be updated for v20 Publish

It'll be in the document, won't it?

[e3dio]

.

[ghost]

If you just emit the data without collecting it in intersectionCache (so having to use the cork buffer all the time), it gets like this:

TestPerformance
Testing worst possible case
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
47 ms
intersectionCache size is: 0
49 ms
intersectionCache size is: 0
47 ms

But now you have kind of optimized for worst case, and the best case is no longer nicely "cached".

[ghost]

I should make a best case benchmark where only 1 single topic is used for all sockets!

[ghost]

I mean it's really about where you do the collecting of messages into one; either you do it as part of intersectionCache, that is, you store the collected messages in a place where it can be accessed again without re-collecting the same messages (which obviously is best for shared topics), or you remove intersectionCache and pack, collect, the messages into one for every single socket (hoping for minimal sharing).

Either way you have to pack the different messages into one string, for every socket.

Having intersectionCache is quite fast (it is a red-black tree with O(log n) insert in worst case, but yeah it can hold a lot of duplicated data, however this is already something commonly done in other solutions:

A common solution to pub/sub is to have every socket hold its own user space buffer, where data is appended to (causing as much duplicated data to be held per socket as the intersectionCache). Then you finally copy that to kernel buffers.

So current cache is not really that much worse (in worst case) than what other pub/sub solutions are in average case. This while being much better in the cases where you do have shared topics (such as when all sockets are subscribed to 1 topic and you publish to that topic).

Overall, this implementation needs to be refactored, benchmarked and optimized and yes there can come more implementations but my benchmarking of it, simply shows that it is fairly okay already, even in worst case.

[ghost]

This code really needs to be rewritten, but I like the idea of it. 64 is not even something you need to limit it to - you can simply replace that for std::bitset<512> or whatever, and it works just fine. Same performance. So that raises the max topics triggered in one batch to 512.

[ghost]

It just happens to be ordered, there's still no guarantee (it can be changed later). The main benefit of sorting them like that is to remove duplicates (such as when subscribing to overlapping topics like wildcards)