Multiple Publishers & subscribers

[DrYSG]

How does one setup multiple publishers & subscribers with a TCP transport. I suspect that you don't do automatic mesh/bus creation. So one needs a unique IP bind point for each publishers, right? They just have the subscribers connect to each publisher on a single socket.

(this was discussed in: https://www.freelists.org/post/nanomsg/does-nanomsg-support-multi-producer-in-pubsub-mode,10 )

Is that basically correct?

The reason I lean to a pub/sub over a bus/mesh approach is because (and I admit I might well be mistaken) -

• I don't need a fully connected mesh
• I figure your radix -tree filtering at each node is better than what I would come up with
• I like the "auto-discovery" aspect of pub/sub as opposed to the manual wiring of the mesh for the bus transports
• (i.e. automatic "entry and exit from mesh")

Basically what I have is 2 producers (that mainly do publishing, but can receive occasional requests to put additional info on the stream being published, so they do need to listen)
And then about five consumers who mainly do receipt of the data stream from the publishers, but do need to occasional send requests to the producers.

And yes, I want the pub to be async, as as well as the recv() for the subscribe (no blocking is allowed in the context where I am using this).

So this is a bi-directional pub/sub architecture. And I am looking at the simplest way to implement this. (traffic is pretty light).

---

Of course a UDP transport would be nice for this, but I am not holding by breath.

[gdamore]

The problem with PUB/SUB is that it is strictly one-way. That is PUB sockets always send, and SUB sockets always receive. There is no way to receive on a PUB socket, nor to send on a SUB socket.

So you can use a secondary socket to do receive operations. It is fairly common to use a REQ/REP socket pair combined with PUB/SUB for example.

Note that connection establishment (bind(), listen(), connect()) is orthogonal to the pattern you use. Typically PUB sockets are servers, and SUB sockets are clients, but there are use cases that are the reverse of that.

PUB/SUB is implemented a "fanout" of unicast TCP (or other transports) streams, and so you'll have to do the work to setup listeners, etc. In this regards its no different than the BUS topology.

Any of these can be done asynchronously, as we have asynchronous APIs for each of the patterns. See nng_recv_aio() and nng_send_aio() for details. (Technically, the PUB side send operation also never blocks; it includes a queue for message delivery, and anytime it encounters backpressure on the queue it just silently discards the message and moves on. SUB side receives do block unless you use an asynchronous API.)

UDP is something we are looking at for the future; I have a design for this and am hoping to obtain commercial sponsorship for the work.

[DrYSG]

Do tell me if I am bothering you too much. I can certainly find out the answers by experimentation. It was just my thought that I could ask a few focused questions, and perhaps that this would make it's way to a FAQ and perhaps also help the book.

I do like the multiple endpoint approach (mix of PUB/SUB with a separate channel for REQ/REP). Can one also try two "channels" of pub/sub with one going in one direction and one in the other (I assume the answer is yes, here again, experimentation could tell me this).

Also I am still trying to wrap my mind around what the topology and listen/bind would look if I want two nodes each doing a publish protocol, with 5+ subscribers that are listening to both of them equally.

Lets assume server A: tcp://192.168.0.0:7777 and B: tcp://192.168.0.1:7777 Then I assume that the servers should use their unique endpoints IPs for the listen() call, and that the clients should dial twice. Once to one server and once to the other server?

Finally my code runs as plug-in DLL inside of a game engine that has a single thread main loop. I am a little coy in trying to do too much fancy stuff with thread pools, completion IOs etc.. I don't know how that would effect things. Although I do like what you illustrated in: https://github.com/nanomsg/nng/blob/master/demo/async/server.c with context aio structs. Very clean. Bless you.

Anyway, I wanted to do something quick that I think would also work to deal with the blocking recv() calls. I see that polling is discouraged .poll(compat3) but what about getting the effect of a poll (my game loop runs at 60FPS) by adding a NNG_FLAG_NONBLOCK, to the .recv()?

https://nanomsg.github.io/nng/man/v1.0.0/nng_recv.3.html

From the documentation, it is not clear what the response if nothing is in the queue for the socket. But again, I could figure this out by trial and error. (but would be nice to put this in the documentation).

[gdamore]

The best place for this is on the mailing list, rather than in github issues. I prefer to have github issues for actionable things (bugs, feature requests), etc.

First question: of course. You can intermix as many sockets as you have system resources for. (Which should mean at least a hundred, maybe thousands. Really depends on system limits relating to open files.)

Your description of servers A and B is correct, and clients would indeed need to dial both of them.

You can indeed do a nonblocking recv -- its really inefficient, but its ok.

The admonition against using poll is specifically against nn_poll() or the pollable file descriptors. Those are rather inefficient in more ways than just being by polling... they consume an extra file descriptor and several extra system calls for each operation.

If you use nng_recv() and NNG_FLAG_NONBLOCK, and no messages are pending for receive, you'll get NNG_EAGAIN. (The man page almost got this right, but now I see it says "for sending" rather than "receiving". Copy pasta error.)

[gdamore]

Going ahead and closing it, since it isn't an actionable bug or RFE.

[DrYSG]

@gdamore you beat me by 8 hours. I was about to do the same. Thank you for the fantastic support. I think I have it working now. I hope we can be in touch later about UDP, but for this project it probably is not necessary (but some of our embedded SWIL and HWIL work might well need it).