RPC or PUB/SUB Not sure, need help!

[cliffdude]

Hi,

This looks like exactly what I need, I just can't figure out how to make it work! I am new to python so please bear with me and I apologize in advance for stupid questions that I probably should know that answer to.
That said, I need to build a "communicator tool" that can run on multiple PC's (Point of Sale Terminals) that sit behind firewalls, NAT and all that good stuff to essentially give me access to the local databases so that I can display sales data on a dashboard API / webapp. The dashboard / API / WEB server is a FastAPI based python app (or will be if I can get some data) and websockets seem to be the way to go as opening ports and forwarding NAT etc.. is just not an option for me.
Communication should be initiated from the client side then wait for SQL statements from the Dashboard server, execute them against the database and return the result as a JSON object, apart from the odd keep alive messages.

Question 1:
RPC or PUB/Sub? which is better suited for this purpose? Please remember that I will have multiple "clients" with databases connected from several locations.

Question2:
Regarding websockets_PUB/SUB from the server side:
My problem is that in the examples it is not clear how to get data back from the client after connecting to the server and sending a query, I have used endpoint.publish from the pub/sub library to send a query to the client that then executes it but I can't figure out how to "listen" for the result.

async def runquery(sqlquery):
    await endpoint.publish(["query"], data=sqlquery) #this works fine
    await asyncio.sleep(1)
    endpoint.subscribe(["query"], callback=get_result) #this does not work???
    return

async def get_result(data, topic):
    print(data)
    return data

Question 3:
Also from the server side regarding websockets_RPC.
I don't see how to get the RPCchannel object that is used here:

async def on_connect(channel): #The channel comes from the on_connect call 
    # now tell the client it can start sending us queries
    asyncio.create_task(channel.other.allow_queries())`

#BUT If I want to create a function like this:
class GetQueryData(RpcMethodsBase):
    async def getdata(sqlquery):
        result = asyncio.create_task(**DONT_KNOW_WHAT_GOES_HERE**.other.run_query_method(sqlquery))
        return result

#ON THE CLIENT SIDE I HAVE CREATED
`
class WaitingClient(RpcMethodsBase):
    async def run_query_method(self, sqlquery):
        queryresponse = await read_all(sqlquery)
        return queryresponse

Question 4:
How do I identify unique clients when there are many? I see a ws/{clientid} in some of the tests, but I have yet to figure out how to use it, so I would need to make sure I send the sqlquery to the intended client and also make sure that the result coming in is paired to the outgoing query. I have not gotten this far yet as I am still stuck on getting data back to the server!

Any help would be greatly appreciated and again sorry if I am asking silly questions.

[orweis]

Hi, Sorry for the delayed response.
And happy this might be a good fit for you :) (Does sound like it ;-) )

Question 1:
RPC or PUB/Sub? which is better suited for this purpose? Please remember that I will have multiple "clients" with databases connected from several locations.

RPC is 1x1, and Pub/Sub is meant as many-to-many. So Pub/Sub is the more natural option here.

Question2:
Regarding websockets_PUB/SUB from the server side:
My problem is that in the examples it is not clear how to get data back from the client after connecting to the server and sending a query, I have used endpoint.publish from the pub/sub library to send a query to the client that then executes it but I can't figure out how to "listen" for the result.

Take a look at this example/test: https://github.com/permitio/fastapi_websocket_pubsub/blob/master/tests/server_subscriber_test.py
Where a server both publishes and subscribes.

1 - It seems like you're trying to listen to your own event from the same PubSub client/server - the Notifier filters out the messages it sends from itself. You should run a seperate client to subscribe and receive the published event.
2- You need to subscribe to an event before it is sent to receive it. You can see in the example the server subscribes to the event on startup (via the FastAPI startup event - @app.on_event("startup"))

Question 3:
Also from the server side regarding websockets_RPC.
I don't see how to get the RPCchannel object that is used here:

In your method signature you forgot self - which has self.channel
See an example here -

fastapi_websocket_rpc/fastapi_websocket_rpc/rpc_methods.py

Line 79 in 647c0b4

async def call_me_back(self, method_name="", args={}) -> str:

Question 4:
How do I identify unique clients when there are many? I see a ws/{clientid} in some of the tests, but I have yet to figure out how to use it, so I would need to make sure I send the sqlquery to the intended client and also make sure that the result coming in is paired to the outgoing query. I have not gotten this far yet as I am still stuck on getting data back to the server!

You can use the Ids generated by the RPC layer- but note this will reset for every time a client restarts.
Usually you'd want to manage the persistent identity in a layer above (configuration, file, env-var, DB) and only communicate via the RPC/PubSub.
For example you can have each client publish a wake-up message to a joint topic that everyone listens on, so everyone would get the event including the ID.

Hope these answers help :)

[cliffdude]

Hi, Thank you very much for taking the time to reply.

Question 1/4: That's is what i thought, just wanted to be sure PUB/SUB it is. I will generate a CLIENT_ID and use that to listen on unique topics thus also solving question 4. Not sure quite how yet but I will figure something out.

Question 2: Dope, that was so obvious... I was missing endpoint.subscribe([TOPIC_FOR_CLIENT_REPLY], event_callback), I had tried to endpoint.subscribe to the original topic that I had endpoint.publish(ed) to but that did not work, it has to be a different topic name. Problem solved! I had looked at your example and that test but I did not see it for some reason.

Question 3 Yes of course... don't need it now (going with PUB/SUB)