Skip to content

support RabbitMQ #132

@VigneshVSV

Description

@VigneshVSV

Please describe.
We want to start off by implementing a PoC for req-rep pattern (so only property and action operations). The messaging contract will be based on our RPC contract as it is, except its not multipart anymore. Some information on the RPC broker is here - https://docs.hololinked.dev/design/zmq/

Use cases
Distributed Storage Persistent Queueing applications

Describe the solution you'd like
Rough steps (that can be adapted according to the solution):

  • deploy rabbitmq, say using a docker container. Define a class RabbitMQProducer/RabbitMQServer under hololinked.servers and import aiopika. Add aiopika to pyproject.toml.
  • The primary job is to be simply pop the message from rabbitmq, convert it into a ZMQ multipart and put it into the ZMQ broker. Once a reply arrives, (which will be ZMQ multipart again), put it back into rabbitmq. Note that all RPC will be carried by ZMQ only usually over shared memory, thus maintaining performance despite an additional layer of complexity. There are some notes here which may be helpful - https://docs.hololinked.dev/design/zmq/
  • There should be two queues, one for requests & one for replies. Upon server initialization, create the two queues, set the persistence, qos etc. as how the user requests. By default, the queue must be persistent and qos should be 1. Note that the next message from the queue must be popped only when the current message has completed, otherwise the message should remain in the queue, which might itself be a persistent queue.
  • Upon run(), start the aio pika consumer
  • On hololinked.client, create a rabbitmq client protocol binding on ClientFactory. Add all operations for properties and actions. The code in hololinked.client.zmq should help create these, as the messaging contract is similar.
  • Add tests protocol_14_rabbitmq.py. If everything is successful, consider adding rabbitmq in the docker compose file in this repo

Additional context
If contributing first time, CONTRIBUTING.md can help.

There might be inherent unknowns while converting a RabbitMQ message to ZMQ multipart. This issue can be harder to complete than expected or uncompleted at all. Please note this before starting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions