Skip to content

Home

Jump to bottom Edit New page
shahprincea edited this page Dec 10, 2014 · 16 revisions

Motivation

As of today there are many queues which follows AMPQ and STOMP protocol. To name few RabbitMQ, Kafka, ActiveQ etc. However, with each queue there are pro and cons. This project is centered around various use cases and evaluate how RabbitMQ handles them.

Environment

SingleNode Setup

  • OS : Ubuntu 14.04 on a VM (VMWare fusion)
  • CPU : Intel® Core™ i7-4870HQ CPU @ 2.50GHz × 2
  • Memory : 3.9 GiB
  • OS Type : 64-bit
  • Disk : 16.8 GB
  • RabbitMQ Server : 3.2.4
  • RabbitMQ java Client : 3.4.1

MultiNode Setup

We have 3 node cluster build using above single Node machines.

ref: http://www.rabbitmq.com/clustering.html

Installing RabbitMQ

  1. Add the following line to your /etc/apt/sources.list: "deb http://www.rabbitmq.com/debian/ testing main"
  2. wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
  3. sudo apt-key add rabbitmq-signing-key-public.asc
  4. Run apt-get update.
  5. sudo apt-get install rabbitmq-server
  6. rabbitmqctl start/stop/status to start/stop/status rabbitmq server
  7. Install rabbitMQ Management Plugin see https://www.rabbitmq.com/management.html

Note: logs are located at /var/log/rabbitmq directory

ref: http://www.rabbitmq.com/install-debian.html

Use cases (Order matters)

  1. Consumers can be added/removed dynamically. (done)
  2. If consumer dies after dequeueing that msg should be re-delivered to other consumer.(done)
  3. Queue should never take away work from consumer even if consumer takes forever to complete.(done)
  4. Messages should survive restart of RabbitMQ. (done)
  5. Queue should never loose msg even if Consumer dies (with/without ack).
  6. Queue should support Batch operations.
  7. Queue should support various msg sizes (2k, 20K, 200K).(done)
  8. Latency between msg produced to consumed (durable and non durable msg) for various msg size.(good to have)
  9. Queue should not slow down when you have multiple consumers (High Availability).(good to have)
  10. Queue should block producers in case of Disk full (or reaching some threshold).(done)
  11. Queue should be bounded.(done)
  12. Queues should be distributed in cluster.
  13. Thoughts on Disk Node vs RAM Node.(good to have)
  14. Publisher should be able to reliably publish msg to exchange/queue.
  15. Does consumer/producer pool for connection or dies if RabbitMQ server restarts. (done)
  16. Tested mirrored queue
  17. Get queue Size

rabbitMQ

Add a custom sidebar
Clone this wiki locally