How it works

we will introduce How Consumer Dispatcher works in this section.

Before we start, let's go through how would you use RabbitMQ in a classic way.

Classic way

• Your publisher publishes jobs to a specific queue inside RabbitMQ
• Your consumer, in type of running a script (infinit loop) for instance, fetches and consumes jobs from the specific queue. assume consumer_script_a for the Consumer A in the above chart.
• If you need to speed up the consuming, you will run more the same script for consuming the same queue. as the chart says, we have many consumer_script_a running there to consume queue A.

Consumer Dispatcher way

• Your publisher publishes jobs to a specific queue inside RabbitMQ. same as classic way
• Consumer Dispatcher fetches jobs from each queue defined in config file then dispatches each job to your consumer code by sending http requests (http request format).
• Consumer Dispatcher waits (timeout for each request is configurable) until it gets http response from your consumer code. then Consumer Dispatcher acknowledges the job to the queue if it gets ok (case insensitive) as the content of HTTP response or resends the job if it gets anything else (like 502 HTTP status, or anything other than ok).
• If you need to speed up consuming one specific queue, you can open Consumer Dispatcher admin interface to add more working executors so that more number of jobs will be fetched from that queue at a time and be consumed.

How it works behind the scene

• you need to provide Consumer Dispatcher a job.xml which has a lot of job definitions.
• Consumer Dispatcher creates a thread pool which contains X number of threads for each job. (x='count' value defined in the job definition)
• each thread fetches job from a specific queue and dispatches it to your code.
• you can maintain the thread pool via JMX client. for example: adding/removing threads... etc.

The calls to your consumer code

Consumer Dispatcher dispatches the job that it gets from queues to your consumer code by sending HTTP calls. In this section, we will talk about the HTTP request and HTTP response.

Http request format

Consumer Dispatcher calls your consumer code by sending a HTTP POST request. There are 2 parameter inside each request:

• parameter name: queueName
  value: this is a misleading name, the value of it actually is the job name defined in your job.xml.

• parameter name: bodyData

  value: this is the content that you put into the queue by your publisher code. you can change the encoding in your job.xml.

Http response

Consumer Dispatcher waits (timeout for each request is configurable) until it gets http response from your consumer code.

• If the http response status is 200 AND the content of response body is ok literally (case insensitive), Consumer Dispatcher acknowledges the job to the queue.

• Any other case (like 502 http status, or any other http body than ok) will result in resending the job to your consumer code.