Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 94 lines (68 sloc) 5.011 kb
2f0f53f7 » ryanbrainard
2012-06-06 Basic readme for async web-worker
1 ## Asynchronous Web-Worker Model Using RabbitMQ in Java
2
3 As explained in the [Worker Dynos, Background Jobs and Queueing](background-jobs-queueing) article, web requests
4 should be completed as fast as possible. If an operation may take a long time, it is best to send it to a worker
5 dyno to be processed in the background. This article demostrates this with an example application using Spring
6 [MVC](http://static.springsource.org/spring/docs/current/spring-framework-reference/html/mvc.html) and
7 [AMPQ](http://www.springsource.org/spring-amqp) with the Heroku [RabbitMQ add-on](https://addons.heroku.com/rabbitmq).
8
9 ### Clone the Reference Application
10
11 To get started, [clone the example reference application](https://api.heroku.com/myapps/devcenter-java-web-worker/clone).
12 This will automatically create your own copy of the app pre-configured with the RabbitMQ add-on.
13 Follow instructions in the cloned app to see a demostration of the model.
14
15 The [source code](https://github.com/heroku/devcenter-java-web-worker) of the reference application is also available for browsing or cloning.
16
17 ### Code Walkthrough
18
19 The application is comprised of two processes: `web` and `worker`.
20 The `web` process is a simple Spring MVC app that receives requests from users on the web and fowards them as messages to RabbitMQ for background processing.
21 The `worker` process is a simple Java app using Spring AMPQ that listens for new messages from RabbitMQ and processes them.
22 The `web` and `worker` processes can be scaled independently depending on application needs.
23
24 The application is structured as a Maven multi-module project with `web` and `worker` modules for each of the two
25 processes as well as a shared `common` module. The `common` module contains the common `BigOperation` model class and the
26 `RabbitConfiguration` class that reads the `RABBITMQ_URL` environment variable provided by the RabbitMQ add-on and
27 makes it available to the rest of the application:
28
29 @Bean
30 public ConnectionFactory connectionFactory() {
31 final URI rabbitMqUrl;
32 try {
33 rabbitMqUrl = new URI(getEnvOrThrow("RABBITMQ_URL"));
34 } catch (URISyntaxException e) {
35 throw new RuntimeException(e);
36 }
37
38 final CachingConnectionFactory factory = new CachingConnectionFactory();
39 factory.setUsername(rabbitMqUrl.getUserInfo().split(":")[0]);
40 factory.setPassword(rabbitMqUrl.getUserInfo().split(":")[1]);
41 factory.setHost(rabbitMqUrl.getHost());
42 factory.setPort(rabbitMqUrl.getPort());
43 factory.setVirtualHost(rabbitMqUrl.getPath().substring(1));
44
45 return factory;
46 }
47
48 #### Web Process
49 The `web` process has this configuration `@autowired` by Spring in `BigOperationWebController`:
50
51 @Autowired private AmqpTemplate amqpTemplate;
52 @Autowired private Queue rabbitQueue;
53
54 When web requests are received by the controller, they are coverted to AMPQ messages and sent to RabbitMQ.
55 The `AmqpTemplate` makes this easy with the following one-liner:
56
57 amqpTemplate.convertAndSend(rabbitQueue.getName(), bigOp);
58
59 The `web` process then immediately returns a confirmation page to the user.
60
61 #### Worker Process
62
63 Because the `worker` process is running in a sepatate dyno and is outside an application context,
64 the configuration must be manually wired from `RabbitConfiguration` in `BigOperationWorker`:
65
66 ApplicationContext rabbitConfig = new AnnotationConfigApplicationContext(RabbitConfiguration.class);
67 ConnectionFactory rabbitConnectionFactory = rabbitConfig.getBean(ConnectionFactory.class);
68 Queue rabbitQueue = rabbitConfig.getBean(Queue.class);
69 MessageConverter messageConverter = new SimpleMessageConverter();
70
71 To avoid polling for new messages the `worker` process sets up a `SimpleMessageListenerContainer`, which asynchronously
72 consumes messages by blocking until a message is delivered. First connection information must be provided:
73
74 SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
75 listenerContainer.setConnectionFactory(rabbitConnectionFactory);
76 listenerContainer.setQueueNames(rabbitQueue.getName());
77
78 Next, the listener is defined by implementing the `MessageListener` interface. This is where the actual message processing happens:
79
80 listenerContainer.setMessageListener(new MessageListener() {
81 public void onMessage(Message message) {
82 // message is converted back into model object
83 final BigOperation bigOp = (BigOperation) messageConverter.fromMessage(message);
84
85 // simply printing out the operation, but expensive computation could happen here
86 System.out.println("Received from RabbitMQ: " + bigOp);
87 }
88 });
89
90 The example application also configures an error handler and shutdown hook for completeness.
91
92 Finally the listener container is starter, which will stay alive until the JVM is shutdown:
93
94 listenerContainer.start();
Something went wrong with that request. Please try again.