Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 96 lines (69 sloc) 5.081 kB
2f0f53f @ryanbrainard Basic readme for async web-worker
ryanbrainard authored
1 ## Asynchronous Web-Worker Model Using RabbitMQ in Java
2
3 As explained in the [Worker Dynos, Background Jobs and Queueing](background-jobs-queueing) article, web requests
4 should be completed as fast as possible. If an operation may take a long time, it is best to send it to a worker
5 dyno to be processed in the background. This article demostrates this with an example application using Spring
6 [MVC](http://static.springsource.org/spring/docs/current/spring-framework-reference/html/mvc.html) and
a824d23 @ryanbrainard Use CloudAMPQ addon instead of private RabbitMQ
ryanbrainard authored
7 [AMPQ](http://www.springsource.org/spring-amqp) with the Heroku [CloudAMPQ add-on](https://addons.heroku.com/cloudamqp),
8 which provides [RabbitMQ](http://www.rabbitmq.com/) as a service.
2f0f53f @ryanbrainard Basic readme for async web-worker
ryanbrainard authored
9
10 ### Clone the Reference Application
11
12 To get started, [clone the example reference application](https://api.heroku.com/myapps/devcenter-java-web-worker/clone).
13 This will automatically create your own copy of the app pre-configured with the RabbitMQ add-on.
14 Follow instructions in the cloned app to see a demostration of the model.
15
16 The [source code](https://github.com/heroku/devcenter-java-web-worker) of the reference application is also available for browsing or cloning.
17
18 ### Code Walkthrough
19
20 The application is comprised of two processes: `web` and `worker`.
21 The `web` process is a simple Spring MVC app that receives requests from users on the web and fowards them as messages to RabbitMQ for background processing.
22 The `worker` process is a simple Java app using Spring AMPQ that listens for new messages from RabbitMQ and processes them.
23 The `web` and `worker` processes can be scaled independently depending on application needs.
24
25 The application is structured as a Maven multi-module project with `web` and `worker` modules for each of the two
26 processes as well as a shared `common` module. The `common` module contains the common `BigOperation` model class and the
a824d23 @ryanbrainard Use CloudAMPQ addon instead of private RabbitMQ
ryanbrainard authored
27 `RabbitConfiguration` class that reads the `CLOUDAMQP_URL` environment variable provided by the RabbitMQ add-on and
2f0f53f @ryanbrainard Basic readme for async web-worker
ryanbrainard authored
28 makes it available to the rest of the application:
29
30 @Bean
31 public ConnectionFactory connectionFactory() {
32 final URI rabbitMqUrl;
33 try {
a824d23 @ryanbrainard Use CloudAMPQ addon instead of private RabbitMQ
ryanbrainard authored
34 rabbitMqUrl = new URI(getEnvOrThrow("CLOUDAMQP_URL"));
2f0f53f @ryanbrainard Basic readme for async web-worker
ryanbrainard authored
35 } catch (URISyntaxException e) {
36 throw new RuntimeException(e);
37 }
38
39 final CachingConnectionFactory factory = new CachingConnectionFactory();
40 factory.setUsername(rabbitMqUrl.getUserInfo().split(":")[0]);
41 factory.setPassword(rabbitMqUrl.getUserInfo().split(":")[1]);
42 factory.setHost(rabbitMqUrl.getHost());
43 factory.setPort(rabbitMqUrl.getPort());
44 factory.setVirtualHost(rabbitMqUrl.getPath().substring(1));
45
46 return factory;
47 }
48
49 #### Web Process
50 The `web` process has this configuration `@autowired` by Spring in `BigOperationWebController`:
51
52 @Autowired private AmqpTemplate amqpTemplate;
53 @Autowired private Queue rabbitQueue;
54
55 When web requests are received by the controller, they are coverted to AMPQ messages and sent to RabbitMQ.
56 The `AmqpTemplate` makes this easy with the following one-liner:
57
58 amqpTemplate.convertAndSend(rabbitQueue.getName(), bigOp);
59
60 The `web` process then immediately returns a confirmation page to the user.
61
62 #### Worker Process
63
64 Because the `worker` process is running in a sepatate dyno and is outside an application context,
65 the configuration must be manually wired from `RabbitConfiguration` in `BigOperationWorker`:
66
67 ApplicationContext rabbitConfig = new AnnotationConfigApplicationContext(RabbitConfiguration.class);
68 ConnectionFactory rabbitConnectionFactory = rabbitConfig.getBean(ConnectionFactory.class);
69 Queue rabbitQueue = rabbitConfig.getBean(Queue.class);
70 MessageConverter messageConverter = new SimpleMessageConverter();
71
72 To avoid polling for new messages the `worker` process sets up a `SimpleMessageListenerContainer`, which asynchronously
73 consumes messages by blocking until a message is delivered. First connection information must be provided:
74
75 SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
76 listenerContainer.setConnectionFactory(rabbitConnectionFactory);
77 listenerContainer.setQueueNames(rabbitQueue.getName());
78
79 Next, the listener is defined by implementing the `MessageListener` interface. This is where the actual message processing happens:
80
81 listenerContainer.setMessageListener(new MessageListener() {
82 public void onMessage(Message message) {
83 // message is converted back into model object
84 final BigOperation bigOp = (BigOperation) messageConverter.fromMessage(message);
85
86 // simply printing out the operation, but expensive computation could happen here
87 System.out.println("Received from RabbitMQ: " + bigOp);
88 }
89 });
90
91 The example application also configures an error handler and shutdown hook for completeness.
92
93 Finally the listener container is starter, which will stay alive until the JVM is shutdown:
94
95 listenerContainer.start();
Something went wrong with that request. Please try again.