Permalink
Browse files

Small word tweaks, use of callouts to denote auxiliary info, proper s…

…yntax highlighting.
  • Loading branch information...
1 parent 041a8cf commit 3c1a18f8b11f78c8e42fa7abe3452508c3f17422 @rwdaigle rwdaigle committed Jun 27, 2012
Showing with 32 additions and 30 deletions.
  1. +32 −30 ARTICLE.md
View
62 ARTICLE.md
@@ -1,36 +1,42 @@
Scheduled Jobs with Custom Clock Processes in Java with Quartz
==============================================================
-There are numerous ways to schedule background jobs in Java applications. This article will teach you how to setup a Java application that uses the Quartz scheduling library along with RabbitMQ to create a scalable and reliable method of doing background processing in Java applications on Heroku.
+There are numerous ways to [schedule background jobs](https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes) in Java applications. This article will teach you how to setup a Java application that uses the Quartz scheduling library along with RabbitMQ to create a scalable and reliable method of doing background processing in Java applications on Heroku.
Many of the common methods for background processing in Java advocate running background jobs within the same application as the web tier. This approach has scalability and reliability constraints. A better approach is to move background jobs into separate processes so that the web tier is distinctly separate from the background processing tier. This allows the web tier to be exclusively for handling web requests. The scheduling of jobs should also be it's own tier that puts jobs onto a queue. The worker processing tier can then be scaled independently from the rest of the application.
-For more information on this architecture see the [Scheduled Jobs and Custom Clock Processes](https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes) article. All of the source code for this article is [available on GitHub](http://github.com/heroku/devcenter-java-quartz-rabbitmq). To follow along you will need the following prerequisites:
+<div class="note" markdown="1">
+The source for this article's reference application is [available on GitHub](http://github.com/heroku/devcenter-java-quartz-rabbitmq).
+</div>
-* [Heroku Toolbelt](http://toolbelt.heroku.com)
-* [Maven 3.0.4](http://maven.apache.org/download.html)
-* [Git Command Line](http://git-scm.com/download)
+## Prerequisites
+
+You will need to have the [Heroku Toolbelt](https://toolbelt.heroku.com) installed as well as [Maven 3.0.4](http://maven.apache.org/download.html) to follow along.
To clone the sample project to your local computer run:
+ :::term
$ git clone https://github.com/heroku/devcenter-java-quartz-rabbitmq.git
-
Scheduling Jobs with Quartz
---------------------------
-A Job Scheduler / Custom Clock Process will be used to create jobs and add them to a queue. To setup a custom clock process use the [Quartz](http://quartz-scheduler.org) library. With Maven the dependency will be:
+A job scheduling [custom clock process](https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes) will be used to create jobs and add them to a queue. To setup a custom clock process use the [Quartz](http://quartz-scheduler.org) library. In Maven the dependency is declared with:
+
+<div class="callout" markdown="1">
+See the reference app's [pom.xml](https://github.com/heroku/devcenter-java-quartz-rabbitmq/blob/master/pom.xml) for the full Maven build definition.
+</div>
+ :::xml
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>2.1.5</version>
</dependency>
-See the project's [pom.xml](https://github.com/heroku/devcenter-java-quartz-rabbitmq/blob/master/pom.xml) for the full Maven build definition.
-
Now a Java application can be used to schedule jobs. Here is an example:
+ :::java
package com.heroku.devcenter;
import org.quartz.*;
@@ -81,12 +87,12 @@ To test this application locally you can run the Maven build and then run the `S
If the `HelloJob` actually did work itself then we would have a runtime bottleneck because we could not scale the scheduler and avoid duplicate jobs being scheduled. Quartz does have a JDBC module that can use a database to prevent jobs from being duplicated but a simpler approach is to only run one instance of the scheduler and have the scheduled jobs added to a message queue where they can be processes in parallel by job worker processes.
-
Queuing Jobs with a RabbitMQ
----------------------------
RabbitMQ can be used as a message queue so the scheduler process can be used just to add jobs to a queue and worker processes can be used to grab jobs from the queue and process them. To add the RabbitMQ client library as a dependency in Maven specify the following in dependencies block of the `pom.xml` file:
+ :::xml
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
@@ -95,11 +101,11 @@ RabbitMQ can be used as a message queue so the scheduler process can be used jus
If you want to test this locally then [install RabbitMQ](http://www.rabbitmq.com/download.html) and set an environment variable that will provide the application the connection information to your RabbitMQ server.
-* On Windows:
+On Windows:
$ set CLOUDAMQP_URL="amqp://guest:guest@localhost:5672/%2f"
-* On Mac/Linux:
+On Mac/Linux:
$ export CLOUDAMQP_URL="amqp://guest:guest@localhost:5672/%2f"
@@ -108,6 +114,7 @@ The `CLOUDAMQP_URL` environment variable will be used by the scheduler and worke
The `SchedulerMain` class needs to be updated to add a new message onto a queue every time the `HelloJob` is executed. Here is the new `HelloJob` class from the [SchedulerMain.java file in the sample project](https://github.com/heroku/devcenter-java-quartz-rabbitmq/blob/master/src/main/java/com/heroku/devcenter/SchedulerMain.java):
+ :::java
public static class HelloJob implements Job {
@Override
@@ -141,12 +148,12 @@ The `SchedulerMain` class needs to be updated to add a new message onto a queue
In this example every time the `HelloJob` is executed it adds a message onto a RabbitMQ message queue simply containing a String with the time the String was created. Running the updated `SchedulerMain` should add a new message to the queue every 5 seconds.
-
Processing Jobs From a Queue
----------------------------
-Now lets create a Java application that will pull messages from the queue and handle them. This application will also use the `RabbitFactoryUtil` to get a connection to RabbitMQ from the `CLOUDAMQP_URL` environment variable. Here is the `WorkerMain` class from the [WorkerMain.java file in the example project](https://github.com/heroku/devcenter-java-quartz-rabbitmq/blob/master/src/main/java/com/heroku/devcenter/WorkerMain.java):
+Next, create a Java application that will pull messages from the queue and handle them. This application will also use the `RabbitFactoryUtil` to get a connection to RabbitMQ from the `CLOUDAMQP_URL` environment variable. Here is the `WorkerMain` class from the [WorkerMain.java file in the example project](https://github.com/heroku/devcenter-java-quartz-rabbitmq/blob/master/src/main/java/com/heroku/devcenter/WorkerMain.java):
+ :::java
package com.heroku.devcenter;
import com.rabbitmq.client.Channel;
@@ -187,32 +194,31 @@ Now lets create a Java application that will pull messages from the queue and ha
}
-
This class simply waits for new messages on the message queue and logs that it received them. You can run this example locally by doing a build and then running the `WorkerMain` class:
$ mvn package
$ java -cp target/classes:target/dependency/* com.heroku.devcenter.WorkerMain
You can also run multiple instances of this example locally to see how the job processing can be horizontally distributed.
-
Running on Heroku
-----------------
-Now that you have everything working locally you can run this on Heroku. First create a new file named `Procfile` containing:
+Now that you have everything working locally you can run this on Heroku. First declare the [process model](https://devcenter.heroku.com/articles/process-model) in a new file named `Procfile` containing:
scheduler: java $JAVA_OPTS -cp target/classes:target/dependency/* com.heroku.devcenter.SchedulerMain
worker: java $JAVA_OPTS -cp target/classes:target/dependency/* com.heroku.devcenter.WorkerMain
-This defines two processes that can be executed on Heroku; one named `scheduler` for the `SchedulerMain` application and one named `worker` for the `WorkerMain` application.
+This defines two process types that can be executed on Heroku; one named `scheduler` for the `SchedulerMain` app and one named `worker` for the `WorkerMain` app.
-To run these applications on Heroku you will need to push a Git repository to Heroku containing the Maven build descriptor, source code, and `Procfile`. If you cloned the example project then you already have a Git repository. If you need to create a new git repository containing these files, run:
+To run on Heroku you will need to push a Git repository to Heroku containing the Maven build descriptor, source code, and `Procfile`. If you cloned the example project then you already have a Git repository. If you need to create a new git repository containing these files, run:
+ :::term
$ git init
$ git add src pom.xml Procfile
$ git commit -m init
-With the necessary files in a Git repository create a new application on Heroku from within the project's root directory:
+Create a new application on Heroku from within the project's root directory:
$ heroku create
@@ -224,32 +230,28 @@ Now push your Git repository to Heroku:
$ git push heroku master
-This will run the Maven build for your project on Heroku and create a slug file containing the executable assets for your application. To run the application you will need to allocate Dynos to run each process. You should only allocate one Dyno to run the `scheduler` process to avoid duplicate job scheduling. You can allocate as many Dynos as needed to the `worker` process since it is event driven and parallelizable through the RabbitMQ message queue.
+This will run the Maven build for your project on Heroku and create a slug file containing the executable assets for your application. To run the application you will need to allocate dynos to run each process type. You should only allocate one dyno to run the `scheduler` process type to avoid duplicate job scheduling. You can allocate as many dynos as needed to the `worker` process type since it is event driven and parallelizable through the RabbitMQ message queue.
-To allocate one Dyno to the `scheduler` process run:
+To allocate one dyno to the `scheduler` process type run:
$ heroku scale scheduler=1
This should begin adding messages to the queue every two seconds.
-To allocate two Dynos to the `worker` process run:
+To allocate two dynos to the `worker` process type run:
$ heroku scale worker=2
-This will allocate two Dynos that will run the `WorkerMain` application which will pull messages from the queue and process them.
-
-You can verify that this is happening by watching the Heroku logs for your application. To open a feed of your logs run:
+This will provision two dynos, each which will run the `WorkerMain` app and pull messages from the queue for processing. You can verify that this is happening by watching the Heroku logs for your application. To open a feed of your logs run:
+ :::term
$ heroku logs -t
-
-You should see something like:
-
2012-06-26T22:26:47+00:00 app[scheduler.1]: 100223 [DefaultQuartzScheduler_Worker-1] INFO com.heroku.devcenter.SchedulerMain - Message Sent: Sent at:1340749607126
2012-06-26T22:26:47+00:00 app[worker.2]: 104798 [main] INFO com.heroku.devcenter.WorkerMain - Message Received: Sent at:1340749607126
2012-06-26T22:26:52+00:00 app[scheduler.1]: 105252 [DefaultQuartzScheduler_Worker-2] INFO com.heroku.devcenter.SchedulerMain - Message Sent: Sent at:1340749612155
2012-06-26T22:26:52+00:00 app[worker.1]: 109738 [main] INFO com.heroku.devcenter.WorkerMain - Message Received: Sent at:1340749612155
-In this example execution the scheduler creates 2 messages which are handled by the two different worker Dynos. This shows that the work is being scheduled and distributed correctly.
+In this example execution the scheduler creates 2 messages which are handled by the two different worker dynos (`worker.1` and `worker.2`). This shows that the work is being scheduled and distributed correctly.
Further Leaning
---------------

0 comments on commit 3c1a18f

Please sign in to comment.