Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

update assets + README - mark (unfinished) Resque support as "comming…

… soon"
  • Loading branch information...
commit f750a67296157e2029c3afb3b63aac20b8ebc94c 1 parent 8939183
@kares authored
View
2  LICENSE
@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
- Copyright (c) 2010 Karol Bucek
+ Copyright (c) 2010-2012 Karol Bucek
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
View
167 README.md
@@ -1,48 +1,47 @@
-JRuby Rack Worker
-=================
+# JRuby Rack Worker
-Java based thread worker implementation over [jruby-rack](http://github.com/jruby/jruby-rack).
+Thread based workers on top of [jruby-rack](http://github.com/jruby/jruby-rack).
-Natively supports [Delayed::Job](http://github.com/collectiveidea/delayed_job) and
-[Navvy](http://github.com/jeffkreeftmeijer/navvy) but one can easily write his own
-worker loop.
+With out of the box [JRuby](http://jruby.org) "adapters" for:
+* [Resque](http://github.com/defunkt/resque) (**COMING SOON**)
+* [Delayed::Job](http://github.com/collectiveidea/delayed_job)
+* [Navvy](http://github.com/jeffkreeftmeijer/navvy)
-Motivation
-----------
+... but one can easily write/adapt his own worker loop.
-While migrating a rails application to [JRuby](http://jruby.org) I found myself
-stuck with [Delayed::Job](http://github.com/collectiveidea/delayed_job). I wanted
-to deploy the application without having to spawn a separate daemon process in
-another *Ruby* (as *JRuby* is not daemonizable the [daemons](http://daemons.rubyforge.org)
-way).
-Well, why not spawn a "daemon" thread looping over the jobs from the servlet
-container ... after all the java world is inherently thread-oriented !
+## Motivation
-This does have the advantage of keeping the deployment simple and saving some
-precious memory (most notably with `threadsafe!` mode) that would have been
-eaten by the separate process. Besides, your daemons start benefiting from
-JRuby's (as well as Java's) runtime optimalizations ...
+Ruby attempts to stay pretty close to UNIX and most popular workers have been
+modeled the spawn a background process way. [JRuby](http://jruby.org) brings
+Java to the table, where "Young Java Knights" are thought to use threads
+whenever in a need to compute something parallel while serving requests.
+
+There's no right or wrong way of doing this. If you do expect chaos like Resque
+proclaims - have long running jobs that consume a lot of memory they have trouble
+releasing (e.g. due C extensions) run a separate process for sure.
+But otherwise (after all C exts usually have a native Java alternative on JRuby)
+having predictable thread-safely written workers, one should be fine with
+running them concurrently as part of the application in a daemon thread.
-On the other hand your jobs should be simple and complete "fast" (in a rate of
-seconds rather than several minutes or hours) as they will restart and live with
-the lifecycle of the deployed application and/or application server.
+This does have the advantage of keeping the deployment simple and saving some
+precious memory (most notably with `threadsafe!` mode) that would have been
+eaten by the separate process. Besides, your application might warm up faster
+and start benefiting from JRuby's runtime optimalizations slightly sooner ...
-Java purist might objects the servlet specification does not advise spawning
-daemon threads in a servlet container, objection noted. Whether this style of
-asynchronous processing suits your limits, needs and taste is entirely up to
-You.
+On the other hand your jobs should be fairly simple and complete "fast" (in a
+rate of seconds rather than several minutes or hours) as they will live and
+restart with the lifecycle of the deployed application and application server.
-Setup
-=====
+## Setup
Copy the `jruby-rack-worker.jar` into the `lib` folder or the directory being
mapped to `WEB-INF/lib` e.g. `lib/java`.
-Configure the worker in `web.xml`, You'll need to add a servlet context listener
-that will start threads when You application boots and a script to be executed
+Configure the worker in `web.xml`, you'll need to add a servlet context listener
+that will start threads when your application boots and a script to be executed
(should be an "endless" loop-ing script). Sample configuration :
<context-param>
@@ -57,55 +56,81 @@ that will start threads when You application boots and a script to be executed
<listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
</listener>
-**NOTE**: The `WorkerContextListener` needs to be executed (and thus configured)
-after the `RailsServletContextListener`/`RackServletContextListener` as it expects
-the *jruby-rack* environment to be available.
+The `WorkerContextListener` needs to be executed (and thus configured) after the
+`RailsServletContextListener`/`RackServletContextListener` as it expects the
+*jruby-rack* environment to be available.
-**NOTE**: If You're not using `threadsafe!` than You really **should** ...
+Sample deployment descriptor including optional parameters:
+[web.xml](/kares/jruby-rack-worker/blob/master/src/test/resources/sample.web.xml).
-**NOTE**: If You're still not using `threadsafe!` mode than You're polling several
-(non-thread-safe) JRuby runtimes instances while serving requests, the *workers
-are nor running as a part of the application* thus each worker thread will remove
-and use (block) an application runtime from the instance pool (consider it while
-setting the `jruby.min.runtimes`/`jruby.max.runtimes` parameters) !
+### Warbler
-Sample Rails `web.xml` usable with [Warbler](http://caldersphere.rubyforge.org/warbler)
-including optional configuration parameters
-[web.xml](/kares/jruby-rack-worker/blob/master/src/test/resources/warbler.web.xml).
+If you're using [Warbler](http://caldersphere.rubyforge.org/warbler) to assemble
+your application you might simply declare a gem dependency with Bundler as your
+gems will be scanned for jars and packaged correctly:
-A simpler configuration using the built-in `Delayed::Job` / `Navvy` support :
+ gem 'jruby-rack-worker', :platform => :jruby, :require => nil
- <context-param>
- <param-name>jruby.worker</param-name>
- <param-value>delayed_job</param-value> <!-- or navvy -->
- </context-param>
+Otherwise copy the jar into your *warble.rb* configured `config.java_libs`.
- <listener>
- <listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
- </listener>
+Warbler checks for a *config/web.xml.erb* thus configure the worker there, e.g. :
+ <!DOCTYPE web-app PUBLIC
+ "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
+ "http://java.sun.com/dtd/web-app_2_3.dtd">
+ <web-app>
+ <% webxml.context_params.each do |k,v| %>
+ <context-param>
+ <param-name><%= k %></param-name>
+ <param-value><%= v %></param-value>
+ </context-param>
+ <% end %>
-Build
-=====
+ <filter>
+ <filter-name>RackFilter</filter-name>
+ <filter-class>org.jruby.rack.RackFilter</filter-class>
+ </filter>
+ <filter-mapping>
+ <filter-name>RackFilter</filter-name>
+ <url-pattern>/*</url-pattern>
+ </filter-mapping>
-[JRuby](http://jruby.org) 1.5+ is required to build the project.
-The build is performed by [rake](http://rake.rubyforge.org) which should be part
-of the JRuby installation, if You're experiencing conflicts with another Ruby and
-it's rake executable use `jruby -S rake` instead of the bare `rake` command.
+ <listener>
+ <listener-class><%= webxml.servlet_context_listener %></listener-class>
+ </listener>
-Build the `jruby-rack-worker.jar` using :
+ <% if webxml.jndi then [webxml.jndi].flatten.each do |jndi| %>
+ <resource-ref>
+ <res-ref-name><%= jndi %></res-ref-name>
+ <res-type>javax.sql.DataSource</res-type>
+ <res-auth>Container</res-auth>
+ </resource-ref>
+ <% end; end %>
- rake jar
+ <!-- jruby-rack-worker setup using the built-in libraries support : -->
-Build the gem (includes the jar) :
+ <context-param>
+ <param-name>jruby.worker</param-name>
+ <param-value>delayed_job</param-value> <!-- or resque or navvy -->
+ </context-param>
- rake gem
+ <listener>
+ <listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
+ </listener>
+
+ </web-app>
-Run the tests with :
- rake test
+If you're deploying a Rails application on JRuby it's highly **recommended** to
+uncomment `config.threadsafe!`. Otherwise, if unsure or you're code is not
+thread-safe yet you'll end up polling several JRuby runtimes in a single process,
+in this case however each worker thread will use (block) an application runtime
+from the pool (consider it while setting
+`jruby.min.runtimes` and `jruby.max.runtimes` parameters).
+### Custom Workers
+
Worker Migration
================
@@ -131,3 +156,21 @@ most probably need to start by looking at the current worker spawning script
See the [Delayed::Job](/kares/jruby-rack-worker/tree/master/src/main/ruby/delayed)
JRuby "adapted" worker code for inspiration.
+
+
+## Build
+
+[JRuby](http://jruby.org) 1.5+ is required to build the project.
+The build is performed by [rake](http://rake.rubyforge.org) which should be part
+of your JRuby installation, if you're experiencing conflicts with another Ruby and
+it's rake executable use `jruby -S rake` instead of the bare `rake` command.
+Besides you'll to need [ant](http://ant.apache.org/) installed for the Java part.
+
+Build the `jruby-rack-worker.jar` using :
+
+ rake jar
+
+Build the gem (with the jar packaged) :
+
+ rake gem
+
View
61 rakefile → Rakefile
@@ -156,32 +156,57 @@ task :'test:compile' => :compile do
end
end
-desc "run tests"
-task :test => :'test:compile' do
- mkdir_p TEST_RESULTS_DIR
- ant.junit :fork => true,
- :haltonfailure => false,
- :haltonerror => true,
- :showoutput => true,
- :printsummary => true do
+task :'bundler:setup' do
+ begin
+ require 'bundler/setup'
+ rescue
+ puts "Please install Bundler and run `bundle install` to ensure you have all dependencies"
+ end
+ require 'appraisal'
+end
- classpath :refid => "main.class.path"
- classpath :refid => "test.class.path"
- classpath do
- pathelement :path => MAIN_BUILD_DIR
- pathelement :path => TEST_BUILD_DIR
- end
+namespace :test do
+
+ desc "run ruby tests"
+ task :ruby => 'bundler:setup' do
+ Rake::Task['jar'].invoke unless File.exists?(out_jar_path)
+ test = ENV['TEST'] || File.join(Dir.getwd, "src/test/ruby/**/*_test.rb")
+ test_opts = (ENV['TESTOPTS'] || '').split(' ')
+ test_opts = test_opts.push *FileList[test].to_a
+ ruby "-Isrc/main/ruby:src/test/ruby", "-S", "testrb", *test_opts
+ end
+
+ desc "run java tests"
+ task :java => :'test:compile' do
+ mkdir_p TEST_RESULTS_DIR
+ ant.junit :fork => true,
+ :haltonfailure => false,
+ :haltonerror => true,
+ :showoutput => true,
+ :printsummary => true do
+
+ classpath :refid => "main.class.path"
+ classpath :refid => "test.class.path"
+ classpath do
+ pathelement :path => MAIN_BUILD_DIR
+ pathelement :path => TEST_BUILD_DIR
+ end
- formatter :type => "xml"
+ formatter :type => "xml"
- batchtest :fork => "yes", :todir => TEST_RESULTS_DIR do
- fileset :dir => TEST_SRC_DIR do
- include :name => "**/*Test.java"
+ batchtest :fork => "yes", :todir => TEST_RESULTS_DIR do
+ fileset :dir => TEST_SRC_DIR do
+ include :name => "**/*Test.java"
+ end
end
end
end
+
end
+desc "run all tests"
+task :test => [ 'test:java', 'test:ruby' ]
+
desc "clean up"
task :clean do
rm_rf OUT_DIR
View
4 TODO
@@ -1,5 +1,3 @@
1. Should we terminate the Ruby runtime ?
2. Java's Thread.interrupt seems to be not affecting (J)Ruby code
- 3. MORE TESTS (+ Container & Warbler Testing) !
-
-does resque support make sense ?
+ 3. MORE TESTS (+ Container & Warbler Testing) !
View
26 src/main/java/org/kares/jruby/rack/WorkerContextListener.java
@@ -105,7 +105,7 @@ public void contextInitialized(final ServletContextEvent event) {
RackApplicationFactory.class.getName() + " not yet initialized - " +
"seems this listener is executing before the " +
RackServletContextListener.class.getName() + "/RailsSevletContextListener !";
- context.log("[" + WorkerContextListener.class.getName() + "] ERROR: " + message);
+ context.log("[" + WorkerContextListener.class.getName() + "] " + message);
throw new IllegalStateException(message);
}
@@ -114,7 +114,7 @@ public void contextInitialized(final ServletContextEvent event) {
if ( workerScript == null ) {
final String message = "no worker script to execute - configure one using '" + SCRIPT_KEY + "' " +
"or '" + SCRIPT_PATH_KEY + "' context-param or see previous errors if already configured";
- context.log("[" + WorkerContextListener.class.getName() + "] WARN: " + message);
+ context.log("[" + WorkerContextListener.class.getName() + "] " + message + " !");
return; // throw new IllegalStateException(message);
}
@@ -131,11 +131,11 @@ public void contextInitialized(final ServletContextEvent event) {
workerThread.start();
}
catch (RackInitializationException e) {
- context.log("[" + WorkerContextListener.class.getName() + "] ERROR: get rack application failed", e);
+ context.log("[" + WorkerContextListener.class.getName() + "] get rack application failed", e);
break;
}
}
- context.log("[" + WorkerContextListener.class.getName() + "] INFO : started " + workers.size() + " worker(s)");
+ context.log("[" + WorkerContextListener.class.getName() + "] started " + workers.size() + " worker(s)");
}
/**
@@ -155,11 +155,11 @@ public void contextDestroyed(final ServletContextEvent event) {
workerThread.join(1000);
}
catch (InterruptedException e) {
- context.log("[" + WorkerContextListener.class.getName() + "] INFO: interrupted", e);
+ context.log("[" + WorkerContextListener.class.getName() + "] interrupted");
Thread.currentThread().interrupt();
}
catch (Exception e) {
- context.log("[" + WorkerContextListener.class.getName() + "] WARN: ignoring exception", e);
+ context.log("[" + WorkerContextListener.class.getName() + "] ignoring exception " + e);
}
}
/*
@@ -167,9 +167,9 @@ public void contextDestroyed(final ServletContextEvent event) {
catch (InterruptedException e) {
// SEVERE: The web application [/] appears to have started a thread named [worker_1]
// but has failed to stop it. This is very likely to create a memory leak.
- context.log("[" + WorkerContextListener.class.getName() + "] INFO: ignoring interrupt", e);
+ context.log("[" + WorkerContextListener.class.getName() + "] ignoring interrupt " + e);
} */
- context.log("[" + WorkerContextListener.class.getName() + "] INFO: stopped " + workers.size() + " worker(s)");
+ context.log("[" + WorkerContextListener.class.getName() + "] stopped " + workers.size() + " worker(s)");
}
protected RubyWorker newRubyWorker(final Ruby runtime, final String script, final String fileName) {
@@ -186,7 +186,7 @@ protected int getThreadCount(final ServletContext context) {
if ( count != null ) return Integer.parseInt(count);
}
catch (NumberFormatException e) {
- context.log("[" + WorkerContextListener.class.getName() + "] WARN: " +
+ context.log("[" + WorkerContextListener.class.getName() + "] " +
"could not parse " + THREAD_COUNT_KEY + " parameter value = " + count, e);
}
return 1;
@@ -203,7 +203,7 @@ protected int getThreadPriority(final ServletContext context) {
}
}
catch (NumberFormatException e) {
- context.log("[" + WorkerContextListener.class.getName() + "] WARN: " +
+ context.log("[" + WorkerContextListener.class.getName() + "] " +
"could not parse " + THREAD_PRIORITY_KEY + " parameter value = " + priority, e);
}
return Thread.NORM_PRIORITY;
@@ -222,8 +222,8 @@ protected int getThreadPriority(final ServletContext context) {
return new String [] { null, script };
}
else {
- context.log("[" + WorkerContextListener.class.getName() + "] WARN: " +
- "unsupported worker name: '" + worker + "'");
+ context.log("[" + WorkerContextListener.class.getName() + "] " +
+ "unsupported worker name: '" + worker + "' !");
}
}
@@ -274,7 +274,7 @@ protected int getThreadPriority(final ServletContext context) {
put("delayed_job", "delayed/start_worker.rb");
put("delayed", "delayed/start_worker.rb"); // alias
put("navvy", "navvy/start_worker.rb");
- put("resque", "resque/start_worker.rb");
+ //put("resque", "resque/start_worker.rb");
}
};
View
86 src/test/resources/sample.web.xml
@@ -0,0 +1,86 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
+ version="2.4">
+
+ <display-name>JRuby Application Template</display-name>
+ <description>
+ Sample deployment descriptor for a JRuby on Rails application.
+ </description>
+
+ <!-- Rails (threadsafe! due max.runtimes == 1) configuration : -->
+
+ <context-param>
+ <param-name>rails.env</param-name>
+ <param-value>production</param-value>
+ </context-param>
+ <context-param>
+ <param-name>public.root</param-name>
+ <param-value>/</param-value>
+ </context-param>
+
+ <context-param>
+ <param-name>jruby.min.runtimes</param-name>
+ <param-value>1</param-value>
+ </context-param>
+ <context-param>
+ <param-name>jruby.max.runtimes</param-name>
+ <param-value>1</param-value>
+ </context-param>
+
+ <filter>
+ <filter-name>RackFilter</filter-name>
+ <filter-class>org.jruby.rack.RackFilter</filter-class>
+ </filter>
+ <filter-mapping>
+ <filter-name>RackFilter</filter-name>
+ <url-pattern>/*</url-pattern>
+ </filter-mapping>
+
+ <listener>
+ <listener-class>org.jruby.rack.rails.RailsServletContextListener</listener-class>
+ </listener>
+
+ <!-- Worker setup including optional configuration parameters : -->
+
+ <!-- declaratively configure a worker : -->
+ <context-param>
+ <param-name>jruby.worker</param-name>
+ <param-value>delayed_job</param-value>
+ </context-param>
+ <!-- or use an inline worker script : -->
+ <!--
+ <context-param>
+ <param-name>jruby.worker.script</param-name>
+ <param-value>
+ require 'delayed/jruby_worker';
+ Delayed::JRubyWorker.new.start
+ </param-value>
+ </context-param>-->
+ <!-- if you script is located in a (.rb) file use : -->
+ <!--
+ <context-param>
+ <param-name>jruby.worker.script.path</param-name>
+ <param-value>delayed/start_worker.rb</param-value>
+ </context-param>-->
+ <!-- (note that jruby.worker.script and jruby.worker.script.path params are
+ mutually exclusive and jruby.worker.script takes precedence !) -->
+ <!-- if one worker thread is not enough, increase the value (defaults to 1) : -->
+ <context-param>
+ <param-name>jruby.worker.thread.count</param-name>
+ <param-value>1</param-value>
+ </context-param>
+ <!-- you might also change the worker thread's priority (use with caution) : -->
+ <!-- accepted values are MIN, MAX, NORM and values <1..10> (default is NORM) -->
+ <context-param>
+ <param-name>jruby.worker.thread.priority</param-name>
+ <param-value>NORM</param-value><!-- NORM == 5, MIN == 1, MAX == 10 -->
+ </context-param>
+
+ <!-- mandatory - make sure it's declared after the org.jruby.rack... listener : -->
+ <listener>
+ <listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
+ </listener>
+
+</web-app>
View
91 src/test/resources/warbler.web.xml
@@ -1,91 +0,0 @@
-<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
- "http://java.sun.com/dtd/web-app_2_3.dtd">
-<!--
-NOTE: some servers e.g. the awesome http://github.com/calavera/trinidad don't
- play well with the XML doctype (or namespace) declaration e.g. :
-
- Oct 9, 2010 1:20:07 PM org.apache.coyote.ajp.AjpProtocol init
- INFO: Initializing Coyote AJP/1.3 on ajp-3031
- Oct 9, 2010 1:20:07 PM org.apache.coyote.http11.Http11Protocol init
- INFO: Initializing Coyote HTTP/1.1 on http-3000
- Oct 9, 2010 1:20:07 PM org.apache.catalina.core.StandardService startInternal
- INFO: Starting service Tomcat
- Oct 9, 2010 1:20:07 PM org.apache.catalina.core.StandardEngine startInternal
- INFO: Starting Servlet Engine: Apache Tomcat/7.0.2
- Oct 9, 2010 1:20:07 PM org.apache.catalina.startup.ContextConfig webConfig
- INFO: No global web.xml found
- Oct 9, 2010 1:20:09 PM org.apache.catalina.startup.ContextConfig configureStart
- SEVERE: Marking this application unavailable due to previous error(s)
- Oct 9, 2010 1:20:09 PM org.apache.catalina.core.StandardContext startInternal
- SEVERE: Error getConfigured
- Oct 9, 2010 1:20:09 PM org.apache.catalina.core.StandardContext startInternal
- SEVERE: Context [/] startup failed due to previous errors
-
-in that case just remove the above web-app DTD or make sure there's no schema
-declared within Your web-app root XML element !
--->
-<web-app>
-
- <context-param>
- <param-name>rails.env</param-name>
- <param-value>production</param-value>
- </context-param>
-
- <context-param>
- <param-name>public.root</param-name>
- <param-value>/</param-value>
- </context-param>
-
- <context-param>
- <param-name>jruby.min.runtimes</param-name>
- <param-value>1</param-value>
- </context-param>
- <context-param>
- <param-name>jruby.max.runtimes</param-name>
- <param-value>1</param-value>
- </context-param>
-
- <filter>
- <filter-name>RackFilter</filter-name>
- <filter-class>org.jruby.rack.RackFilter</filter-class>
- </filter>
- <filter-mapping>
- <filter-name>RackFilter</filter-name>
- <url-pattern>/*</url-pattern>
- </filter-mapping>
-
- <listener>
- <listener-class>org.jruby.rack.rails.RailsServletContextListener</listener-class>
- </listener>
-
- <!-- worker(s) will execute this script : -->
- <context-param>
- <param-name>jruby.worker.script</param-name>
- <param-value>require 'delayed/jruby_worker'; Delayed::JRubyWorker.new.start</param-value>
- </context-param>
- <!-- if You script is located in a rb file use : -->
- <!-- (please note that jruby.worker.script and jruby.worker.script.path params
- are exclusive and the first one takes precedence over the second one !) -->
- <!--
- <context-param>
- <param-name>jruby.worker.script.path</param-name>
- <param-value>delayed/start_worker.rb</param-value>
- </context-param>-->
- <!-- if one worker thread is not enough, increase the value (defaults to 1) : -->
- <context-param>
- <param-name>jruby.worker.thread.count</param-name>
- <param-value>1</param-value>
- </context-param>
- <!-- You might also change the worker thread priority (use with caution) : -->
- <!-- accepted values are MIN, MAX, NORM and integer values <1..10> (default is NORM) -->
- <context-param>
- <param-name>jruby.worker.thread.priority</param-name>
- <param-value>NORM</param-value>
- </context-param>
-
- <!-- make sure it's declared after the "default" jruby-rack listener : -->
- <listener>
- <listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
- </listener>
-
-</web-app>
Please sign in to comment.
Something went wrong with that request. Please try again.