Skip to content

Commit

Permalink
Introducing: ActiveJob, and support for other queue adapters (#18)
Browse files Browse the repository at this point in the history
* Add new ActiveJobs to replace DJs
* Fix test error (wasn't resolving constant for me)
* Enqueue ActiveJob instead of DJ (+spec coverage)
* Remove 'enqueue' abstraction since it doesn't really map to AJs
* Enable appraisal when running top-level rake
* Rework README to be more adapter-agnostic (as long as it's DB-backed, i.e. supports cotransactionality!)
* Raise an exception in production unless the queue adapter is supported
* Decouple entirely from delayed_job dependency
* No need for binding.pry when we have binding.irb
* Remove unused file
* Switch to sqlite3 in-memory DB for testing
* Trim down deps - railties, activejob, activerecord
* Increment version to 3.0.0
* Regenerate coach config (no more postgres DB)
* Remove unused sprockets test dep
* Mention priority support being only Rails 6+
* Apply README suggestion
Co-authored-by: Sam Moore <sam.moore@hey.com>
* Push queue adapter safety check out until after initialization
Co-authored-by: Sam Moore <sam.moore@hey.com>
  • Loading branch information
smudge committed May 18, 2021
1 parent 855d9ca commit 226fd64
Show file tree
Hide file tree
Showing 34 changed files with 529 additions and 306 deletions.
85 changes: 39 additions & 46 deletions .circleci/config.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 7 additions & 3 deletions Appraisals
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
appraise 'rails-5-1' do
gem 'rails', '~> 5.1.0'
gem 'railties', '~> 5.1.0'
end

appraise 'rails-5-2' do
gem 'rails', '~> 5.2.0'
gem 'railties', '~> 5.2.0'
end

appraise 'rails-6-0' do
gem 'rails', '~> 6.0.0'
gem 'railties', '~> 6.0.0'
end

appraise 'rails-6-1' do
gem 'railties', '~> 6.1.0'
end
29 changes: 21 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Journaled

A Rails engine to durably deliver schematized events to Amazon Kinesis via DelayedJob.
A Rails engine to durably deliver schematized events to Amazon Kinesis via ActiveJob.

More specifically, `journaled` is composed of three opinionated pieces:
schema definition/validation via JSON Schema, transactional enqueueing
via Delayed::Job (specifically `delayed_job_active_record`), and event
via ActiveJob (specifically, via a DB-backed queue adapter), and event
transmission via Amazon Kinesis. Our current use-cases include
transmitting audit events for durable storage in S3 and/or analytical
querying in Amazon Redshift.

Journaled provides an at-least-once event delivery guarantee assuming
Delayed::Job is configured not to delete jobs on failure.
ActiveJob's queue adapter is not configured to delete jobs on failure.

Note: Do not use the journaled gem to build an event sourcing solution
as it does not guarantee total ordering of events. It's possible we'll
Expand All @@ -20,9 +20,20 @@ durable, eventually consistent record that discrete events happened.

## Installation

1. [Install `delayed_job_active_record`](https://github.com/collectiveidea/delayed_job_active_record#installation)
if you haven't already.
1. If you haven't already,
[configure ActiveJob](https://guides.rubyonrails.org/active_job_basics.html)
to use one of the following queue adapters:

- `:delayed_job` (via `delayed_job_active_record`)
- `:que`
- `:good_job`
- `:delayed`

Ensure that your queue adapter is not configured to delete jobs on failure.

**If you launch your application in production mode and the gem detects that
`ActiveJob::Base.queue_adapter` is not in the above list, it will raise an exception
and prevent your application from performing unsafe journaling.**

2. To integrate Journaled into your application, simply include the gem in your
app's Gemfile.
Expand Down Expand Up @@ -85,9 +96,11 @@ Journaling provides a number of different configuation options that can be set i

#### `Journaled.job_priority` (default: 20)

This can be used to configure what `priority` the Delayed Jobs are enqueued with. This will be applied to all the Journaled::Devivery jobs that are created by this application.
This can be used to configure what `priority` the ActiveJobs are enqueued with. This will be applied to all the `Journaled::DeliveryJob`s that are created by this application.
Ex: `Journaled.job_priority = 14`

_Note that job priority is only supported on Rails 6.0+. Prior Rails versions will ignore this parameter and enqueue jobs with the underlying ActiveJob adapter's default priority._

#### `Journaled.http_idle_timeout` (default: 1 second)

The number of seconds a persistent connection is allowed to sit idle before it should no longer be used.
Expand All @@ -100,9 +113,9 @@ Journaling provides a number of different configuation options that can be set i

The number of seconds before the :http_handler should timeout while waiting for a HTTP response.

#### DJ `enqueue` options
#### ActiveJob `set` options

Both model-level directives accept additional options to be passed into DelayedJob's `enqueue` method:
Both model-level directives accept additional options to be passed into ActiveJob's `set` method:

```ruby
# For change journaling:
Expand Down
8 changes: 7 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,13 @@ if %w(development test).include? Rails.env
RuboCop::RakeTask.new

task(:default).clear
task default: %i(rubocop spec)
if ENV['APPRAISAL_INITIALIZED'] || ENV['CI']
task default: %i(rubocop spec)
else
require 'appraisal'
Appraisal::Task.new
task default: :appraisal
end

task 'db:test:prepare' => 'db:setup'
end
4 changes: 4 additions & 0 deletions app/jobs/journaled/application_job.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
module Journaled
class ApplicationJob < Journaled.job_base_class_name.constantize
end
end
Loading

0 comments on commit 226fd64

Please sign in to comment.