Maitredee

An opinionated pub/sub framework.

Table of Contents

• Overview
• Installation
• Configuration
• Publisher
• Subscriber
• Listening to messages
• Validation schema
• Misc
  • Development
  • Contributing
  • License
  • Code of Conduct

Overview

We made maitredee to simplify publishing and subscribing to events for our junior developers. We tried using kafka but ordered eventing was too complicated.

We tried to have zero setup required to get this up and running and make it work as simply as sidekiq.

We hope in the future to add more adapters beyond sns/sqs.

For API docks visit https://www.rubydoc.info/gems/maitredee

Installation

Add this line to your application's Gemfile:

gem 'maitredee'

And then execute:

$ bundle

Or install it yourself as:

$ gem install maitredee

Configuration

Required Configuration

Maitredee.namespace = "plated-production"
Maitredee.schema_path = Rails.root.join("your/path").to_s
Maitredee.client = :sns_sqs

These namespace can also be set with the environment variable MAITREDEE_NAMESPACE

Available clients

Maitredee currently supports the following clients:

SNS/SQS :sns_sqs

You can set the AWS parameters in a variety of ways. Either environment variables or explicitly. Supported environment variables are MAITREDEE_AWS_ACCESS_KEY_ID, MAITREDEE_AWS_SECRET_ACCESS_KEY, MAITREDEE_AWS_REGION and then falls back to default AWS keys.

if you wish to set it explicitly:

Maitredee.set_client(
  :sns_sqs,
  access_key_id: "",
  secret_access_key: "",
  region: ""
)

Test :test

This is used for testing.

Maitredee.client = :test

When you publish anything through Maitredee it will be logged in the test client for test verification.

You should reset the client at the beginning of every test with Maitredee.client.reset

Publisher

Create a publisher class for your topic and inherit from Maitredee::Publisher Optionally define the default topic, event_name, or validation schema with publish_defaults Define an #initialize and #process. #process will get called when you execute the publisher. Call #publish from #process to publish messages. #publish will default the parameters topic, event_name, and schema_name from your .publish_defaults if not given.

require "maitredee"

class RecipePublisher < Maitredee::Publisher
  publish_defaults(
    topic_name: :default_topic,
    event_name: :optional_default_event_name,
    schema_name: :default_schema
  )

  attr_reader :recipe

  def initialize(recipe)
    @recipe = recipe
  end

  def process
    publish(
      topic_name: :my_topic_override,
      event_name: :event_name_is_optional,
      schema_name: :schema_name,
      primary_key: "optionalKey",
      body: {
        id: recipe.id,
        name: recipe.name
      }
    )
  end
end

Publishing a message

To publish a message, simply call .call on your publisher:

RecipePublisher.call(model)

By default .call delegate the arguments to .new then call #process and return #published_messages which is an array of published messages.

#publish will first validate your schema before publishing the message.

If you have ActiveJob configured you can also #call_later and it will be called asyncronously

Subscriber

class RecipeSubscriber < Maitredee::Subscriber
  # this is the topic name
  subscribe_to :recipes do

    # this is the event name optionally say which method to use to process
    event(:create, to: create)

    # event_name will be used as the method name if it is a valid method name, otherwise to: must be set
    event(:delete)

    # for empty event name just use nil
    event(nil, to: :process)

    # you can specify a catch all route
    default_event to: :process
  end

  # optional initializer to do message pre processing
  # def initialize(message)
  #   super
  #   # do business here
  # end

  def create
    Recipe.create!(message.body)
  end

  def process
    Recipe.find(message.body[:id]).update(message.body)
  end

  def delete
    Recipe.find(message.body[:id]).destroy
  end
end

Listening to messages

We use shoryuken as our worker backend. Maitredee supports all shoryuken options except queues which is replace with subscribers.

https://github.com/phstc/shoryuken/wiki/Shoryuken-options

subscribers:
  - RecipeSubscriber

maitredee -s RecipeSubscriber 

Validating Schemas

Maitredee validates your message body schemas using JSON schema ([JSON-schemer] (https://github.com/davishmcclurg/json_schemer)) for both publishing and consuming messages. [Configure] (#configuration) the location of your schemas and provide a JSON file for each of your schemas.

Example recipe_v1.json:

{
  "type": "object",
  "required": ["id", "name", "servings"],
  "properties": {
    "id": {
      "type": "string"
    },
    "name": {
      "type": "string"
    },
    "servings": {
      "type": "number"
    }
  }
}

Misc

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/plated/maitredee. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Maitredee project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.