Hedwig Library for Go

Hedwig is a inter-service communication bus that works on AWS SQS/SNS, while keeping things pretty simple and straight forward. It uses JSON schema draft v4 for schema validation so all incoming and outgoing messages are validated against pre-defined schema.

Hedwig allows separation of concerns between consumers and publishers so your services are loosely coupled, and the contract is enforced by the schema validation. Hedwig may also be used to build asynchronous APIs.

For intra-service messaging, see Taskhawk.

Fan Out

Hedwig utilizes SNS for fan-out configuration. A publisher publishes messages on a topic. This message may be received by zero or more consumers. The publisher need not be aware of the consuming application. There are a variety of messages that may be published as such, but they generally fall into two buckets:

• Asynchronous API Requests: Hedwig may be used to call APIs asynchronously. The contract is enforced by your infra-structure by connecting SNS topics to SQS queues, and payload is validated using the schema you define. Response is a delivered using a separate message if required.
• Notifications: The most common use case is to notify other services/apps that may be interested in events. For example, your User Management app can publish a user.created message notification to all your apps. As publishers and consumers are loosely coupled, this separation of concerns is very effective in ensuring a stable eco-system.

Provisioning

Hedwig works on SQS and SNS as backing queues. Before you can publish/consume messages, you need to provision the required infra. This may be done manually, or, preferably, using Terraform. Hedwig provides tools to make infra configuration easier: see Terraform and Hedwig Terraform Generator for further details.

Quick Start

First, install the library:

go get github.com/Automatic/hedwig-go

Create a JSON-schema and save as schema.json:

    {
        "id": "https://hedwig.automatic.com/schema#",
        "$schema": "http://json-schema.org/draft-04/schema",
        "schemas": {
            "email.send": {
                "1.*": {
                    "description": "Request to send email",
                    "type": "object",
                    "required": [
                        "to",
                        "subject"
                    ],
                    "properties": {
                        "to": {
                            "type": "string",
                            "pattern": "^\\S+@\\S+$"
                        },
                        "subject": {
                            "type": "string",
                            "minLength": 2
                        }
                    }
                }
            }
        }
    }

Next, set up a few configuration settings:

    validator, err := hedwig.NewMessageValidator("schema.json")
    if err != nil {
        panic("Failed to create validator")
    }
    settings := &hedwig.Settings{
        AWSAccessKey:              <YOUR AWS KEY>,
        AWSAccountID:              <YOUR AWS ACCOUNT ID>,
        AWSRegion:                 <YOUR AWS REGION>,
        AWSSecretKey:              <YOUR AWS SECRET KEY>,
        AWSSessionToken:           <YOUR AWS SESSION TOKEN>,
        CallbackRegistry:          hedwig.NewCallbackRegistry(),
        Publisher:                 "MYAPP",
        QueueName:                 "DEV-MYAPP",
        MessageRouting:            map[hedwig.MessageRouteKey]string{
            hedwig.MessageRouteKey{
                MessageType:    "email.send",
    		        MessageMajorVersion: 1,
    	      }: "send_email",
        },
        Validator:                 validator,
    }

These configuration settings will be passed into the library for initialization.

Next define the models associated with the schemas. These models should have factory functions as well.

    type SendEmail struct {
        Subject string `json:"subject"`
        To      string `json:"to"`
    }

    // Factory that returns empty struct
    func NewSendEmailData() interface{} { return new(SendEmail) }

Then, simply define your topic handler and register the handler:

    // Handler
    func HandleSendEmail(ctx context.Context, msg *hedwig.Message) error {
        // Send email
    }

    // Register handler
    cbk := CallbackKey{
        MessageType:    "email.send",
        MessageMajorVersion: 1,
    }
    settings.CallbackRegistry.RegisterCallback(cbk, HandleSendEmail, NewSendEmailData)

Initialize the publisher

    sessionCache := hedwig.NewAWSSessionsCache()
    publisher := hedwig.NewPublisher(sessionCache, settings)

And finally, send a message:

    headers := map[string]string{}
    msg, err := hedwig.NewMessage(settings, "email.send", "1.0", headers, data)
    if err != nil {
        return err
    }
    publisher.Publish(context.Background(), msg)

Development

Prerequisites

Install govendor

Getting Started

$ cd ${GOPATH}/src/github.com/Automatic/hedwig-go
$ govendor sync

Running Tests

$ make test

Getting Help

We use GitHub issues for tracking bugs and feature requests.

• If it turns out that you may have found a bug, please open an issue

Release notes

Current version: v1.0.1-dev

v1.0.0

• Initial version