time-tunnel

An event collection and dispatching µservice.

     

Overview

time-tunnel accepts timestamped events with loosely defined semantics over a set of common APIs, and then dispatches those events on a best-effort basis to various event sinks according to a ruleset that is part of the configuration.

The main goal is to offer clients a way to easily push their events to one uniform event submission API of their choosing, and then go back and concentrate on their actual task. Once those events hit the dispatcher, they can be filtered and duplicated according to the needs of the event sinks.

There's a set of built-in implementations for both delivery APIs and event sinks, and they can be extended with custom ones via a plugin system. time-tunnel is implemented using Python (tested on 2.7 and 3.4) and Flask+WSME.

Portals

Events enter or exit the system via portals, which normally take the form of some API endpoint implementation. They adapt to/from the internal event model, so that (hopefully) any connection between them is possible.

Entry Portals

• 🔜 GitLab Web Hook

Exit Portals

• 🔜 HipChat
• 🔜 WebHooks (e.g. Jenkins Job triggers)

Events

The canonical representation of an event is a flat JSON object with the following fields.

• timestamp – The timestamp of this event in ISO-8601 format.
• kind – An URN specifying the type of this event.
• message – Human readable description.
• source – An URN or URL specifying the exact location of the event source (e.g. a deployed webapp).
• severity – Importance of alarms or log entries.
• view_link – URL of a view in the source system.
• ack_link – URL for acknowledging alarm events.

timestamp, kind and message are obligatory, but timestamp is added server-side when it's missing, and kind defaults to unknown. All other fields are optional, but if the semantics fit, the field name as defined above should be used. Any additional number of fields can be included.

Installation

To create a development environment, use these commands:

git clone "https://github.com/Build-The-Web/time-tunnel.git"
cd time-tunnel
. .env # answer the prompt with (y)es
invoke build --docs

See CONTRIBUTING for details on how to give back your improvements and fixes to upstream, so every user can benefit from them.

Usage

TODO

Related Projects

Event sources + sinks

• HipChat
• InfluxDB
• Sentry
• StatsD
• Fluentd
• feedr
• python-beaver
• TeaFiles
• Graphite Beacon

Visualization / UI

• EventDrops
• Timesketch
• Anthracite

Acknowledgements

 Project sponsored by 1&1.