Skip to content
This repository has been archived by the owner on Nov 9, 2017. It is now read-only.

Commit

Permalink
Decouple the hell out of things.
Browse files Browse the repository at this point in the history
  • Loading branch information
@codahale
codahale committed Apr 30, 2014
1 parent 2b1c38b commit 0322314
Show file tree
Hide file tree
Showing 7 changed files with 163 additions and 245 deletions.
56 changes: 28 additions & 28 deletions doc.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,24 @@
// Consider a user-initiated web request. Their browser sends an HTTP request to
// an edge server, which extracts the credentials (e.g., OAuth token) and
// authenticates the request by communicating with an internal authentication
// service, which returns a signed set of internal credentials (e.g., signed user
// ID). The edge web server then proxies the request to a cluster of web servers,
// each running a PHP application. The PHP application loads some data from
// several databases, places the user in a number of treatment groups for running
// A/B experiments, writes some data to a Dynamo-style distributed database, and
// returns an HTML response. The edge server receives this response and proxies
// it to the user's browser.
// service, which returns a signed set of internal credentials (e.g., signed
// user ID). The edge web server then proxies the request to a cluster of web
// servers, each running a PHP application. The PHP application loads some data
// from several databases, places the user in a number of treatment groups for
// running A/B experiments, writes some data to a Dynamo-style distributed
// database, and returns an HTML response. The edge server receives this
// response and proxies it to the user's browser.
//
// In this scenario we have a number of infrastructure-specific events:
//
// 1. The edge server handled a request, which took 142ms and whose response
// had a status of "200 OK".
// 1. The edge server handled a request, which took 142ms and whose
// response had a status of "200 OK".
// 2. The edge server sent a request to the authentication service, which
// took 5ms to handle and identified the principal as user 14002.
// 3. The authentication service handled a request, which took 4ms to handle
// and was served entirely from memory.
// 4. The edge server proxied a request to the app cluster, which took 132ms
// and whose response had a status of "200 OK".
// 3. The authentication service handled a request, which took 4ms to
// handle and was served entirely from memory.
// 4. The edge server proxied a request to the app cluster, which took
// 132ms and whose response had a status of "200 OK".
// 5. The app load balancer handled a request, which took 131ms and whose
// response had a status of "200 OK".
// 6. The app load balancer proxied a request to the app, which took 130ms
Expand All @@ -47,15 +47,15 @@
// the infrastructure, but are still critical information for the business the
// system supports:
//
// 14. The app gave the user the control treatment for experiment 15 ("Really
// Big Buttons v2").
// 14. The app gave the user the control treatment for experiment 15
// ("Really Big Buttons v2").
// 15. The app gave the user the experimental treatment for experiment 54
// ("More Yelling v1").
// 16. User 14002 viewed photo 1819 ("rude-puppy.gif").
//
// There are a number of different teams all trying to monitor and improve
// aspects of this system. Operational staff need to know if a particular host or
// service is experiencing a latency spike or drop in throughput. Development
// aspects of this system. Operational staff need to know if a particular host
// or service is experiencing a latency spike or drop in throughput. Development
// staff need to know if their application's response times have gone down as a
// result of a recent deploy. Customer support staff need to know if the system
// is operating nominally as a whole, and for customers in particular. Product
Expand Down Expand Up @@ -99,14 +99,14 @@
// latency is over 300ms, mostly as a result of requests like those in tree
// XXXXX).
//
// For offline causational analysis, events can be written in batches to
// batch processing systems like Hadoop or OLAP databases like Vertica. These
// aggregates can be queried to answer questions traditionally reserved for
// A/B testing systems. "Did users who were show the new navbar view more
// photos?" "Did the new image optimization algorithm we enabled for 1% of views
// run faster? Did it produce smaller images? Did it have any effect on user
// engagement?" "Did any services have increased exception rates after any recent
// deploys?" &tc &tc
// For offline causational analysis, events can be written in batches to batch
// processing systems like Hadoop or OLAP databases like Vertica. These
// aggregates can be queried to answer questions traditionally reserved for A/B
// testing systems. "Did users who were show the new navbar view more photos?"
// "Did the new image optimization algorithm we enabled for 1% of views run
// faster? Did it produce smaller images? Did it have any effect on user
// engagement?" "Did any services have increased exception rates after any
// recent deploys?" &tc &tc
//
// Observing Specific Events
//
Expand All @@ -124,7 +124,7 @@
// Accept: application/json
// Event-ID: d6cb1d852bbf32b6/6eeee64a8ef56225
//
// The header value is simply the root ID and event ID, hex-encoded in
// little-endian form and concatenated with a slash. A server that receives a
// request with this header can use this to properly parent its own events.
// The header value is simply the root ID and event ID, hex-encoded and
// concatenated with a slash. A server that receives a request with this header
// can use this to properly parent its own events.
package lunk
12 changes: 9 additions & 3 deletions doc_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,15 @@ import "os"

func Example() {
l := NewJSONEventLogger(os.Stdout)
root := l.LogRoot(Message("root action"))
sub := l.Log(root, Message("sub action"))
l.Log(sub, Message("leaf action"))

rootID := NewRootEventID()
l.Log(rootID, Message("root action"))

subID := NewEventID(rootID)
l.Log(subID, Message("sub action"))

leafID := NewEventID(rootID)
l.Log(leafID, Message("leaf action"))

// Produces something like this:
// {
Expand Down
107 changes: 50 additions & 57 deletions event.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package lunk
import (
"errors"
"fmt"
"net/url"
"os"
"strings"
"time"
)

Expand All @@ -19,40 +19,66 @@ var (
ErrBadEventID = errors.New("bad event ID")
)

// EventID is the ID of an event and its root event.
// EventID is the ID of an event, its parent event, and its root event.
type EventID struct {
// Root is the root ID of the tree which contains all of the events related
// to this one.
Root ID `json:"root"`

// ID is an ID uniquely identifying the event.
ID ID `json:"id"`

// Parent is the ID of the parent event, if any.
Parent ID `json:"parent,omitempty"`
}

// String returns the EventID as a URL-encoded set of parameters.
func (id EventID) String() string {
return fmt.Sprintf("%s/%s", id.Root, id.ID)
return fmt.Sprintf("root=%s&id=%s", id.Root, id.ID)
}

// Format formats according to a format specifier and returns the resulting
// string. The receiver's root and ID are the first and second arguments.
// string. The receiver's string representation is the first argument.
func (id EventID) Format(s string, args ...interface{}) string {
return fmt.Sprintf(s, append([]interface{}{id.Root, id.ID}, args...)...)
args = append([]interface{}{id.String()}, args...)
return fmt.Sprintf(s, args...)
}

// ParseEventID parses the given string as two ID strings separated by a slash,
// or returns an error.
// NewRootEventID generates a new event ID for a root event. This should only be
// used to generate entries for events caused exclusively by events which are
// outside of your system as a whole (e.g., a root event for the first time you
// see a user request).
func NewRootEventID() EventID {
id := NewID()
return EventID{
Root: id,
ID: id,
}
}

// NewEventID returns a new ID for an event which is the child of the given
// parent ID. This should be used to track causal relationships between events.
func NewEventID(parent EventID) EventID {
return EventID{
Root: parent.Root,
ID: NewID(),
Parent: parent.ID,
}
}

// ParseEventID parses the given string as a URL-encoded set of parameters.
func ParseEventID(s string) (*EventID, error) {
parts := strings.Split(s, "/")
if len(parts) != 2 {
u, err := url.ParseQuery(s)
if err != nil {
return nil, ErrBadEventID
}

root, err := ParseID(parts[0])
root, err := ParseID(u.Get("root"))
if err != nil {
return nil, ErrBadEventID
}

id, err := ParseID(parts[1])
id, err := ParseID(u.Get("id"))
if err != nil {
return nil, ErrBadEventID
}
Expand All @@ -65,14 +91,9 @@ func ParseEventID(s string) (*EventID, error) {

// Metadata is a collection of metadata about an Event.
type Metadata struct {
EventID

// Schema is the schema of the event.
Schema string `json:"schema"`

// Parent is the ID of the parent event, if any.
Parent ID `json:"parent,omitempty"`

// Time is the timestamp of the event.
Time time.Time `json:"time"`

Expand All @@ -87,54 +108,26 @@ type Metadata struct {
PID int `json:"pid"`
}

// NewMetdata returns a populated Metadata instance for the given event.
func NewMetadata(e Event) Metadata {
return Metadata{
Schema: e.Schema(),
Time: time.Now(),
Host: host,
Deploy: deploy,
PID: pid,
}
}

// An Entry is the combination of an event and its metadata.
type Entry struct {
EventID
Metadata

// Event is the actual event object, to be serialized as a object.
// Event is the actual event object, to be serialized to JSON.
Event Event `json:"event"`
}

// NewRootEntry creates a new Entry instance for the root event in the given
// tree. This should only be used to generate entries for events caused
// exclusively by events which are outside of your system as a whole (e.g., a
// root entry for the first time you see a user request).
func NewRootEntry(e Event) *Entry {
id := NewID()
return &Entry{
Metadata: Metadata{
EventID: EventID{
Root: id,
ID: id,
},
Schema: e.Schema(),
Time: time.Now(),
Host: host,
Deploy: deploy,
PID: pid,
},
Event: e}
}

// NewEntry creates a new Entry instance for the given event in the given tree
// with the given parent.
func NewEntry(parent EventID, e Event) *Entry {
return &Entry{
Metadata: Metadata{
EventID: EventID{
Root: parent.Root,
ID: NewID(),
},
Schema: e.Schema(),
Parent: parent.ID,
Time: time.Now(),
Host: host,
Deploy: deploy,
PID: pid,
},
Event: e}
}

var (
host, deploy string
pid int
Expand Down
Loading

0 comments on commit 0322314

Please sign in to comment.