Add a client object to libhoney for isolating traffic

[maplebed]

This change adds a Client object to libhoney.

When sending traffic of different importance, it can be frustrating to have them run through the same transmission queue. It would be nice to be able to identify some traffic as important enough to block and slow down if sending backs up while allowing other (say logging) traffic to be dropped. With one transmission and a global BlockOnSend that's not possible. This change introduces a libhoney Client with its own queues and transmission. It does not share any state with the global client.

Addresses #11, not by removing the global state, but by giving libhoney users a method of ignoring it.

[tredman]

OK with landing as-is but had a few questions/comments below.

[maplebed]

ok, making Client an interface didn't work at all. I've changed it so that Transmission is now off in its own package, Responses are owned by Transmissions, and if you want mocks, you mock Transmission, not Client.

[ianwilkes]

Concurrency, yo.

[maplebed]

Re-requested reviews. Let's see how we do for round 3

changed sufficiently to invalidate the stamp

[tredman]

let's do this thing!

[tredman]

seems unfortunate to have to duplicate the event struct and copy everything over. Assuming this was to prevent circular imports? Could we eventually refactor event into its own package?

[maplebed]

We could, but not easily. The public interface of the SDK exports libhoney.Event as a thing, so I need to maintain that. We could move Event to its own package and then have libhoney.Event include that type (in the same way there's a transitionalOutput type), but it gets messy with the fieldholder. This diff is big enough; I'd rather keep it as is for now and try and move it later if it remains an issue.

[toshok]

in general I'm not the biggest fan of APIs that permit both zero-initialized allocation forms (&Client) as well as a constructor+config options (NewClient). So many concurrency problems go away if we only support the latter and guarantee that the instance is fully initialized before being returned. Why do we need both again?

[toshok]

why not have another block above the c initialization, and guarantee that NewClient always returns an initialized logger?

if conf.Logger == nil {
  conf.Logger = &nullLogger{} 
}

c := &Client{
  logger: conf.Logger
}

the Transmission stuff below already it this way.

come to think of it, the Transmission checks below should be rewritten into the same style as the things above:

if conf.Transmission == nil {
  conf.Transmission = ...
}

and then can be included in the &Client initialization as with logger. wish Builder's initialization could be as well. that circular .client ref is a nice future refactoring target.

[maplebed]

I’m not clear why that would be better. The way it is not the sync.Once gets triggered during initialization so all future calls quickly noop. Delaying that till the first place they’re hit makes for a more likely collision (and therefore block) doesn’t it?

Can you say more about the motivation for this suggested change?

[toshok]

It just makes initialization more consistent - right now there are 3 different patterns for conf fallback to initialize different fields in the Client struct.

The behavior you're trying to avoid (no ensure* call leading to sync.Once collision later) is also how the c.transmission initialization below acts :/

How about both?

1. change the initialization pattern (for everything but .builder, since it requires the backref) to be consistent:

• Check conf setting for nil, fallback by putting the one we want in conf
• Client init solely out of conf

2. post Client init, call c.transmission.Start() so we can early error, then call both ensureLogger() and ensureTransmission() to use up their sync.Once's before returning the client.

[maplebed]

in general I'm not the biggest fan of APIs that permit both zero-initialized allocation forms (&Client) as well as a constructor+config options (NewClient). So many concurrency problems go away if we only support the latter and guarantee that the instance is fully initialized before being returned. Why do we need both again?

The usual way to explicitly prohibit empty initialization of the struct is to make the Client an interface and the default implementation an unexported struct but doing that in this case caused so many more problems than it solves...

The pattern of "an empty struct will noop; use the constructor for one that works" seems like a reasonable alternative to me. In this case, if you make a &Client{}, you'll get one with a DiscardSender as the transmission and all your instrumentation will go in to a black hole. Which might actually be a feature - for all your tests you don't need to construct a working discard client, you can just use an uninitialized one.

[toshok]

The pattern of "an empty struct will noop; use the constructor for one that works" seems like a reasonable alternative to me

Sure, but it doesn't noop - it doesn't do any outwardly visible work, but it still does stuff, and is the reason for most of the concurrency cognitive load, right? If we never had to worry about lazy initializing an uninitialized &Client{}, we wouldn't need the sync.Once's, we wouldn't need tests for data races, etc.

If the uninitialized path is only meant for a noop client, why not a private initialized field in Client that is only set to true in the constructor. Then we can get rid of (i think?) all the ensure* calls because we can guarantee that a client is either fully initialized or not initialized at all. the nil client checks could early return on it too, if client == nil || !client.initialized

[toshok]

Like I said in slack, feels like I came in a little late to this PR to give these sorts of comments. It's also not going to be an outwardly visible change, so if what's there works, 👍

From a maintenance perspective, though, I can't think of a good argument for continuing with the all the concurrency stuff, though. future PR?

[ianwilkes]

If the uninitialized path is only meant for a noop client, why not a private initialized field in Client that is only set to true in the constructor. Then we can get rid of (i think?) all the ensure* calls because we can guarantee that a client is either fully initialized or not initialized at all. the nil client checks could early return on it too, if client == nil || !client.initialized

This idea is not terrible. I'm sorry this has led to so much back and forth.

[maplebed]

I like the idea of setting an initialized flag in the client and then stripping out the ensures, but I'd like to save that for a future diff. It can be 1.9.1 because everybody loves a point release!