Refactor Datadog::HTTPTransport

[delner]

The Datadog::HTTPTransport class is quite large and contains a lot of responsibility, which makes it difficult to test, and to extend.

The goal for this pull request is to refactor the HTTPTransport class to accomplish a few things:

• Make the transport simpler: Break it into smaller pieces so those pieces are easier to understand and use.
• Make it easier to test: By virtue of having smaller pieces, it should be easier to test the behavior of discrete parts.
• Make it easier to use in tests: Some tests want to stub requests to the agent completely, others want to mock and inspect the expected HTTP output. By substituting in a test component at the last step of the transport, we can accomplish both of these goals while still running a maximal amount of the typical code paths.
• Make it easier to extend: As we develop improved means of sending data to the agent (e.g. Keep-Alives, connection pooling, alternative HTTP libraries, etc), having a transport layer made of simpler, interchangeable, extendable parts will make implementation and maintenance of such strategies easier.
• Keep it performant: Maintain or improve performance vs the old transport.

What's left to do:

• Add new Transport::HTTP components
• Add a compatibility layer to new transport (so it can be interchanged with the old)
• Add complimentary test components for new transport
• Run performance comparison of old vs new transport
• Update Writer to use new transport
• Update old tests
• Add metrics from Add internal tracing metrics #587 (Won't do; will revisit this later)
• Add unit tests
• Deprecate old transport

[delner]

Ran some basic performance tests, which run the old transport and new transport end-to-end with the agent:

TL;DR New and old transport have comparable performance.

Old transport:

it 'HTTPTransport' do
  transport = Datadog::HTTPTransport.new(
    ENV.fetch('TEST_DDAGENT_HOST', 'localhost'),
    ENV.fetch('TEST_DDAGENT_PORT', 8126)
  )

  Benchmark.bm do |x|
    x.report('HTTPTransport') do
      1000.times do
        traces = get_test_traces(2)
        transport.send(:traces, traces)
      end
    end
  end
end

Randomized with seed 33851
       user     system      total        real
HTTPTransport  0.710000   0.240000   0.950000 (  1.051479)
.

Finished in 1.05 seconds (files took 0.55395 seconds to load)
1 example, 0 failures

New transport:

it 'Transport::HTTP' do
  client = Datadog::Transport::HTTP::Client.new(
    Datadog::Transport::HTTP::Service.new(
      ENV.fetch('TEST_DDAGENT_HOST', 'localhost'),
      ENV.fetch('TEST_DDAGENT_PORT', 8126)
    )
  )

  Benchmark.bm do |x|
    x.report('Transport::HTTP') do
      1000.times do
        traces = get_test_traces(2)
        parcel = Datadog::Transport::Traces::Parcel.new(traces)
        client.deliver(parcel)
      end
    end
  end
end

Randomized with seed 12812
       user     system      total        real
Transport::HTTP  0.620000   0.260000   0.880000 (  0.979254)
.

Finished in 0.98074 seconds (files took 0.55062 seconds to load)
1 example, 0 failures

[pawelchcki]

The new design looks quite clean! Great job @delner

[delner]

#738 removed services from the trace library. This PR should be rebased on those changes, and the corresponding service components removed.

[delner]

Rebased this PR on 0.23-dev and removed services from it. Seems to be working okay.

[brettlangdon]

lgtm

[pawelchcki]

One small question about using an uninstrumented copy of Net::HTTP for transport, otherwise looks good.
Love the new code coverage