Add detailed CPU benchmark #1188
Conversation
| after { tracer.shutdown! } | ||
|
|
||
| let(:writer) { Datadog::Writer.new(buffer_size: 1000, flush_interval: 0) } |
There was a problem hiding this comment.
This ensures that spans are actually being consumed quickly by the writer, instead of being mostly dropped by the buffer due to the 1000 span default limit.
Our benchmarks aim to simulate a realistic user scenario, but as quickly as possible. Having the worker flush spans more frequently helps us accomplish that goal.
|
|
||
| # Warm up | ||
| def warm_up |
There was a problem hiding this comment.
Warm up was previously being done on a before RSpec block, which could run at any point of the setup phase, which can cause issues when benchmark setups are required to run before the warming up starts.
We now moved warm_up to inside the test run block, which ensures that it runs after all setup is done.
| # Read HTTP request to allow other side to have enough | ||
| # buffer write room. If we don't, the client won't be | ||
| # able to send the full request until the buffer is cleared. | ||
| conn.read(1 << 31 - 1) |
There was a problem hiding this comment.
This is happening because, in the end-to-end test, we are trying to send very large payloads, due to the buffer constantly having close to 1000 items.
The network write buffer was getting full on the worker side, which would just block and not continue flushing until it timed out.
There was a problem hiding this comment.
Lgtm. Maybe we want to include a line or small section development docs about using it, so 3rd party contribs know to use tools like KCachegrind or QCachegrind to view the results, or generally any things we want a contributor to pay attention to in the results?
|
@ericmustin I added a section to our developer guide about benchmarks. Benchmark specific instructions, like how to process results in this PR, are printed in each benchmark run as they are potentially different for each benchmark result. It would be hard to keep our markdown file and benchmark instructions in sync, while printing them with the benchmarks makes that part easier. |
* Add CPU benchmark * Remove incompatible ruby-prof for JRuby runs * Remove incompatible 'ruby-prof' for Ruby < 2.4 runs * Add benchmarks to developer guide
This PR adds CPU profiling using ruby-prof .
All our existing benchmarks only provide deep analysis for memory behavior. This PR introduces a detailed analysis tool to measure application timing.
By default, the results are output in Cachegrind format, and can be analyzed with tools like KCachegrind or QCachegrind.
Here's an example:
When the benchmark runs, instructions are printed on how to get to this colorful screen I posted above.