Add initial benchmark suite #390
Conversation
mayurkale22
requested review from
bg451,
danielkhan,
iredelmeier,
OlivierAlbertini,
rochdev and
vmarchaud
as code owners
| @@ -0,0 +1,25 @@ | |||
| 'use strict'; | |||
There was a problem hiding this comment.
Would it be possible to use TypeScript for this and execute it with ts-node? Right now it's pretty simple but I expect that over time the benchmarks themselves may become more complex and could benefit from type checking, linting, etc.
Not sure how best to do that - could we possibly but the benchmarks embedded in the packages they test e.g. with a "/benchmark" subfolder like we have a "/test" subfolder? That also organizes benchmarks with each package in case some only apply e.g. to a particular exporter, etc.
There was a problem hiding this comment.
should we do it to examples as well ?
There was a problem hiding this comment.
Interesting question ... I actually think that giving users the ability to get to examples quickly feels valuable and so keeping it a top-level GitHub folder probably helps with that.
In contrast, a typical new user might not care about the details of benchmarks, though maybe if we advertised performance in some high performance level graphs in the README that could be cool. I'd mainly be interested in an instrumented vs. un-instrumented app performance there.
I can imagine us eventually wanting a top level "/tests" and "/benchmarks" folders as well that could test or benchmark a whole application that uses lots of different packages together, but for micro-benchmarks, I wonder if putting them with the package makes sense.
There was a problem hiding this comment.
+1 to instrumented vs non-instrumented. I think either benchmarking each plugin, or just providing a package which lets you easily DIY as you've suggested would be awesome.
Codecov Report
@@ Coverage Diff @@
## master #390 +/- ##
==========================================
+ Coverage 95.39% 95.41% +0.02%
==========================================
Files 124 124
Lines 6142 6171 +29
Branches 510 509 -1
==========================================
+ Hits 5859 5888 +29
Misses 283 283
|
| @@ -0,0 +1,25 @@ | |||
| 'use strict'; | |||
There was a problem hiding this comment.
+1 to instrumented vs non-instrumented. I think either benchmarking each plugin, or just providing a package which lets you easily DIY as you've suggested would be awesome.
mayurkale22
force-pushed
the
benchmark
branch
from
7042c5c
to
8d30b35
Compare
* Use more precise tests than isOk Signed-off-by: Yuri Shkuro <ys@uber.com> * Remove getTags introduced earlier Signed-off-by: Yuri Shkuro <ys@uber.com>
Which problem is this PR solving?
Resolves Add baseline benchmark suite #261
This is my initial take on benchmarking suite. I'd love to get your feedback to take this in the right direction.
Currently the benchmarks run some very basic operations with the
NoopTracer,BasicTracerandNodeTracerimplementation onstartSpanmethod with and w/o parent.Update1: Added more benchmarks (Tracers and Propagators).
Key observation:
HttpTraceContextinject is better than theB3Format, but the reverse is true for extract operation.