Adopt package-benchmark
#2534
Adopt package-benchmark
#2534
Conversation
FranzBusch
force-pushed
the
fb-benchmark
branch
from
7ad8c8f
to
af255af
Compare
| @@ -0,0 +1,9 @@ | |||
| .DS_Store | |||
There was a problem hiding this comment.
Can we nest Benchmarks in IntegrationTests instead of the root of the repo?
There was a problem hiding this comment.
I intentionally did not nest it since I expect us to migrate all of them at some point and having it at the top level is way nicer. This also follows what we do in other packages.
There was a problem hiding this comment.
I don't think that's a good reason given IntegrationTests contains more than just allocation tests.
There was a problem hiding this comment.
I'm ok with having them at the top level.
FranzBusch
force-pushed
the
fb-benchmark
branch
from
af255af
to
a649ec8
Compare
FranzBusch
force-pushed
the
fb-benchmark
branch
from
a649ec8
to
575d15d
Compare
FranzBusch
force-pushed
the
fb-benchmark
branch
from
575d15d
to
7b73f08
Compare
| scalingFactor: .mega | ||
| ) | ||
| ) { benchmark in | ||
| try runTCPEcho(numberOfWrites: 10_000) |
There was a problem hiding this comment.
Scalingfactor 1M but writing 10K? Perhaps consider numberOfWrites to be scalingfactor if you want to make sense of output from --scale later. Otherwise probably skip setting scalingfactor here, as it's really intended for inner loops in practice.
There was a problem hiding this comment.
So I set the scaling factor because with the default the p90 was to flaky for allocations. With .mega scaling factor they seem really stable. I just pushed a commit that uses the scaledIterations.upperBound for the number of writes
# Motivation We want to migrate our allocation and later on also our performance tests to use the `package-benchmark` plugin. This plugin makes writing benchmarks way easier than our current setup. Furthermore, debugging benchmarks is also possible from within Xcode now. # Modification This PR adds the setup for the benchmarking infrastructure and connects it with out # Result Allocations tests are more accessible and easier to iterate.
FranzBusch
force-pushed
the
fb-benchmark
branch
from
7b73f08
to
7263385
Compare
FranzBusch
added
the
needs-no-version-bump
Motivation
We want to migrate our allocation and later on also our performance tests to use the
package-benchmarkplugin. This plugin makes writing benchmarks way easier than our current setup. Furthermore, debugging benchmarks is also possible from within Xcode now.Modification
This PR adds the setup for the benchmarking infrastructure and connects it with out
Result
Allocations tests are more accessible and easier to iterate.