Update benchmarks.jl

[mforets]

update benchmarks to use BenchmarkGroup() syntax

[coveralls]

Coverage increased (+0.05%) to 92.293% when pulling e7080d6 on mforets:patch-1 into 777ad9d on JuliaIntervals:master.

[codecov-io]

Codecov Report

Merging #156 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #156      +/-   ##
==========================================
+ Coverage   92.24%   92.29%   +0.04%     
==========================================
  Files          21       23       +2     
  Lines        1006     1064      +58     
==========================================
+ Hits          928      982      +54     
- Misses         78       82       +4

Impacted Files	Coverage Δ
src/intervals/intervals.jl	81.39% <0%> (-4.32%)	⬇️
src/multidim/intervalbox.jl	85.71% <0%> (-0.96%)	⬇️
src/intervals/set_operations.jl	100% <0%> (ø)	⬆️
src/multidim/setdiff.jl	100% <0%> (ø)	⬆️
src/intervals/macros.jl	100% <0%> (ø)	⬆️
src/IntervalArithmetic.jl	100% <0%> (ø)	⬆️
src/parsing.jl	96.96% <0%> (ø)	⬆️
src/multidim/arithmetic.jl	80% <0%> (ø)
src/bisect.jl	100% <0%> (ø)
src/intervals/arithmetic.jl	97.47% <0%> (+1.1%)	⬆️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 777ad9d...e7080d6. Read the comment docs.

[dpsanders]

Thanks for the contribution!
What advantage does the new version have?

[mforets]

You're welcome. I only made the script work again, since the macros @benchgroup and @bench do not longer exist upstream (changed to BenchmarkGroup() and @benchmarkable respectively):

julia> using BenchmarkTools, PkgBenchmark

julia> @bench
ERROR: UndefVarError: @bench not defined

julia> @benchgroup
ERROR: UndefVarError: @benchgroup not defined

julia> Pkg.installed("BenchmarkTools"), Pkg.installed("PkgBenchmark")
(v"0.3.1", v"0.1.1")

julia> @benchmarkable
ERROR: ArgumentError: tuple must be non-empty

julia> BenchmarkGroup
BenchmarkTools.BenchmarkGroup

[dpsanders]

Great, thanks.

I'm still getting the following error, though:

ERROR: LoadError: UndefVarError: @benchgroup not defined
Stacktrace:
 [1] include_from_node1(::String) at ./loading.jl:576
 [2] include(::String) at ./sysimg.jl:14
 [3] _runbenchmark_local(::String, ::String, ::String, ::Bool) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/runbenchmark.jl:152
while loading /Users/dpsanders/.julia/v0.6/IntervalArithmetic/benchmark/benchmarks.jl, in expression starting on line 16
ERROR: failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia --depwarn=no --code-coverage=none --color=yes --compilecache=yes -e 'using PkgBenchmark
PkgBenchmark._runbenchmark_local("/Users/dpsanders/.julia/v0.6/IntervalArithmetic/benchmark/benchmarks.jl", "/var/folders/h0/68t1xc991mq20mrkn6yy6n5h0000gn/T/juliaLvyBMJ", "/Users/dpsanders/.julia/v0.6/.pkgbenchmark/IntervalArithmetic_tune.json", false )
'`, ProcessExited(1)) [1]
Stacktrace:
 [1] uv_error at ./libuv.jl:68 [inlined]
 [2] unlink(::String) at ./file.jl:500
 [3] #rm#7(::Bool, ::Bool, ::Function, ::String) at ./file.jl:145
 [4] (::Base.Filesystem.#kw##rm)(::Array{Any,1}, ::Base.Filesystem.#rm, ::String) at ./<missing>:0
 [5] _withtemp(::PkgBenchmark.##28#32{Bool,String}, ::String) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/util.jl:25
 [6] _with_reqs(::PkgBenchmark.##27#31{Bool,String}, ::String, ::Function) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/util.jl:7
 [7] (::PkgBenchmark.#do_benchmark#30{Bool,String,String})() at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/runbenchmark.jl:83
 [8] (::PkgBenchmark.##58#59{PkgBenchmark.#do_benchmark#30{Bool,String,String},String,String})(::Base.LibGit2.GitRepo) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/util.jl:38
 [9] transact(::PkgBenchmark.##58#59{PkgBenchmark.#do_benchmark#30{Bool,String,String},String,String}, ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:882
 [10] #benchmarkpkg#26(::Void, ::Void, ::Bool, ::String, ::Function, ::String, ::PkgBenchmark.BenchmarkConfig) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/runbenchmark.jl:105
 [11] #judge#49(::Function, ::Dict{Any,Any}, ::Array{Any,1}, ::Function, ::String, ::PkgBenchmark.BenchmarkConfig, ::String) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/judge.jl:30
 [12] #judge#50(::Array{Any,1}, ::Function, ::String, ::String) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/judge.jl:36
 [13] judge(::String, ::String) at /Users/dpsanders/.julia/v0.6/PkgBenchmark/src/judge.jl:36

[mforets]

Huh, and you have at least these versions?

julia> Pkg.installed("BenchmarkTools"), Pkg.installed("PkgBenchmark")
(v"0.3.1", v"0.1.1")

this is what i get:

julia> using PkgBenchmark

julia> benchmarkpkg("IntervalArithmetic")
PkgBenchmark: Running benchmarks...
PkgBenchmark: using benchmark tuning data in /Users/forets/.julia/v0.6/.pkgbenchmark/IntervalArithmetic_tune.json
Benchmarking: 100%|█████████████████████████████████████| Time: 0:00:13
    [5/5]:    "Intervals"
      [3/3]:  "pi"
Benchmarkresults:
    Package: IntervalArithmetic
    Date: 27 Jun 2018 - 17:05
    Package commit: 148bdb
    Julia commit: 0d7248
    BenchmarkGroup:
        5-element BenchmarkTools.BenchmarkGroup:
          tags: []
          "Sum" => 2-element BenchmarkGroup([])
          "Constructors" => 1-element BenchmarkGroup([])
          "Arithmetic" => 4-element BenchmarkGroup([])
          "Elementary functions" => 4-element BenchmarkGroup([])
          "Intervals" => 3-element BenchmarkGroup([])
 

[dpsanders]

Sorry, yes that worked, but then I was trying to follow the instructions in the file to do

results = judge("IntervalArithmetic", "v0.13.0") 

That seems to be where things went wrong.

[mforets]

Ah, OK i see. It is complaining because "v0.13.0" still uses @benchgroup. Do you want me to remove those instructions?

[dpsanders]

Ah I see. Isn't that a bug not to use the same version of the benchmarks file for the two different versions?

…

On Wed, 27 Jun 2018, 4:22 pm Marcelo Forets, ***@***.***> wrote: Ah, OK i see. It is complaining because "v0.13.0" still uses @benchgroup. Do you want me to remove those instructions? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#156 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALtTp6kylekVT2uYMj6IrVpGK6hvHYxks5uA-lwgaJpZM4USaLc> .

[lbenet]

It is perhaps a good idea to include REQUIRE file in the benchmarks directory, with the minimum requirements of the packages needed.

[mforets]

Admittedly if there are two different versions of benchmarks.jl in each commit/version, it seems like comparing apples to oranges... but i don't know if this should be considered a bug.

[dpsanders]

Thanks for the contribution!