Setup BenchmarkCI #737

tkf · 2020-02-18T22:57:33Z

This PR sets up BenchmarkCI.jl. This is what happens if/after this PR is merged:

Every push to the PR triggers a CI job that runs benchmarks defined in benchmark/benchmarks.jl for the head of PR branch and the master.
The benchmark result is posted as comment like this Setup BenchmarkCI #737 (comment) for every push, when the PR is created from the branch from this repository.
If the PR is created from a fork, CI job is run but the comment will not be posted.
- As a simple solution, the contributors can create a PR in their GitHub repository from the same branch. This will run the benchmark CI on their PR.

Since the benchmarks are run against master by the default setting, I tweaked the setting to use the branch tkf/pkgbenchmark (#736). This way, we can check that it is working. We should revert this commit once #736 is merged.

Questions:

Do we want to run benchmarks for every push on every PR?
Do we want to get a comment for every push on every PR?
Is this how we want to organize benchmark/*? Let's discuss this in Minimal benchmark setup #736.

For example, if we want to just run benchmarks (without spammed by the comments), we can just remove these lines:

StaticArrays.jl/.github/workflows/benchmark.yml

Lines 18 to 21 in 84ca6a4

    
           - name: Post results 
        
             run: julia -e 'using BenchmarkCI; BenchmarkCI.postjudge()' 
        
             env: 
        
               GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

TODOs before merging this PR:

merge Minimal benchmark setup #736
remove baseline = "origin/tkf/pkgbenchmark" c157922

github-actions · 2020-02-18T23:02:34Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmarks:
- Target: 18 Feb 2020 - 23:00
- Baseline: 18 Feb 2020 - 23:02
Package commits:
- Target: e0a8fc
- Baseline: c511e5
Julia commits:
- Target: 2d5741
- Baseline: 2d5741
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr"]

Julia versioninfo

Target

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      13800 s          0 s       1049 s      10160 s          0 s
       #2  2095 MHz       3422 s          0 s       1501 s      19660 s          0 s
       
  Memory: 6.782741546630859 GB (3747.8359375 MB free)
  Uptime: 263.0 sec
  Load Avg:  1.1484375  0.6923828125  0.30322265625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      19678 s          0 s       1200 s      14991 s          0 s
       #2  2095 MHz       8498 s          0 s       1551 s      25387 s          0 s
       
  Memory: 6.782741546630859 GB (3730.26953125 MB free)
  Uptime: 372.0 sec
  Load Avg:  1.1171875  0.82177734375  0.396484375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 18 Feb 2020 - 23:0
Package commit: e0a8fc
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["qr", "Matrix"]`	40.701 μs (5%)		11.84 KiB (1%)	4
`["qr", "SMatrix"]`	54.201 μs (5%)		31.56 KiB (1%)	10

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      13800 s          0 s       1049 s      10160 s          0 s
       #2  2095 MHz       3422 s          0 s       1501 s      19660 s          0 s
       
  Memory: 6.782741546630859 GB (3747.8359375 MB free)
  Uptime: 263.0 sec
  Load Avg:  1.1484375  0.6923828125  0.30322265625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 18 Feb 2020 - 23:2
Package commit: c511e5
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["qr", "Matrix"]`	39.300 μs (5%)		11.84 KiB (1%)	4
`["qr", "SMatrix"]`	54.800 μs (5%)		31.56 KiB (1%)	10

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      19678 s          0 s       1200 s      14991 s          0 s
       #2  2095 MHz       8498 s          0 s       1551 s      25387 s          0 s
       
  Memory: 6.782741546630859 GB (3730.26953125 MB free)
  Uptime: 372.0 sec
  Load Avg:  1.1171875  0.82177734375  0.396484375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`openblas64`
`Sys.CPU_THREADS`	2

lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Stepping:            4
CPU MHz:             2095.077
BogoMIPS:            4190.15
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            36608K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves

Cpu Property	Value
Brand	Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Vendor	:Intel
Architecture	:Skylake
Model	Family: 0x06, Model: 0x55, Stepping: 0x04, Type: 0x00
Cores	2 physical cores, 2 logical cores (on executing CPU)
	No Hyperthreading detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 1024, 36608) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 44 bits physical
SIMD	512 bit = 64 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

github-actions · 2020-02-18T23:57:59Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmarks:
- Target: 18 Feb 2020 - 23:54
- Baseline: 18 Feb 2020 - 23:57
Package commits:
- Target: 00c124
- Baseline: 111ad5
Julia commits:
- Target: 2d5741
- Baseline: 2d5741
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["qr", "S=10", "Matrix"]`	0.88 (5%) ✅	1.00 (1%)
`["qr", "S=12", "SMatrix"]`	0.88 (5%) ✅	1.00 (1%)
`["qr", "S=13", "Matrix"]`	1.17 (5%) ❌	1.00 (1%)
`["qr", "S=16", "SMatrix"]`	0.88 (5%) ✅	1.00 (1%)
`["qr", "S=17", "Matrix"]`	0.90 (5%) ✅	1.00 (1%)
`["qr", "S=17", "SMatrix"]`	1.09 (5%) ❌	1.00 (1%)
`["qr", "S=18", "Matrix"]`	1.10 (5%) ❌	1.00 (1%)
`["qr", "S=18", "SMatrix"]`	1.05 (5%) ❌	1.00 (1%)
`["qr", "S=19", "SMatrix"]`	0.90 (5%) ✅	1.00 (1%)
`["qr", "S=2", "Matrix"]`	1.16 (5%) ❌	1.00 (1%)
`["qr", "S=2", "SMatrix"]`	0.95 (5%) ✅	1.00 (1%)
`["qr", "S=20", "SMatrix"]`	0.93 (5%) ✅	1.00 (1%)
`["qr", "S=22", "Matrix"]`	0.91 (5%) ✅	1.00 (1%)
`["qr", "S=22", "SMatrix"]`	0.90 (5%) ✅	1.00 (1%)
`["qr", "S=5", "Matrix"]`	1.07 (5%) ❌	1.00 (1%)
`["qr", "S=6", "Matrix"]`	0.93 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Target

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      16471 s          0 s       3500 s      21218 s          0 s
       #2  2095 MHz      25126 s          0 s       3686 s      11824 s          0 s
       
  Memory: 6.782741546630859 GB (3598.51953125 MB free)
  Uptime: 423.0 sec
  Load Avg:  1.33349609375  1.07177734375  0.5517578125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      21823 s          0 s       4094 s      33940 s          0 s
       #2  2095 MHz      38768 s          0 s       3949 s      16603 s          0 s
       
  Memory: 6.782741546630859 GB (3529.265625 MB free)
  Uptime: 610.0 sec
  Load Avg:  1.0751953125  1.04833984375  0.64599609375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 18 Feb 2020 - 23:54
Package commit: 00c124
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	113.034 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.900 ns (5%)
`["qr", "S=10", "Matrix"]`	11.300 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	782.721 ns (5%)
`["qr", "S=11", "Matrix"]`	12.701 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	1.270 μs (5%)
`["qr", "S=12", "Matrix"]`	14.301 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.370 μs (5%)
`["qr", "S=13", "Matrix"]`	18.001 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	1.900 μs (5%)
`["qr", "S=14", "Matrix"]`	17.201 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	2.233 μs (5%)
`["qr", "S=15", "Matrix"]`	21.300 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	3.225 μs (5%)
`["qr", "S=16", "Matrix"]`	31.001 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	3.963 μs (5%)
`["qr", "S=17", "Matrix"]`	29.902 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	45.102 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	35.302 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	47.602 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	33.802 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	45.702 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	1.310 μs (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	10.410 ns (5%)
`["qr", "S=20", "Matrix"]`	36.001 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	49.702 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	42.301 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	56.903 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	40.802 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	54.202 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	2.433 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	39.417 ns (5%)
`["qr", "S=4", "Matrix"]`	3.413 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	67.793 ns (5%)
`["qr", "S=5", "Matrix"]`	4.843 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	153.645 ns (5%)
`["qr", "S=6", "Matrix"]`	6.167 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	206.831 ns (5%)
`["qr", "S=7", "Matrix"]`	7.475 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	308.961 ns (5%)
`["qr", "S=8", "Matrix"]`	8.467 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	353.746 ns (5%)
`["qr", "S=9", "Matrix"]`	9.800 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	586.463 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      16471 s          0 s       3500 s      21218 s          0 s
       #2  2095 MHz      25126 s          0 s       3686 s      11824 s          0 s
       
  Memory: 6.782741546630859 GB (3598.51953125 MB free)
  Uptime: 423.0 sec
  Load Avg:  1.33349609375  1.07177734375  0.5517578125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 18 Feb 2020 - 23:57
Package commit: 111ad5
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	117.927 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.900 ns (5%)
`["qr", "S=10", "Matrix"]`	12.801 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	785.654 ns (5%)
`["qr", "S=11", "Matrix"]`	12.501 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	1.270 μs (5%)
`["qr", "S=12", "Matrix"]`	14.101 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.560 μs (5%)
`["qr", "S=13", "Matrix"]`	15.401 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	1.880 μs (5%)
`["qr", "S=14", "Matrix"]`	17.102 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	2.267 μs (5%)
`["qr", "S=15", "Matrix"]`	20.802 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	3.175 μs (5%)
`["qr", "S=16", "Matrix"]`	31.503 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	4.525 μs (5%)
`["qr", "S=17", "Matrix"]`	33.303 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	41.504 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	32.203 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	45.203 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	35.503 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	50.505 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	1.130 μs (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	11.012 ns (5%)
`["qr", "S=20", "Matrix"]`	36.303 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	53.705 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	42.304 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	56.804 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	44.605 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	59.906 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	2.511 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	38.511 ns (5%)
`["qr", "S=4", "Matrix"]`	3.388 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	66.878 ns (5%)
`["qr", "S=5", "Matrix"]`	4.515 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	153.282 ns (5%)
`["qr", "S=6", "Matrix"]`	6.651 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	207.075 ns (5%)
`["qr", "S=7", "Matrix"]`	7.401 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	299.245 ns (5%)
`["qr", "S=8", "Matrix"]`	8.301 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	347.796 ns (5%)
`["qr", "S=9", "Matrix"]`	9.600 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	605.701 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      21823 s          0 s       4094 s      33940 s          0 s
       #2  2095 MHz      38768 s          0 s       3949 s      16603 s          0 s
       
  Memory: 6.782741546630859 GB (3529.265625 MB free)
  Uptime: 610.0 sec
  Load Avg:  1.0751953125  1.04833984375  0.64599609375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`openblas64`
`Sys.CPU_THREADS`	2

lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Stepping:            4
CPU MHz:             2095.233
BogoMIPS:            4190.46
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            36608K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves

Cpu Property	Value
Brand	Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Vendor	:Intel
Architecture	:Skylake
Model	Family: 0x06, Model: 0x55, Stepping: 0x04, Type: 0x00
Cores	2 physical cores, 2 logical cores (on executing CPU)
	No Hyperthreading detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 1024, 36608) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 44 bits physical
SIMD	512 bit = 64 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

coveralls · 2020-02-19T00:19:02Z

Coverage remained the same at 81.975% when pulling 84ca6a4 on tkf/benchmarkci into 7a78efd on master.

github-actions · 2020-02-19T01:42:14Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmarks:
- Target: 19 Feb 2020 - 01:39
- Baseline: 19 Feb 2020 - 01:42
Package commits:
- Target: 5cc95e
- Baseline: bd834f
Julia commits:
- Target: 2d5741
- Baseline: 2d5741
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["qr", "S=1", "Matrix"]`	0.93 (5%) ✅	1.00 (1%)
`["qr", "S=1", "SMatrix"]`	0.87 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Target

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      25417 s          0 s       3523 s      11213 s          0 s
       #2  2095 MHz      14511 s          0 s       3547 s      22264 s          0 s
       
  Memory: 6.782741546630859 GB (3582.4453125 MB free)
  Uptime: 417.0 sec
  Load Avg:  1.4482421875  1.10009765625  0.5498046875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      26463 s          0 s       4119 s      27226 s          0 s
       #2  2095 MHz      31454 s          0 s       3779 s      22762 s          0 s
       
  Memory: 6.782741546630859 GB (3532.2421875 MB free)
  Uptime: 595.0 sec
  Load Avg:  1.23046875  1.12109375  0.6591796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 19 Feb 2020 - 1:39
Package commit: 5cc95e
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	98.841 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.400 ns (5%)
`["qr", "S=10", "Matrix"]`	9.600 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	689.823 ns (5%)
`["qr", "S=11", "Matrix"]`	11.200 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	993.750 ns (5%)
`["qr", "S=12", "Matrix"]`	12.500 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.180 μs (5%)
`["qr", "S=13", "Matrix"]`	13.800 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	1.660 μs (5%)
`["qr", "S=14", "Matrix"]`	15.200 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	1.940 μs (5%)
`["qr", "S=15", "Matrix"]`	16.300 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	2.800 μs (5%)
`["qr", "S=16", "Matrix"]`	24.901 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	3.475 μs (5%)
`["qr", "S=17", "Matrix"]`	26.501 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	36.901 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	28.501 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	39.201 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	30.002 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	41.302 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	985.714 ns (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	9.510 ns (5%)
`["qr", "S=20", "Matrix"]`	32.001 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	44.401 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	33.601 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	46.501 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	35.201 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	48.802 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	1.900 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	33.838 ns (5%)
`["qr", "S=4", "Matrix"]`	2.945 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	57.377 ns (5%)
`["qr", "S=5", "Matrix"]`	3.925 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	132.920 ns (5%)
`["qr", "S=6", "Matrix"]`	5.050 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	182.635 ns (5%)
`["qr", "S=7", "Matrix"]`	6.040 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	264.646 ns (5%)
`["qr", "S=8", "Matrix"]`	7.250 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	305.233 ns (5%)
`["qr", "S=9", "Matrix"]`	8.334 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	521.068 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      25417 s          0 s       3523 s      11213 s          0 s
       #2  2095 MHz      14511 s          0 s       3547 s      22264 s          0 s
       
  Memory: 6.782741546630859 GB (3582.4453125 MB free)
  Uptime: 417.0 sec
  Load Avg:  1.4482421875  1.10009765625  0.5498046875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 19 Feb 2020 - 1:42
Package commit: bd834f
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	106.669 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.600 ns (5%)
`["qr", "S=10", "Matrix"]`	9.701 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	701.429 ns (5%)
`["qr", "S=11", "Matrix"]`	11.301 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	981.313 ns (5%)
`["qr", "S=12", "Matrix"]`	12.501 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.190 μs (5%)
`["qr", "S=13", "Matrix"]`	13.902 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	1.650 μs (5%)
`["qr", "S=14", "Matrix"]`	15.401 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	1.950 μs (5%)
`["qr", "S=15", "Matrix"]`	16.201 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	2.822 μs (5%)
`["qr", "S=16", "Matrix"]`	25.003 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	3.475 μs (5%)
`["qr", "S=17", "Matrix"]`	26.402 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	36.303 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	28.103 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	39.703 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	29.902 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	41.504 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	985.786 ns (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	9.510 ns (5%)
`["qr", "S=20", "Matrix"]`	31.903 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	43.803 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	33.803 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	47.404 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	35.803 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	48.504 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	1.920 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	33.839 ns (5%)
`["qr", "S=4", "Matrix"]`	2.945 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	57.482 ns (5%)
`["qr", "S=5", "Matrix"]`	3.938 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	132.587 ns (5%)
`["qr", "S=6", "Matrix"]`	5.050 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	182.350 ns (5%)
`["qr", "S=7", "Matrix"]`	6.040 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	266.180 ns (5%)
`["qr", "S=8", "Matrix"]`	7.226 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	304.450 ns (5%)
`["qr", "S=9", "Matrix"]`	8.367 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	521.089 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2095 MHz      26463 s          0 s       4119 s      27226 s          0 s
       #2  2095 MHz      31454 s          0 s       3779 s      22762 s          0 s
       
  Memory: 6.782741546630859 GB (3532.2421875 MB free)
  Uptime: 595.0 sec
  Load Avg:  1.23046875  1.12109375  0.6591796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`openblas64`
`Sys.CPU_THREADS`	2

lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Stepping:            4
CPU MHz:             2095.198
BogoMIPS:            4190.39
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            36608K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves

Cpu Property	Value
Brand	Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Vendor	:Intel
Architecture	:Skylake
Model	Family: 0x06, Model: 0x55, Stepping: 0x04, Type: 0x00
Cores	2 physical cores, 2 logical cores (on executing CPU)
	No Hyperthreading detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 1024, 36608) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 44 bits physical
SIMD	512 bit = 64 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

This reverts commit e4062db.

tkf · 2020-02-19T08:03:03Z

@c42f I think this is good to go, if you want to use the default setup.

github-actions · 2020-02-19T08:11:27Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmarks:
- Target: 19 Feb 2020 - 08:07
- Baseline: 19 Feb 2020 - 08:11
Package commits:
- Target: ebfcdc
- Baseline: d31793
Julia commits:
- Target: 2d5741
- Baseline: 2d5741
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["qr", "S=1", "SMatrix"]`	0.93 (5%) ✅	1.00 (1%)
`["qr", "S=12", "Matrix"]`	1.09 (5%) ❌	1.00 (1%)
`["qr", "S=12", "SMatrix"]`	0.88 (5%) ✅	1.00 (1%)
`["qr", "S=13", "Matrix"]`	1.12 (5%) ❌	1.00 (1%)
`["qr", "S=15", "SMatrix"]`	1.16 (5%) ❌	1.00 (1%)
`["qr", "S=16", "SMatrix"]`	0.95 (5%) ✅	1.00 (1%)
`["qr", "S=17", "Matrix"]`	1.07 (5%) ❌	1.00 (1%)
`["qr", "S=3", "SMatrix"]`	0.94 (5%) ✅	1.00 (1%)
`["qr", "S=4", "Matrix"]`	0.94 (5%) ✅	1.00 (1%)
`["qr", "S=8", "Matrix"]`	0.90 (5%) ✅	1.00 (1%)
`["qr", "S=9", "Matrix"]`	0.93 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Target

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz      15830 s          0 s       3228 s      30383 s          0 s
       #2  2294 MHz      30496 s          0 s       2704 s      16096 s          0 s
       
  Memory: 6.782741546630859 GB (3561.8125 MB free)
  Uptime: 508.0 sec
  Load Avg:  1.310546875  1.06201171875  0.548828125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)

Baseline

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz      35438 s          0 s       3478 s      30650 s          0 s
       #2  2294 MHz      31560 s          0 s       3223 s      34646 s          0 s
       
  Memory: 6.782741546630859 GB (3489.5 MB free)
  Uptime: 710.0 sec
  Load Avg:  1.140625  1.068359375  0.65966796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 19 Feb 2020 - 8:7
Package commit: ebfcdc
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	105.550 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.400 ns (5%)
`["qr", "S=10", "Matrix"]`	12.900 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	781.553 ns (5%)
`["qr", "S=11", "Matrix"]`	14.600 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	1.150 μs (5%)
`["qr", "S=12", "Matrix"]`	17.700 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.270 μs (5%)
`["qr", "S=13", "Matrix"]`	20.000 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	2.110 μs (5%)
`["qr", "S=14", "Matrix"]`	19.800 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	2.622 μs (5%)
`["qr", "S=15", "Matrix"]`	21.900 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	4.175 μs (5%)
`["qr", "S=16", "Matrix"]`	35.700 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	4.529 μs (5%)
`["qr", "S=17", "Matrix"]`	39.000 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	50.700 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	38.800 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	54.101 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	42.100 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	57.800 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	1.020 μs (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	12.525 ns (5%)
`["qr", "S=20", "Matrix"]`	43.500 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	61.200 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	46.200 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	66.801 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	48.900 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	68.200 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	2.533 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	41.355 ns (5%)
`["qr", "S=4", "Matrix"]`	3.929 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	75.880 ns (5%)
`["qr", "S=5", "Matrix"]`	5.217 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	146.484 ns (5%)
`["qr", "S=6", "Matrix"]`	6.750 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	192.150 ns (5%)
`["qr", "S=7", "Matrix"]`	8.033 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	288.693 ns (5%)
`["qr", "S=8", "Matrix"]`	9.800 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	356.585 ns (5%)
`["qr", "S=9", "Matrix"]`	11.200 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	621.643 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz      15830 s          0 s       3228 s      30383 s          0 s
       #2  2294 MHz      30496 s          0 s       2704 s      16096 s          0 s
       
  Memory: 6.782741546630859 GB (3561.8125 MB free)
  Uptime: 508.0 sec
  Load Avg:  1.310546875  1.06201171875  0.548828125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Time of benchmark: 19 Feb 2020 - 8:11
Package commit: d31793
Julia commit: 2d5741
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["qr", "S=1", "Matrix"]`	105.763 ns (5%)	320 bytes (1%)	4
`["qr", "S=1", "SMatrix"]`	1.500 ns (5%)
`["qr", "S=10", "Matrix"]`	12.900 μs (5%)	2.66 KiB (1%)	4
`["qr", "S=10", "SMatrix"]`	777.670 ns (5%)
`["qr", "S=11", "Matrix"]`	14.500 μs (5%)	3.22 KiB (1%)	4
`["qr", "S=11", "SMatrix"]`	1.140 μs (5%)
`["qr", "S=12", "Matrix"]`	16.201 μs (5%)	3.69 KiB (1%)	4
`["qr", "S=12", "SMatrix"]`	1.440 μs (5%)
`["qr", "S=13", "Matrix"]`	17.900 μs (5%)	4.39 KiB (1%)	4
`["qr", "S=13", "SMatrix"]`	2.020 μs (5%)
`["qr", "S=14", "Matrix"]`	19.700 μs (5%)	5.33 KiB (1%)	4
`["qr", "S=14", "SMatrix"]`	2.644 μs (5%)
`["qr", "S=15", "Matrix"]`	21.400 μs (5%)	5.98 KiB (1%)	4
`["qr", "S=15", "SMatrix"]`	3.612 μs (5%)
`["qr", "S=16", "Matrix"]`	34.500 μs (5%)	6.41 KiB (1%)	4
`["qr", "S=16", "SMatrix"]`	4.786 μs (5%)
`["qr", "S=17", "Matrix"]`	36.600 μs (5%)	7.34 KiB (1%)	4
`["qr", "S=17", "SMatrix"]`	51.100 μs (5%)	19.56 KiB (1%)	10
`["qr", "S=18", "Matrix"]`	40.200 μs (5%)	8.09 KiB (1%)	4
`["qr", "S=18", "SMatrix"]`	53.700 μs (5%)	21.56 KiB (1%)	10
`["qr", "S=19", "Matrix"]`	40.900 μs (5%)	9.03 KiB (1%)	4
`["qr", "S=19", "SMatrix"]`	57.200 μs (5%)	24.06 KiB (1%)	10
`["qr", "S=2", "Matrix"]`	1.010 μs (5%)	368 bytes (1%)	4
`["qr", "S=2", "SMatrix"]`	12.525 ns (5%)
`["qr", "S=20", "Matrix"]`	43.700 μs (5%)	9.78 KiB (1%)	4
`["qr", "S=20", "SMatrix"]`	61.500 μs (5%)	26.06 KiB (1%)	10
`["qr", "S=21", "Matrix"]`	45.900 μs (5%)	10.91 KiB (1%)	4
`["qr", "S=21", "SMatrix"]`	65.000 μs (5%)	29.06 KiB (1%)	10
`["qr", "S=22", "Matrix"]`	48.200 μs (5%)	11.84 KiB (1%)	4
`["qr", "S=22", "SMatrix"]`	67.800 μs (5%)	31.56 KiB (1%)	10
`["qr", "S=3", "Matrix"]`	2.522 μs (5%)	512 bytes (1%)	4
`["qr", "S=3", "SMatrix"]`	43.883 ns (5%)
`["qr", "S=4", "Matrix"]`	4.200 μs (5%)	656 bytes (1%)	4
`["qr", "S=4", "SMatrix"]`	76.812 ns (5%)
`["qr", "S=5", "Matrix"]`	5.233 μs (5%)	896 bytes (1%)	4
`["qr", "S=5", "SMatrix"]`	144.577 ns (5%)
`["qr", "S=6", "Matrix"]`	6.750 μs (5%)	1.11 KiB (1%)	4
`["qr", "S=6", "SMatrix"]`	194.393 ns (5%)
`["qr", "S=7", "Matrix"]`	8.033 μs (5%)	1.48 KiB (1%)	4
`["qr", "S=7", "SMatrix"]`	289.753 ns (5%)
`["qr", "S=8", "Matrix"]`	10.900 μs (5%)	1.86 KiB (1%)	4
`["qr", "S=8", "SMatrix"]`	345.366 ns (5%)
`["qr", "S=9", "Matrix"]`	12.000 μs (5%)	2.19 KiB (1%)	4
`["qr", "S=9", "SMatrix"]`	615.789 ns (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["qr", "S=1"]
["qr", "S=10"]
["qr", "S=11"]
["qr", "S=12"]
["qr", "S=13"]
["qr", "S=14"]
["qr", "S=15"]
["qr", "S=16"]
["qr", "S=17"]
["qr", "S=18"]
["qr", "S=19"]
["qr", "S=2"]
["qr", "S=20"]
["qr", "S=21"]
["qr", "S=22"]
["qr", "S=3"]
["qr", "S=4"]
["qr", "S=5"]
["qr", "S=6"]
["qr", "S=7"]
["qr", "S=8"]
["qr", "S=9"]

Julia versioninfo

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.0.0-1031-azure #33-Ubuntu SMP Thu Feb 6 22:26:13 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz      35438 s          0 s       3478 s      30650 s          0 s
       #2  2294 MHz      31560 s          0 s       3223 s      34646 s          0 s
       
  Memory: 6.782741546630859 GB (3489.5 MB free)
  Uptime: 710.0 sec
  Load Avg:  1.140625  1.068359375  0.65966796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`openblas64`
`Sys.CPU_THREADS`	2

lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               79
Model name:          Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Stepping:            1
CPU MHz:             2294.686
BogoMIPS:            4589.37
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            51200K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt md_clear

Cpu Property	Value
Brand	Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Vendor	:Intel
Architecture	:Broadwell
Model	Family: 0x06, Model: 0x4f, Stepping: 0x01, Type: 0x00
Cores	2 physical cores, 2 logical cores (on executing CPU)
	No Hyperthreading detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 256, 51200) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 44 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

andyferris · 2020-02-19T11:01:21Z

Cool! I for one think this is a great idea.

If I had the energy I'd move the existing CI to github workflows too!

c42f · 2020-02-19T11:41:20Z

Do we want to run benchmarks for every push on every PR?
Do we want to get a comment for every push on every PR?

I feel the comments might get a bit excessive, but maybe we try it and see?

tkf · 2020-02-19T21:18:00Z

OK. I'll merge this as-is and open a PR for an alternative setup to suppress comments. You can merge that setup if you want to avoid spamming from BenchmarkCI.

tkf force-pushed the tkf/benchmarkci branch from c157922 to 7092a84 Compare February 18, 2020 23:08

tkf added 3 commits February 18, 2020 15:47

[ci skip] Minimal benchmark setup

111ad5c

Setup BenchmarkCI

b092b00

[ci skip] WIP: set baseline = "origin/tkf/pkgbenchmark"

e4062db

tkf force-pushed the tkf/benchmarkci branch from 7092a84 to e4062db Compare February 18, 2020 23:47

tkf added 3 commits February 18, 2020 17:23

[ci skip] Slightly more robust benchmarks.jl

403ea5a

[ci skip] Note on how files in benchmark/ are organized

bd834ff

Merge branch 'tkf/pkgbenchmark' into tkf/benchmarkci

84ca6a4

tkf marked this pull request as ready for review February 19, 2020 01:49

tkf mentioned this pull request Feb 19, 2020

Minimal benchmark setup #736

Merged

tkf added benchmarking DO NOT MERGE labels Feb 19, 2020

tkf added 2 commits February 18, 2020 23:57

[ci skip] Merge branch 'master' into tkf/benchmarkci

feff262

[ci skip] Revert "WIP: set baseline = "origin/tkf/pkgbenchmark""

fc06ab9

This reverts commit e4062db.

tkf removed the DO NOT MERGE label Feb 19, 2020

tkf mentioned this pull request Feb 19, 2020

Use const for suite #739

Merged

andyferris approved these changes Feb 19, 2020

View reviewed changes

tkf merged commit ff2493f into master Feb 19, 2020

tkf deleted the tkf/benchmarkci branch February 19, 2020 21:18

tkf mentioned this pull request Feb 19, 2020

Suppress comments from BenchmarkCI #741

Merged

	- name: Post results
	run: julia -e 'using BenchmarkCI; BenchmarkCI.postjudge()'
	env:
	GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Setup BenchmarkCI #737

Setup BenchmarkCI #737

Conversation

tkf commented Feb 18, 2020 • edited

github-actions bot commented Feb 18, 2020

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

github-actions bot commented Feb 18, 2020

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

coveralls commented Feb 19, 2020 • edited

github-actions bot commented Feb 19, 2020

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

tkf commented Feb 19, 2020

github-actions bot commented Feb 19, 2020

Judge result

Benchmark Report for /home/runner/work/StaticArrays.jl/StaticArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

tkf commented Feb 18, 2020 •

edited

coveralls commented Feb 19, 2020 •

edited