Skip to content
This repository has been archived by the owner on Dec 18, 2021. It is now read-only.

Transposed storage #30

Merged
merged 18 commits into from
Oct 10, 2019
Merged

Transposed storage #30

merged 18 commits into from
Oct 10, 2019

Conversation

GiggleLiu
Copy link
Member

@GiggleLiu GiggleLiu commented Aug 30, 2019

Tranposed storage are more batch friendly, with better performance.
All changes are included in this commit: 86d74bd and 17ec5df

@Roger-luo Could you please do benchmark on this branch, and see how it work?

@codecov
Copy link

codecov bot commented Aug 30, 2019

Codecov Report

Merging #30 into master will increase coverage by 0.41%.
The diff coverage is 95%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #30      +/-   ##
=========================================
+ Coverage   80.68%   81.1%   +0.41%     
=========================================
  Files           8       8              
  Lines         497     508      +11     
=========================================
+ Hits          401     412      +11     
  Misses         96      96
Impacted Files Coverage Δ
src/utils.jl 84.21% <100%> (+0.51%) ⬆️
src/register.jl 75.55% <94.11%> (+2.38%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7ff3a1...f41864f. Read the comment docs.

@Roger-luo
Copy link
Member

Yes, I can check if this works better.

I'm actually trying out my own version as well!

@Roger-luo Roger-luo mentioned this pull request Aug 30, 2019
@GiggleLiu
Copy link
Member Author

Ready to merge?

@Roger-luo
Copy link
Member

Benchmark Report for YaoArrayRegister

Job Properties

  • Time of benchmarks:
    • Target: 18 Sep 2019 - 18:03
    • Baseline: 18 Sep 2019 - 18:27
  • Package commits:
    • Target: c27752
    • Baseline: 7cdb70
  • Julia commits:
    • Target: 55e36c
    • Baseline: 55e36c
  • Julia command flags:
    • Target: -O3
    • Baseline: -O3
  • Environment variables:
    • Target: JULIA_NUM_THREADS => 4
    • Baseline: JULIA_NUM_THREADS => 1

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.35 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.52 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.32 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.40 (5%) ❌ 2.57 (1%) ❌
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.37 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.38 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(3, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 0.95 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.91 (5%) ✅ 0.94 (1%) ✅
["matrices", "contiguous", "ordered", "(5, \"Complex{Float64}\", \"MArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 0.93 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(5, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.90 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(5, \"Complex{Float64}\", \"SArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 0.87 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "ordered", "(7, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.91 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.36 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.56 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.33 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.33 (5%) ❌ 2.57 (1%) ❌
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.46 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(1, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.47 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.07 (5%) ❌ 1.00 (1%)
["matrices", "contiguous", "random", "(3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.87 (5%) ✅ 0.96 (1%) ✅
["matrices", "contiguous", "random", "(3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.95 (5%) 0.92 (1%) ✅
["matrices", "contiguous", "random", "(3, \"Complex{Float64}\", \"SArray{Tuple{8,8},Complex{Float64},2,64}\")"] 0.64 (5%) ✅ 0.62 (1%) ✅
["matrices", "contiguous", "random", "(3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.97 (5%) 0.93 (1%) ✅
["matrices", "contiguous", "random", "(5, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.11 (5%) ❌ 1.00 (1%)
["matrices", "contiguous", "random", "(5, \"Complex{Float64}\", \"MArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 0.92 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(5, \"Complex{Float64}\", \"SArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 1.20 (5%) ❌ 1.00 (1%)
["matrices", "contiguous", "random", "(5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.01 (5%) 0.93 (1%) ✅
["matrices", "contiguous", "random", "(7, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.06 (5%) ❌ 1.00 (1%)
["matrices", "contiguous", "random", "(7, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.91 (5%) ✅ 1.00 (1%)
["matrices", "contiguous", "random", "(7, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.94 (5%) ✅ 0.98 (1%) ✅
["matrices", "contiguous", "random", "(9, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.15 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(1, 3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.93 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(1, 3, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(1, 3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.74 (5%) ✅ 0.70 (1%) ✅
["matrices", "controlled", "ordered", "(1, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.97 (5%) 1.03 (1%) ❌
["matrices", "controlled", "ordered", "(3, 3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.91 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(3, 3, \"Complex{Float64}\", \"SArray{Tuple{8,8},Complex{Float64},2,64}\")"] 0.89 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(3, 5, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(3, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.99 (5%) 0.93 (1%) ✅
["matrices", "controlled", "ordered", "(5, 3, \"Complex{Float64}\", \"SArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 1.29 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(5, 5, \"Complex{Float64}\", \"MArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 0.89 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(5, 5, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.92 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "ordered", "(7, 3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.08 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(7, 5, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.07 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(7, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.08 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(9, 3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.09 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(9, 3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.05 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "ordered", "(9, 5, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.10 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(1, 3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.03 (5%) 1.01 (1%) ❌
["matrices", "controlled", "random", "(1, 3, \"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.97 (5%) 0.98 (1%) ✅
["matrices", "controlled", "random", "(1, 3, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.99 (5%) 1.02 (1%) ❌
["matrices", "controlled", "random", "(1, 3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.78 (5%) ✅ 0.71 (1%) ✅
["matrices", "controlled", "random", "(1, 5, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.00 (5%) 1.01 (1%) ❌
["matrices", "controlled", "random", "(1, 5, \"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.93 (5%) ✅ 1.00 (1%)
["matrices", "controlled", "random", "(1, 5, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 1.05 (5%) 1.02 (1%) ❌
["matrices", "controlled", "random", "(1, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.04 (5%) 1.03 (1%) ❌
["matrices", "controlled", "random", "(3, 3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.14 (5%) ❌ 1.01 (1%)
["matrices", "controlled", "random", "(3, 3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.07 (5%) ❌ 1.02 (1%) ❌
["matrices", "controlled", "random", "(3, 3, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 0.66 (5%) ✅ 0.64 (1%) ✅
["matrices", "controlled", "random", "(3, 3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.00 (5%) 0.92 (1%) ✅
["matrices", "controlled", "random", "(3, 3, \"Complex{Float64}\", \"SArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.08 (5%) ❌ 1.01 (1%)
["matrices", "controlled", "random", "(3, 5, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.99 (5%) 0.84 (1%) ✅
["matrices", "controlled", "random", "(3, 5, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.15 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(3, 5, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.10 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(3, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.94 (5%) ✅ 0.75 (1%) ✅
["matrices", "controlled", "random", "(5, 3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.11 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(5, 3, \"Complex{Float64}\", \"MArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 1.14 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(5, 3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.99 (5%) 0.99 (1%) ✅
["matrices", "controlled", "random", "(5, 3, \"Complex{Float64}\", \"SArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 1.05 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(5, 5, \"Complex{Float64}\", \"MArray{Tuple{32,32},Complex{Float64},2,1024}\")"] 0.89 (5%) ✅ 0.27 (1%) ✅
["matrices", "controlled", "random", "(5, 5, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.04 (5%) 1.02 (1%) ❌
["matrices", "controlled", "random", "(5, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.00 (5%) 0.98 (1%) ✅
["matrices", "controlled", "random", "(7, 3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.12 (5%) ❌ 1.00 (1%)
["matrices", "controlled", "random", "(7, 5, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.06 (5%) ❌ 1.09 (1%) ❌
["matrices", "controlled", "random", "(7, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.04 (5%) 1.02 (1%) ❌
["matrices", "controlled", "random", "(9, 3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.96 (5%) 1.04 (1%) ❌
["matrices", "controlled", "random", "(9, 3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.08 (5%) ❌ 1.02 (1%) ❌
["matrices", "controlled", "random", "(9, 3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.05 (5%) ❌ 0.98 (1%) ✅
["matrices", "controlled", "random", "(9, 5, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.00 (5%) ✅ 0.04 (1%) ✅
["matrices", "controlled", "random", "(9, 5, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.98 (5%) 0.99 (1%) ✅
["matrices", "controlled", "random", "(9, 5, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.04 (5%) 1.02 (1%) ❌
["matrices", "controlled", "random", "(9, 5, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.88 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(1, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.89 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(1, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.83 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(1, \"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(1, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.53 (5%) ❌ 2.57 (1%) ❌
["matrices", "in-contiguous", "ordered", "(1, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.88 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(2, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.03 (5%) 1.02 (1%) ❌
["matrices", "in-contiguous", "ordered", "(2, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.00 (5%) 0.96 (1%) ✅
["matrices", "in-contiguous", "ordered", "(2, \"Complex{Float64}\", \"SArray{Tuple{4,4},Complex{Float64},2,16}\")"] 1.09 (5%) ❌ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(2, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.09 (5%) ❌ 1.04 (1%) ❌
["matrices", "in-contiguous", "ordered", "(3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.06 (5%) ❌ 1.02 (1%) ❌
["matrices", "in-contiguous", "ordered", "(3, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.04 (5%) 1.03 (1%) ❌
["matrices", "in-contiguous", "ordered", "(3, \"Complex{Float64}\", \"SArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.09 (5%) ❌ 1.00 (1%)
["matrices", "in-contiguous", "ordered", "(3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.00 (5%) 1.03 (1%) ❌
["matrices", "in-contiguous", "random", "(1, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 0.92 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "random", "(1, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 0.87 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "random", "(1, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 0.71 (5%) ✅ 2.57 (1%) ❌
["matrices", "in-contiguous", "random", "(1, \"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\")"] 0.82 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "random", "(1, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.95 (5%) ✅ 1.00 (1%)
["matrices", "in-contiguous", "random", "(2, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.08 (5%) ❌ 1.06 (1%) ❌
["matrices", "in-contiguous", "random", "(2, \"Complex{Float64}\", \"MArray{Tuple{4,4},Complex{Float64},2,16}\")"] 1.04 (5%) 1.02 (1%) ❌
["matrices", "in-contiguous", "random", "(2, \"Complex{Float64}\", \"SArray{Tuple{4,4},Complex{Float64},2,16}\")"] 1.36 (5%) ❌ 1.63 (1%) ❌
["matrices", "in-contiguous", "random", "(2, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 1.24 (5%) ❌ 1.27 (1%) ❌
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"Array{Complex{Float64},2}\")"] 1.06 (5%) ❌ 1.00 (1%)
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\")"] 1.10 (5%) ❌ 0.99 (1%) ✅
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"MArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.30 (5%) ❌ 1.61 (1%) ❌
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"] 1.06 (5%) ❌ 0.91 (1%) ✅
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"SArray{Tuple{8,8},Complex{Float64},2,64}\")"] 1.21 (5%) ❌ 1.53 (1%) ❌
["matrices", "in-contiguous", "random", "(3, \"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\")"] 0.99 (5%) 0.83 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 1)"] 0.34 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 13)"] 1.29 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 17)"] 1.06 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 25)"] 1.14 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 5)"] 0.48 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 9)"] 0.85 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 1)"] 0.51 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 13)"] 0.92 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 9)"] 0.84 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 1)"] 0.36 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 13)"] 0.85 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 17)"] 0.93 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 21)"] 1.22 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 5)"] 0.59 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 9)"] 0.79 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 1)"] 0.30 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 13)"] 0.93 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 17)"] 0.70 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 21)"] 0.88 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 25)"] 1.00 (5%) 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 5)"] 0.34 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 9)"] 0.85 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 1)"] 0.37 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 13)"] 0.80 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 17)"] 1.06 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 21)"] 1.05 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 5)"] 0.65 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 9)"] 0.93 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 1)"] 0.52 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 13)"] 0.80 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 21)"] 0.82 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 5)"] 0.70 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "ordered", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 9)"] 0.87 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 1)"] 0.33 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 13)"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 17)"] 1.08 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 21)"] 1.78 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 5)"] 0.48 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Array{Complex{Float64},2}\", 9)"] 0.73 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 1)"] 0.52 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 13)"] 0.85 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 17)"] 1.21 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 25)"] 1.12 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"Diagonal{Complex{Float64},Array{Complex{Float64},1}}\", 5)"] 1.14 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 1)"] 0.36 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 13)"] 1.06 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 21)"] 0.89 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 5)"] 0.66 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"MArray{Tuple{2,2},Complex{Float64},2,4}\", 9)"] 0.67 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 1)"] 0.29 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 13)"] 0.86 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 17)"] 1.20 (5%) ❌ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 21)"] 0.88 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 25)"] 0.62 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 5)"] 0.38 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\", 9)"] 0.63 (5%) ✅ 0.39 (1%) ✅
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 1)"] 0.37 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 13)"] 0.92 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 17)"] 0.91 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 21)"] 1.20 (5%) ❌ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 5)"] 0.58 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SArray{Tuple{2,2},Complex{Float64},2,4}\", 9)"] 0.83 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 1)"] 0.46 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 21)"] 0.94 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 5)"] 0.66 (5%) ✅ 1.00 (1%)
["matrices", "single qubit", "random", "(\"Complex{Float64}\", \"SparseMatrixCSC{Complex{Float64},Int64}\", 9)"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"S\", 12, 2:12, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12))"] 0.92 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"S\", 20, 2:20, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20))"] 0.95 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"S\", 4, 2:4, (2, 3, 4))"] 0.70 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"S\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Sdag\", 20, 2:20, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20))"] 1.09 (5%) ❌ 1.00 (1%)
["specialized", "multi control", "(\"Sdag\", 4, 2:4, (2, 3, 4))"] 0.79 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Sdag\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.88 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"T\", 16, 2:16, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16))"] 1.50 (5%) ❌ 1.00 (1%)
["specialized", "multi control", "(\"T\", 20, 2:20, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20))"] 1.63 (5%) ❌ 1.00 (1%)
["specialized", "multi control", "(\"T\", 24, 2:24, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24))"] 0.75 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"T\", 4, 2:4, (2, 3, 4))"] 0.80 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"T\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Tdag\", 12, 2:12, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12))"] 0.75 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Tdag\", 20, 2:20, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20))"] 1.42 (5%) ❌ 1.00 (1%)
["specialized", "multi control", "(\"Tdag\", 24, 2:24, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24))"] 0.88 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Tdag\", 4, 2:4, (2, 3, 4))"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Tdag\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.80 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"X\", 12, 2:12, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12))"] 0.94 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"X\", 4, 2:4, (2, 3, 4))"] 0.49 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"X\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.78 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Y\", 4, 2:4, (2, 3, 4))"] 0.49 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Y\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.79 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Z\", 12, 2:12, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12))"] 0.64 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Z\", 20, 2:20, (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20))"] 1.09 (5%) ❌ 1.00 (1%)
["specialized", "multi control", "(\"Z\", 4, 2:4, (2, 3, 4))"] 0.74 (5%) ✅ 1.00 (1%)
["specialized", "multi control", "(\"Z\", 8, 2:8, (2, 3, 4, 5, 6, 7, 8))"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"S\", 12, 4)"] 0.87 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"S\", 20, 6)"] 0.87 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"S\", 24, 8)"] 0.87 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"S\", 4, 2)"] 0.70 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"S\", 8, 3)"] 0.88 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Sdag\", 12, 4)"] 1.14 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Sdag\", 20, 6)"] 1.12 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Sdag\", 4, 2)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Sdag\", 8, 3)"] 0.84 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"T\", 12, 4)"] 1.37 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"T\", 20, 6)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"T\", 24, 8)"] 0.64 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"T\", 4, 2)"] 0.78 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"T\", 8, 3)"] 0.93 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Tdag\", 12, 4)"] 1.34 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Tdag\", 24, 8)"] 0.69 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Tdag\", 4, 2)"] 0.82 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Tdag\", 8, 3)"] 0.87 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"X\", 16, 5)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"X\", 20, 6)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"X\", 4, 2)"] 0.52 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"X\", 8, 3)"] 0.85 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Y\", 12, 4)"] 1.41 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Y\", 16, 5)"] 1.48 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Y\", 20, 6)"] 1.53 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Y\", 4, 2)"] 0.53 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Y\", 8, 3)"] 0.69 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Z\", 12, 4)"] 1.26 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Z\", 20, 6)"] 0.93 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Z\", 24, 8)"] 1.54 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Z\", 4, 2)"] 0.69 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit multi control", "(\"Z\", 8, 3)"] 0.91 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", 12)"] 0.93 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", 16)"] 0.95 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", 4)"] 0.42 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", 8)"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", \"random\", 20)"] 1.59 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", \"random\", 24)"] 1.54 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", \"random\", 4)"] 0.45 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"SWAP\", \"random\", 8)"] 0.83 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"S\", 4)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"S\", 8)"] 0.95 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Sdag\", 12)"] 0.94 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Sdag\", 24)"] 0.95 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Sdag\", 4)"] 0.52 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Sdag\", 8)"] 0.92 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"T\", 4)"] 0.64 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Tdag\", 16)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"Tdag\", 4)"] 0.68 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"X\", 16)"] 1.05 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"X\", 20)"] 1.43 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"X\", 24)"] 1.45 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"X\", 4)"] 0.41 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"X\", 8)"] 0.78 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Y\", 12)"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Y\", 24)"] 0.48 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Y\", 4)"] 0.50 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Y\", 8)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Z\", 12)"] 1.05 (5%) ❌ 1.00 (1%)
["specialized", "multi qubit", "(\"Z\", 16)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Z\", 4)"] 0.45 (5%) ✅ 1.00 (1%)
["specialized", "multi qubit", "(\"Z\", 8)"] 0.78 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"S\", 4, (2,), (1,))"] 0.52 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"S\", 8, (2,), (1,))"] 0.73 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Sdag\", 4, (2,), (1,))"] 0.54 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Sdag\", 8, (2,), (1,))"] 0.69 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"T\", 4, (2,), (1,))"] 0.54 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"T\", 8, (2,), (1,))"] 0.65 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Tdag\", 12, (2,), (1,))"] 0.78 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Tdag\", 16, (2,), (1,))"] 0.84 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Tdag\", 4, (2,), (1,))"] 0.51 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Tdag\", 8, (2,), (1,))"] 0.65 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"X\", 4, (2,), (1,))"] 0.55 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"X\", 8, (2,), (1,))"] 0.72 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Y\", 4, (2,), (1,))"] 0.55 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Y\", 8, (2,), (1,))"] 0.73 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Z\", 12, (2,), (1,))"] 0.94 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Z\", 4, (2,), (1,))"] 0.56 (5%) ✅ 1.00 (1%)
["specialized", "single control", "(\"Z\", 8, (2,), (1,))"] 0.62 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"S\", 12, 4)"] 0.89 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"S\", 20, 6)"] 0.85 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"S\", 4, 2)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"S\", 8, 3)"] 0.88 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Sdag\", 12, 4)"] 1.13 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Sdag\", 4, 2)"] 0.70 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Sdag\", 8, 3)"] 0.82 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"T\", 12, 4)"] 1.07 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"T\", 24, 8)"] 1.40 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"T\", 4, 2)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"T\", 8, 3)"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Tdag\", 12, 4)"] 0.73 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Tdag\", 24, 8)"] 1.60 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Tdag\", 4, 2)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Tdag\", 8, 3)"] 0.73 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"X\", 12, 4)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"X\", 16, 5)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"X\", 20, 6)"] 1.10 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"X\", 4, 2)"] 0.37 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"X\", 8, 3)"] 0.81 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Y\", 12, 4)"] 0.84 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Y\", 16, 5)"] 1.09 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Y\", 4, 2)"] 0.46 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Y\", 8, 3)"] 0.59 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Z\", 12, 4)"] 0.69 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Z\", 16, 5)"] 1.51 (5%) ❌ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Z\", 20, 6)"] 0.83 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Z\", 4, 2)"] 0.71 (5%) ✅ 1.00 (1%)
["specialized", "single qubit multi control", "(\"Z\", 8, 3)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"S\", 1)"] 0.37 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"S\", 17)"] 0.95 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"S\", 5)"] 0.53 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"S\", 9)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Sdag\", 1)"] 0.35 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Sdag\", 5)"] 0.51 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Sdag\", 9)"] 0.85 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"T\", 1)"] 0.44 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"T\", 13)"] 1.07 (5%) ❌ 1.00 (1%)
["specialized", "single qubit", "(\"T\", 5)"] 0.49 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"T\", 9)"] 0.90 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Tdag\", 1)"] 0.41 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Tdag\", 5)"] 0.52 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Tdag\", 9)"] 0.86 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"X\", 1)"] 0.30 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"X\", 13)"] 0.85 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"X\", 17)"] 0.89 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"X\", 5)"] 0.49 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"X\", 9)"] 0.75 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Y\", 1)"] 0.38 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Y\", 21)"] 1.06 (5%) ❌ 1.00 (1%)
["specialized", "single qubit", "(\"Y\", 5)"] 0.64 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Z\", 1)"] 0.42 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Z\", 5)"] 0.53 (5%) ✅ 1.00 (1%)
["specialized", "single qubit", "(\"Z\", 9)"] 0.82 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["matrices", "contiguous", "ordered"]
  • ["matrices", "contiguous", "random"]
  • ["matrices", "controlled", "ordered"]
  • ["matrices", "controlled", "random"]
  • ["matrices", "in-contiguous", "ordered"]
  • ["matrices", "in-contiguous", "random"]
  • ["matrices", "single qubit", "ordered"]
  • ["matrices", "single qubit", "random"]
  • ["specialized", "multi control"]
  • ["specialized", "multi qubit multi control"]
  • ["specialized", "multi qubit"]
  • ["specialized", "single control"]
  • ["specialized", "single qubit multi control"]
  • ["specialized", "single qubit"]

Julia versioninfo

Target

Julia Version 1.1.1
Commit 55e36cc308* (2019-05-16 04:10 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  uname: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64 i386
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz: 
              speed         user         nice          sys         idle          irq
       #1  2800 MHz     602190 s          0 s     539332 s   11151478 s          0 s
       #2  2800 MHz      31882 s          0 s      47199 s   12213079 s          0 s
       #3  2800 MHz     516505 s          0 s     311461 s   11464201 s          0 s
       #4  2800 MHz      29555 s          0 s      39714 s   12222882 s          0 s
       #5  2800 MHz     409611 s          0 s     230443 s   11652111 s          0 s
       #6  2800 MHz      28798 s          0 s      35218 s   12228133 s          0 s
       #7  2800 MHz     349740 s          0 s     182511 s   11759914 s          0 s
       #8  2800 MHz      28493 s          0 s      30983 s   12232673 s          0 s
       
  Memory: 16.0 GB (816.95703125 MB free)
  Uptime: 1.531546e6 sec
  Load Avg:  1.52001953125  1.798828125  1.80078125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.1.1
Commit 55e36cc308* (2019-05-16 04:10 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  uname: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64 i386
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz: 
              speed         user         nice          sys         idle          irq
       #1  2800 MHz     605584 s          0 s     540024 s   11161950 s          0 s
       #2  2800 MHz      31936 s          0 s      47263 s   12227518 s          0 s
       #3  2800 MHz     520315 s          0 s     312068 s   11474341 s          0 s
       #4  2800 MHz      29610 s          0 s      39770 s   12237328 s          0 s
       #5  2800 MHz     413196 s          0 s     230979 s   11662548 s          0 s
       #6  2800 MHz      28852 s          0 s      35267 s   12242588 s          0 s
       #7  2800 MHz     353358 s          0 s     183018 s   11770346 s          0 s
       #8  2800 MHz      28547 s          0 s      31025 s   12247134 s          0 s
       
  Memory: 16.0 GB (937.27734375 MB free)
  Uptime: 1.533002e6 sec
  Load Avg:  2.2109375  1.84521484375  1.7314453125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

@Roger-luo
Copy link
Member

@GiggleLiu it seems that there are some regressions, do you have time to investigate them?

src/instruct.jl Outdated
end
operator = sort_unitary(operator, locs)
locs_raw, ic = _prepare_instruct(state, operator, locs, control_locs, control_bits)
return _instruct!(state, autostatic(operator), locs_raw, ic)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to transpose the operator and state for general case in order to make memory contiguous for BLAS?

Copy link
Member Author

@GiggleLiu GiggleLiu Sep 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state is transposed, transposing the operator is not nessesary. After all, we don't have BLAS...

@Roger-luo
Copy link
Member

Roger-luo commented Sep 18, 2019

["matrices", "contiguous", "ordered", "(1, \"Complex{Float64}\", \"PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}\")"]

it seems there are some performance regression against PermMatrix? I'm not sure why however. And this one also doesn't look like noise

["specialized", "single qubit multi control", "(\"T\", 24, 8)"]

can you also run the benchmark on your laptop? some of the regression might just be noise.

@Roger-luo
Copy link
Member

batched register benchmarks are not defined yet, I'll do that later, meanwhile you can investigate the regression on single batch.

@GiggleLiu
Copy link
Member Author

On Delta 102, the 1.5 time regresion looks not true

# master branch
julia> results = run(SUITE["matrices"]["in-contiguous"]["ordered"][(1, "Complex{Float64}", "PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}")])
BenchmarkTools.Trial: 
  memory estimate:  288 bytes
  allocs estimate:  4
  --------------
  minimum time:     1.310 μs (0.00% GC)
  median time:      1.740 μs (0.00% GC)
  mean time:        1.622 μs (0.00% GC)
  maximum time:     17.520 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1

# this branch
julia> results = run(SUITE["matrices"]["in-contiguous"]["ordered"][(1, "Complex{Float64}", "PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}")])
BenchmarkTools.Trial: 
  memory estimate:  288 bytes
  allocs estimate:  4
  --------------
  minimum time:     1.135 μs (0.00% GC)
  median time:      1.188 μs (0.00% GC)
  mean time:        13.748 μs (0.00% GC)
  maximum time:     125.430 ms (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1

else
raw = zeros(T, nbatch, 1<<total)
raw[:,Int(bit_config)+1] .= 1
raw = transpose(raw)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should copy here? since transpose is lazy.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and state will always return a transposeed version so state(r) * mat still works in YaoBlocks? and add a new function storage to return the raw storage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need Tranpose wrapper here to create a effective row major array.

@Roger-luo
Copy link
Member

This PR currently breaks YaoBlocks:

Test Summary:                   | Pass  Error  Total
test composite block            |  258      7    265
  test chain                    |   62            62
  test kron                     |   44            44
  test control                  |   28            28
  test put                      |   26      7     33
    construct                   |    1             1
    apply!                      |    7      5     12
    test swap gate              |   17            17
    rotation gate               |    1      2      3
  test repeated                 |    7             7
  test concentrate              |   11            11
  test tag                      |   59            59
  test pauli string             |   15            15
  test single block chsubblocks |    4             4
ERROR: LoadError: Some tests did not pass: 258 passed, 0 failed, 7 errored, 0 broken.
in expression starting at /Users/roger/.julia/dev/YaoBlocks/test/runtests.jl:7

@GiggleLiu do you have time to fix it?

@GiggleLiu
Copy link
Member Author

reverted @Roger-luo

@Roger-luo Roger-luo merged commit cf2c9c6 into master Oct 10, 2019
@Roger-luo Roger-luo deleted the transpose_storage branch October 10, 2019 13:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants