Benchmark data repository accompanying the paper: "Einsum Benchmark: Enabling the Development of Next-Generation Tensor Execution Engines". It includes a package to easily work with the benmark data and all methods we used to convert and create our instances so you can create new ones yourself.
Install our base package, you will need at least python 3.10:
pip install einsum_benchmarkNow you can load and run an instance like this:
import opt_einsum as oe
import einsum_benchmark
instance = einsum_benchmark.instances["qc_circuit_n49_m14_s9_e6_pEFGH_simplified"]
opt_size_path_meta = instance.paths.opt_size
print("Size optimized path")
print("log10[FLOPS]:", round(opt_size_path_meta.flops, 2))
print("log2[SIZE]:", round(opt_size_path_meta.size, 2))
result = oe.contract(
instance.format_string, *instance.tensors, optimize=opt_size_path_meta.path
)
print("sum[OUTPUT]:", result.sum(), instance.result_sum)For more information please read our docs.
The code for our experiments is here: https://github.com/ti2-group/einsum_benchmark_experiments
The broader data collection process included contributions from individuals whose data was transformed. We duly acknowledge the following for making their data publicly available:
- Fichte, Johannes; Hecher, Markus; Florim Hamiti: Model Counting Competition 2020
- Fichte, Johannes; Hecher, Markus: Model Counting Competition 2021 2022 2023
- Fichte, Johannes; Hecher, Markus; Woltran, Stefan; Zisser, Markus: A Benchmark Collection of #SAT Instances and Tree Decompositions
- Meel, Kuldeep S.: Model Counting and Uniform Sampling Instances
- Automated Reasoning Group at the University of California, Irvine: UAI Competitions
- Martinis, John M. et al.: Quantum supremacy using a programmable superconducting processor Dataset. Dryad.
Moreover, we thank the following authors of open source software used to generated instances: