Every mainstream language has a native test framework — JUnit for Java, PyTest for Python — because developers expect to test code in the language they write it in. OpenQASM has had no such tool. The workaround today is to wrap your .qasm files with Python glue: load the circuit with Qiskit, configure a backend, run it, pull the counts back into Python, and write assertions there. The result is a test suite where the code under test lives in one language and the tests in another, coupled to a specific SDK, impossible to read without knowing both, and fragile against API changes in the classical layer that have nothing to do with the quantum algorithm.
QUTest eliminates that gap. Test cases are written as def test*() functions directly in .qasm files, side by side with the circuits they verify. Backend configuration and assertions are expressed as pragma comments (//%), so every file remains valid OpenQASM 3 syntax and non-QUTest tools can ignore the test metadata as ordinary comments. Executing the file unchanged still depends on the target toolchain's OpenQASM 3 feature support; QUTest currently inlines supported def subroutines before handing circuits to Qiskit. A quantum algorithm author can specify exactly what correctness means — output values, probability distributions, entanglement, entropy, gate budgets — without writing a single line of Python. The tests are portable across SDKs, self-documenting, and runnable in CI with one command.
pip install qutestWrite a test file (test_bell.qasm):
OPENQASM 3;
include "stdgates.inc";
// Software under test
def bell(qubit[2] q) {
h q[0];
cx q[0], q[1];
}
def test_bell_distribution() {
//
// Backend constraints
//
//% shots 4096
//% seed 42
//% backend ideal
//
// Arrange
//
qubit[2] q;
bit[2] m;
//
// Act
//
bell(q);
m = measure q;
//
// Assert
//
//% expect distribution ref {"00": 0.5, "11": 0.5}
//% expect distribution total_variation_distance <= 0.05
//% expect marginal m[0] p1 ~= 0.5 atol=0.05
//% expect marginal m[1] p1 ~= 0.5 atol=0.05
}Run it:
qutest run test_bell.qasmOutput:
Discovered 1 test(s)
✓ test_bell.qasm::test_bell_distribution
==================================================
1 passed in 0.21s (1 total)
- Discovery — qutest recursively scans for
.qasmfiles and identifies functions whose name starts withtest(e.g.def test_bell()). - Parsing — Each test function is extracted. Helper / SUT functions are automatically inlined at call sites. Pragma comments (
//% ...) are parsed into configuration and assertions. - Execution — The resulting OpenQASM source is loaded into a Qiskit
QuantumCircuit, transpiled for the configured backend, and executed with the specified number of shots. - Evaluation — Structural assertions are checked on the transpiled circuit; result assertions are checked against the measurement counts.
- Reporting — Results are printed to the console (with color). Optionally, a QUnit XML report can be generated for CI integration.
qutest lint [PATH] [OPTIONS]
qutest run [PATH] [OPTIONS]
qutest lint checks .qasm files without executing circuits:
| Option | Description |
|---|---|
PATH |
A .qasm file or directory to scan (default: .) |
--strict |
Exit non-zero when warnings are found, not only errors |
The linter validates that //% directives are inside def test*() functions,
checks directive syntax, reports unsupported or misspelled assertion forms,
flags missing or misplaced distribution references, validates probability ranges
and bitstring literals, checks that directives reference measured classical
registers, and warns about likely mistakes such as duplicate configuration
directives or tests with no assertion oracle.
Example:
qutest lint test_bell.qasmOutput:
Checked 1 file(s), 2 test(s), 15 directive(s): 0 error(s), 0 warning(s)
qutest run discovers and executes tests:
| Option | Description |
|---|---|
PATH |
A .qasm file or directory to scan (default: .) |
--shots N |
Override the number of shots for all tests |
--backend ideal|noisy |
Override the backend for all tests |
--seed N |
Override the simulator seed for all tests |
--qunit-xml PATH |
Write a QUnit XML report to the given file |
--lint |
Lint QUTest directives before running; abort if errors are found |
-v, --verbose |
Show all assertion details, not just failures |
--version |
Print the version and exit |
Pragmas are special comments prefixed with //%. They are placed inside def test*() functions.
These pragmas configure how the quantum circuit is executed. They are optional; defaults are shown below.
| Pragma | Default | Description |
|---|---|---|
//% shots <N> |
1024 |
Number of measurement shots. |
//% seed <N> |
(none) | Simulator seed for reproducible results. |
//% backend ideal|noisy|hardware |
ideal |
Execution backend. ideal uses a noiseless statevector-based simulator. noisy uses a depolarizing noise model. hardware is reserved for future IBM Quantum support. |
Example:
def test_my_circuit() {
//% shots 8192
//% seed 42
//% backend ideal
// ...
}Assert pragmas define the pass/fail criteria for a test. They are evaluated after the circuit is executed (result asserts) or after transpilation (structural asserts).
Checked on the transpiled circuit, before execution. Useful for verifying that the circuit meets hardware constraints or resource budgets.
| Pragma | Description |
|---|---|
//% expect gateset subset_of [<gate>, ...] |
All gates in the transpiled circuit must be in the given set. measure and barrier are excluded from the check. |
//% expect depth <op> <N> |
Assert the circuit depth. <op> is a comparison: <=, >=, ==, <, >, !=. |
Example:
def test_efficient_bell() {
//% expect gateset subset_of [h, cx]
//% expect depth <= 5
qubit[2] q;
bit[2] m;
bell(q);
m = measure q;
}Checks that the classical measurement outcome is a deterministic value across all shots.
| Pragma | Description |
|---|---|
//% expect output <variable> == <value> |
All shots must produce the given bitstring for the named register. Also supports !=. |
Example:
def test_x_gate() {
//% shots 100
qubit q;
bit m;
x q;
m = measure q;
//% expect output m == 1
}Compare the measured probability distribution against a reference distribution using a statistical distance metric.
| Pragma | Description |
|---|---|
//% expect distribution ref {<json>} |
Set the reference (ideal) distribution. Keys are bitstrings, values are probabilities. Must appear before any metric assert. |
//% expect distribution total_variation_distance <op> <val> |
Total variation distance (TVD) between measured and reference. Range: 0 (identical) to 1 (orthogonal). |
//% expect distribution hellinger <op> <val> |
Hellinger distance. Range: 0 to 1. |
//% expect distribution kl <op> <val> |
Kullback-Leibler divergence D_KL(measured || reference). Range: 0 to ∞. |
//% expect distribution chi2 <op> <val> |
Pearson chi-square goodness-of-fit p-value over raw counts. Use //% expect distribution chi2 >= 0.05 for a 5% significance threshold, matching scipy.stats.chisquare(...).pvalue >= 0.05. |
Example:
def test_bell_state() {
//% shots 4096
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect distribution ref {"00": 0.5, "11": 0.5}
//% expect distribution total_variation_distance <= 0.05
//% expect distribution hellinger <= 0.05
}Check the marginal probability of a single qubit (one bit of the output register).
| Pragma | Description |
|---|---|
//% expect marginal <reg>[<i>] p<outcome> ~= <val> atol=<tol> |
Assert that the probability of qubit i in register reg being <outcome> (0 or 1) is approximately <val>, within absolute tolerance <tol>. Default atol is 0.01. |
Example:
def test_hadamard() {
//% shots 4096
qubit q;
bit m;
h q;
m = measure q;
//% expect marginal m[0] p0 ~= 0.5 atol=0.05
//% expect marginal m[0] p1 ~= 0.5 atol=0.05
}Compute the expectation value of a Pauli-Z observable from measurement counts.
| Pragma | Description |
|---|---|
//% expect observable "<pauli_string>" ~= <val> atol=<tol> |
Assert the expectation value of the given Pauli string. Currently supports Z-basis observables only (e.g. "Z0", "Z0 Z1", "Z0 Z2"). Each bitstring contributes (-1)^parity where parity is the sum of the measured bits at the specified qubit positions. Default atol is 0.01. |
Example:
def test_ghz_correlations() {
//% shots 8192
qubit[3] q;
bit[3] m;
h q[0];
cx q[0], q[1];
cx q[0], q[2];
m = measure q;
//% expect observable "Z0 Z1" ~= 1.0 atol=0.05
//% expect observable "Z0 Z2" ~= 1.0 atol=0.05
}Assert the Shannon entropy (in bits) of the output distribution. Useful for verifying that a circuit produces the expected amount of randomness.
| Pragma | Description |
|---|---|
//% expect entropy ~= <val> atol=<tol> |
Assert entropy is approximately <val> bits. Default atol is 0.01. |
//% expect entropy <op> <val> |
Assert entropy with a comparison (<=, >=, ==, <, >, !=). |
A Bell state has entropy = 1 bit (two equally likely outcomes). A deterministic circuit has entropy = 0. A uniform n-qubit distribution has entropy = n.
Example:
def test_bell_entropy() {
//% shots 4096
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect entropy ~= 1.0 atol=0.05
}Assert the Pearson correlation coefficient between two output bits. Useful for verifying entanglement-induced classical correlations.
| Pragma | Description |
|---|---|
//% expect correlation <reg_a>[<i>] <reg_b>[<j>] ~= <val> atol=<tol> |
Assert the correlation between bit i of register reg_a and bit j of register reg_b. Range: -1 (anti-correlated) to +1 (perfectly correlated). Default atol is 0.01. |
Example:
def test_bell_correlation() {
//% shots 4096
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect correlation m[0] m[1] ~= 1.0 atol=0.05
}Assert the probability of a specific bitstring in the output distribution. Simpler than a full distribution reference when you only care about one or two outcomes.
| Pragma | Description |
|---|---|
//% expect probability "<bitstring>" ~= <val> atol=<tol> |
Assert that the probability of observing <bitstring> is approximately <val>. Default atol is 0.01. |
Example:
def test_bell_probability() {
//% shots 4096
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect probability "00" ~= 0.5 atol=0.05
//% expect probability "11" ~= 0.5 atol=0.05
//% expect probability "01" ~= 0.0 atol=0.01
}Check that the most frequently observed bitstring equals an expected value. No statistical test is applied — this is a simple argmax over the measurement counts. Useful for quick sanity checks on circuits with a clearly dominant outcome.
| Pragma | Description |
|---|---|
//% expect most_frequent "<bitstring>" |
Assert that the bitstring with the highest count equals <bitstring>. |
Example:
def test_bell_most_frequent() {
//% shots 4096
//% seed 42
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect most_frequent "00"
}Assert the classical fidelity between the measured distribution and an ideal noiseless simulation. The framework automatically runs the same circuit on a noiseless simulator and computes the Bhattacharyya coefficient squared: F = (Σ√(pᵢqᵢ))². Most useful when the test backend is noisy.
| Pragma | Description |
|---|---|
//% expect fidelity <op> <val> |
Assert classical fidelity against an ideal simulation. Range: 0 (orthogonal) to 1 (identical). |
Example:
def test_bell_noisy_fidelity() {
//% shots 4096
//% backend noisy
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect fidelity >= 0.95
}Verify that two qubit partitions are entangled by computing the von Neumann entropy of the reduced density matrix. Requires backend ideal (statevector access).
| Pragma | Description |
|---|---|
//% expect entangled [<qubits_a>] [<qubits_b>] |
Assert that partition A and partition B are entangled. Entanglement is confirmed when S(ρ_A) > 0. |
Example:
def test_bell_entanglement() {
//% backend ideal
qubit[2] q;
bit[2] m;
h q[0];
cx q[0], q[1];
m = measure q;
//% expect entangled [0] [1]
}Test files are standard .qasm files. qutest discovers them by scanning for any function named test*.
A test function follows the Arrange / Act / Assert pattern:
OPENQASM 3;
include "stdgates.inc";
// Software under test (will be inlined into each test)
def my_algorithm(qubit[2] q) {
h q[0];
cx q[0], q[1];
}
def test_my_algorithm() {
//
// Backend constraints
//
//% shots 4096
//% seed 42
//% backend ideal
//% expect depth <= 10
//
// Arrange
//
qubit[2] q;
bit[2] m;
//
// Act
//
my_algorithm(q);
m = measure q;
//
// Assert
//
//% expect distribution ref {"00": 0.5, "11": 0.5}
//% expect distribution total_variation_distance <= 0.05
//% expect marginal m[0] p1 ~= 0.5 atol=0.05
//% expect observable "Z0 Z1" ~= 1.0 atol=0.05
}A single .qasm file can contain multiple test functions. Each runs independently:
OPENQASM 3;
include "stdgates.inc";
def sut(qubit[2] q) {
h q[0];
cx q[0], q[1];
}
def test_distribution() {
//% shots 4096
qubit[2] q; bit[2] m;
sut(q); m = measure q;
//% expect distribution ref {"00": 0.5, "11": 0.5}
//% expect distribution total_variation_distance <= 0.05
}
def test_structure() {
//% expect depth <= 5
qubit[2] q; bit[2] m;
sut(q); m = measure q;
}