The simulators need more testing

[ajavadia]

There is currently very little testing of simulators against expected behavior.

Each test will be in the form of an input Qobj (JSON) and the results should be checked against an expected output JSON. The input JSON will specify the circuit, as well as the config options on how to run it. Available config options are here.

Below are some things to test:

1- There are already some stuff in src/qiskit_simulator/test with input Qobj (JSON) and expected output Qobj (JSON). These are currently not used within the QISKit test framework, and should be.

2- Test local_clifford_simulator by providing circuits containing these gates: cx,id,x,y,z,h,s,sdg,wait,noise,save,load

3- Test local_qiskit_simulator by providing circuits containing these gates: u1,u2,u3,cx,id,x,y,z,h,s,sdg,t,tdg,wait,noise,save,load,uzz. (note: this simulator will undergo a namechange soon to become local_qasm_simulator).

4- Test local_statevector_simulator by providing configs with no shots, measure, noise, reset and same gate set as 3 (note: this simulator doesn't exist now, but it will soon)

4- Test circuits with 1, 2, 5 qubits.

5- Test circuits with "shots" : 1 and "shots" : 1024.

6- Test circuits with no measurement.
Test circuits with measurement only at the end.
Test circuits with measure in the middle + conditional if depending on that measure result.
Test circuits with measure in the middle but no subsequent if depending on the measurement result.

8- Test circuits with reset in the middle.

9- Test circuits containing a snapshot command (note: this currently does not exist but will be added soon for the local_statevector_simulator. Syntax similar to the save command in the tests mentioned in 1). Vary the data field of the config to get different outputs from the snapshot.

10 - Test noise by providing noise_params to the config.

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

• Version used:
• Environment name and version (e.g. Python 3.6.1):
• Operating System and version: