Migrate lightning.qubit to the new API (#646)

* fix reset state

* Update pennylane_lightning/lightning_qubit/_state_vector.py

* Update pennylane_lightning/lightning_qubit/_state_vector.py

* Update pennylane_lightning/lightning_qubit/_state_vector.py

* remove LightningQubit2 references

* remove unnecessary modules

* merging Serializer classes

* update serialize tests

* update measurements with new serialize class

* remove outdated test

* register with setup.py, state vector fixes

* remove obsolete tests

* remove unused dtype input from simulate

* update measurements

* update state_vector

* update lightning_qubit2

* format

* pylint

* Update pennylane_lightning/lightning_qubit/_state_vector.py

* remove old comment

* some review suggestions

* Auto update version

* remove print

* update state vector class

* add state vector class tests

* adding measurement tests

* update state vector and tests

* move and rename test files, and format

* Auto update version

* skip measurements class for other devices and in the absence of binaries

* format

* add LightningQubit2 to init and format

* update measurements class

* expand measurement class testing

* garbage collection

* typo

* update coverage and StateVector class

* expand measurements class coverage

* Auto update version

* add coverage for n-controlled operations

* add map to standard wires to get_final_state for safety

* update jax config import

* Auto update version

* trigger CI

* update state vector class and tests for improved coverage

* update measurement class tests

* update dev version

* add cpp binary available variable

* remove device definition

* update dev version

* Auto update version

* reduce dependency on DefaultQubit for tests

* update LightningQubit2

* clean test_measurements_class.py

* isort+black

* review suggestion

* fix docs

* Add qml.var support.

* Add probs support.

* increase tolerance

* Auto update version

* isort

* Add double-obs tests.

* Pin pytest version (#624)

* update dev version

* update changelog

* pin pytest version in requirement files

* add a requirements file for tests against Pennylane master

* update wheels' workflows

* Version Bump (#626)

* post release version bump

* increase tolerance

* Introduce isort. (#623)

* Introduce isort.

* Auto update version

* Update changelog

* Auto update version

* Update changelog.

* trigger ci

* Auto update version

* isort

* Add qml.var support.

* Add probs support.

* Add measurement tests with wires.

* review suggestions

* remove unused imports

* Introduce _new_API and fix/skip few tests.

* Fix few more tests.

* Skip shots, adjoint, vjp with new API.

* remove diagonalization gate application from state vector

* pytest.skip tests

* Auto update version

* Fix format

* Fix no-bin interface.

* WIP

* Initial shots support + fix test_measurement tests.

* update

* adding tests from add-simulate branch

* merge conflicts

* create state vector on initialization

* remove import of modifier from lightning

* Update pennylane_lightning/lightning_qubit/lightning_qubit2.py

* minor test updates

* register with setup.py, state vector fixes

* add LightningQubit2 to init and format

* add cpp binary available variable

* reduce dependency on DefaultQubit for tests

* update LightningQubit2

* Fixing rebase artifacts

* Add fewLQ2 tests.

* remove adjoint diff support from supports derivatives

* Remove print from test_apply

* Add expval/var tests.

* Remove duplicate class data.

* Include LQ2 in linux tests.

* Add _group_measurements support.

* --cov-append

* Add mcmc capability + tests.

* Auto update version

* update dev version

* add LightningAdjointJacobian class

* add unit tests for the LightningAdjointJacobian class

* format

* add changelog for PR #613

* [skip ci] Added skeleton file for LQ2 unit tests

* update changelog

* update adjoint Jacobian

* Auto update version

* codefactor

* Add shots tests and fix bugs in LQ, LQ2.

* Lightning qubit2 upgrade api (#628)

* update

* adding tests from add-simulate branch

* merge conflicts

* create state vector on initialization

* remove import of modifier from lightning

* Update pennylane_lightning/lightning_qubit/lightning_qubit2.py

* minor test updates

* register with setup.py, state vector fixes

* add LightningQubit2 to init and format

* add cpp binary available variable

* Auto update version

* reduce dependency on DefaultQubit for tests

* update LightningQubit2

* Introduce _new_API and fix/skip few tests.

* Fix few more tests.

* Skip shots, adjoint, vjp with new API.

* Fix no-bin interface.

* Remove duplicate class data.

* Include LQ2 in linux ests.

* --cov-append

* fix processing_fn_expval

* make a proper new_tape

* Added init tests; Added skeleton tests for helpers

* Fix more bug with shots.

* trigger CI

* Change pennylane branch for CI.

* Update .github/CHANGELOG.md

* Update pennylane_lightning/lightning_qubit/_adjoint_jacobian.py

* Update pennylane_lightning/lightning_qubit/_adjoint_jacobian.py

* Add probs support.

* Add double-obs tests.

* Add qml.var support.

* Add probs support.

* Add measurement tests with wires.

* pytest.skip tests

* Fix format

* update

* adding tests from add-simulate branch

* merge conflicts

* create state vector on initialization

* remove import of modifier from lightning

* Update pennylane_lightning/lightning_qubit/lightning_qubit2.py

* minor test updates

* register with setup.py, state vector fixes

* add LightningQubit2 to init and format

* add cpp binary available variable

* reduce dependency on DefaultQubit for tests

* update LightningQubit2

* Fixing rebase artifacts

* remove adjoint diff support from supports derivatives

* [skip ci] Added skeleton file for LQ2 unit tests

* Lightning qubit2 upgrade api (#628)

* update

* adding tests from add-simulate branch

* merge conflicts

* create state vector on initialization

* remove import of modifier from lightning

* Update pennylane_lightning/lightning_qubit/lightning_qubit2.py

* minor test updates

* register with setup.py, state vector fixes

* add LightningQubit2 to init and format

* add cpp binary available variable

* Auto update version

* reduce dependency on DefaultQubit for tests

* update LightningQubit2

* Introduce _new_API and fix/skip few tests.

* Fix few more tests.

* Skip shots, adjoint, vjp with new API.

* Fix no-bin interface.

* Remove duplicate class data.

* Include LQ2 in linux ests.

* --cov-append

* Added init tests; Added skeleton tests for helpers

* Resolving rebase artifacts

* Refactor shots test.

* Added tests; integrated jacobian

* Update pennylane_lightning/lightning_qubit/lightning_qubit2.py

* Auto update version

* Small update to simulate_and_jacobian

* Auto update version

* Rerun isort.

* Uncomment integration tests.

* Reformat

* Delete symlink

* Fix pylint.

* Run linux tests in parallel (when possible).

* Run double obs tests with shots.

* Revert linux tests

* Fix bg in diag_gates.

* Call isort/black with python -m

* update dev version

* Add docstrings, rm C_DTYPE.

* Auto update version

* comment isort check

* trigger ci

* Update tests/test_expval.py

* Init mcmc params to None in measurements.

* Reformat with python3.11

* Reformat black

* Auto update version

* update QuantumScriptSerializer

* remove LightningQubit2 from init

* update setup.py

* remove lightning.qubit2 from tests configuration

* remove extra tests for lightning.qubit2

* migrate lightning.qubit2 to lightning.qubit on tests

* make lightning.qubit2 the new lightning.qubit

* add device name (necessary for pl-device-test)

* Add _measure_hamiltonian_with_samples _measure_sum_with_samples

* fix tests without binary

* check for jac size before reshaping

* remove obsolete tests

* organize tests

* fix test for Windows wheels

* fix the fix for LightningKokkos

* undo 'fix'

* trigger CI

* update changelog

* Auto update version

* commenting tests for Win wheels

* trigger build wheels

.github/CHANGELOG.md

@@ -23,6 +23,9 @@
 
 ### Breaking changes
 
+* Migrate `lightning.qubit` to the new device API.
+  [(#646)](https://github.com/PennyLaneAI/pennylane-lightning/pull/646)
+
 * Introduce `ci:build_wheels` label, which controls wheel building on `pull_request` and other triggers.
   [(#648)](https://github.com/PennyLaneAI/pennylane-lightning/pull/648)
 

.github/workflows/tests_linux.yml

@@ -185,7 +185,6 @@ jobs:
           cd main/
           DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"`
           PL_DEVICE=${DEVICENAME} python -m pytest tests/ $COVERAGE_FLAGS
-          PL_DEVICE=lightning_qubit2 python -m pytest tests/ $COVERAGE_FLAGS --cov-append
           pl-device-test --device ${DEVICENAME} --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append
           pl-device-test --device ${DEVICENAME} --shots=None --skip-ops $COVERAGE_FLAGS --cov-append
           mv .coverage .coverage-${{ github.job }}-${{ matrix.pl_backend }}

.github/workflows/tests_without_binary.yml

@@ -107,9 +107,6 @@ jobs:
           cd main/
           DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"`
           PL_DEVICE=${DEVICENAME} python -m pytest tests/ $COVERAGE_FLAGS
-          if [ ${{ matrix.pl_backend }} == "lightning_qubit" ]; then PL_DEVICE=lightning_qubit2 python -m pytest tests/ $COVERAGE_FLAGS --cov-append; fi
-          pl-device-test --device ${DEVICENAME} --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append
-          pl-device-test --device ${DEVICENAME} --shots=None --skip-ops $COVERAGE_FLAGS --cov-append
 
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v3

.github/workflows/wheel_win_x86_64.yml

@@ -127,11 +127,12 @@ jobs:
           CIBW_BEFORE_BUILD: |
             python -m pip install pybind11 cmake~=3.24.0 build
 
-          CIBW_BEFORE_TEST: |
-            python -m pip install -r requirements-tests.txt
+          #Temporarily commenting while solving problems to find binaries in CIBW tests.
+          # CIBW_BEFORE_TEST: |
+          #   python -m pip install -r requirements-tests.txt
 
-          CIBW_TEST_COMMAND: |
-            pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report
+          # CIBW_TEST_COMMAND: |
+          #   pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report
 
           CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
 

pennylane_lightning/core/_serialize.py

@@ -62,7 +62,7 @@ def __init__(
         self.use_csingle = use_csingle
         self.device_name = device_name
         self.split_obs = split_obs
-        if device_name in ("lightning.qubit", "lightning.qubit2"):
+        if device_name == "lightning.qubit":
             try:
                 import pennylane_lightning.lightning_qubit_ops as lightning_ops
             except ImportError as exception:

pennylane_lightning/core/_version.py

@@ -16,4 +16,4 @@
    Version number (major.minor.patch[-label])
 """
 
-__version__ = "0.36.0-dev13"
+__version__ = "0.36.0-dev14"

pennylane_lightning/lightning_qubit/__init__.py

@@ -16,4 +16,3 @@
 from pennylane_lightning.core import __version__
 
 from .lightning_qubit import LightningQubit
-from .lightning_qubit2 import LightningQubit2

pennylane_lightning/lightning_qubit/_adjoint_jacobian.py

@@ -243,7 +243,7 @@ def calculate_jacobian(self, tape: QuantumTape):
                 trainable_params,
             )
         jac = np.array(jac)
-        jac = jac.reshape(-1, len(trainable_params))
+        jac = jac.reshape(-1, len(trainable_params)) if len(jac) else jac
         jac_r = np.zeros((jac.shape[0], processed_data["all_params"]))
         jac_r[:, processed_data["record_tp_rows"]] = jac
 

pennylane_lightning/lightning_qubit/_measurements.py

@@ -320,18 +320,21 @@ def measure_with_samples(
             if isinstance(group[0], (ExpectationMP, VarianceMP)) and isinstance(
                 group[0].obs, SparseHamiltonian
             ):
-                raise TypeError("ExpectationMP(SparseHamiltonian) cannot be computed with samples.")
-            if isinstance(group[0], (ExpectationMP, VarianceMP)) and isinstance(
-                group[0].obs, Hamiltonian
-            ):
-                raise TypeError("ExpectationMP(Hamiltonian) cannot be computed with samples.")
-            if isinstance(group[0], (ExpectationMP, VarianceMP)) and isinstance(group[0].obs, Sum):
-                raise TypeError("ExpectationMP(Sum) cannot be computed with samples.")
+                raise TypeError(
+                    "ExpectationMP/VarianceMP(SparseHamiltonian) cannot be computed with samples."
+                )
+            if isinstance(group[0], VarianceMP) and isinstance(group[0].obs, (Hamiltonian, Sum)):
+                raise TypeError("VarianceMP(Hamiltonian/Sum) cannot be computed with samples.")
             if isinstance(group[0], (ClassicalShadowMP, ShadowExpvalMP)):
                 raise TypeError(
                     "ExpectationMP(ClassicalShadowMP, ShadowExpvalMP) cannot be computed with samples."
                 )
-            all_res.extend(self._measure_with_samples_diagonalizing_gates(group, shots))
+            if isinstance(group[0], ExpectationMP) and isinstance(group[0].obs, Hamiltonian):
+                all_res.extend(self._measure_hamiltonian_with_samples(group, shots))
+            elif isinstance(group[0], ExpectationMP) and isinstance(group[0].obs, Sum):
+                all_res.extend(self._measure_sum_with_samples(group, shots))
+            else:
+                all_res.extend(self._measure_with_samples_diagonalizing_gates(group, shots))
 
         # reorder results
         flat_indices = []
@@ -438,3 +441,43 @@ def _process_single_shot(samples):
         self._apply_diagonalizing_gates(mps, adjoint=True)
 
         return _process_single_shot(samples)
+
+    def _measure_hamiltonian_with_samples(
+        self,
+        mp: List[SampleMeasurement],
+        shots: Shots,
+    ):
+        # the list contains only one element based on how we group measurements
+        mp = mp[0]
+
+        # if the measurement process involves a Hamiltonian, measure each
+        # of the terms separately and sum
+        def _sum_for_single_shot(s):
+            results = self.measure_with_samples(
+                [ExpectationMP(t) for t in mp.obs.terms()[1]],
+                s,
+            )
+            return sum(c * res for c, res in zip(mp.obs.terms()[0], results))
+
+        unsqueezed_results = tuple(_sum_for_single_shot(type(shots)(s)) for s in shots)
+        return [unsqueezed_results] if shots.has_partitioned_shots else [unsqueezed_results[0]]
+
+    def _measure_sum_with_samples(
+        self,
+        mp: List[SampleMeasurement],
+        shots: Shots,
+    ):
+        # the list contains only one element based on how we group measurements
+        mp = mp[0]
+
+        # if the measurement process involves a Sum, measure each
+        # of the terms separately and sum
+        def _sum_for_single_shot(s):
+            results = self.measure_with_samples(
+                [ExpectationMP(t) for t in mp.obs],
+                s,
+            )
+            return sum(results)
+
+        unsqueezed_results = tuple(_sum_for_single_shot(type(shots)(s)) for s in shots)
+        return [unsqueezed_results] if shots.has_partitioned_shots else [unsqueezed_results[0]]