[unitaryhack] Add SPSA optimization

[ixime]

Context:
Adding a feature as part of unitaryhack related to issue #2451

Description of the Change:
The qml.SPSAOptimizer was added with its corresponding tests based on:

J. Spall, "An Overview of the Simultaneous Perturbation Method for Efficient Optimization." Johns Hopkins api technical digest, 19(4), 482-492, 1998

Benefits:

• The Simultaneous Perturbation Stochastic Approximation method (SPSA) is an iterative algorithm for optimization ideal for noisy loss functions.
• In contrast to other methods that perform multiple operations to determine the gradient, SPSA only measures two times the loss function to obtain it.

Possible Drawbacks:

Related GitHub Issues:
Closes #2451

[ixime]

I think I finished everything, but I am not sure about the cases of hybrid classical-quantum workflows. as I commented in #2451

[codecov]

Codecov Report

Merging #2661 (a016cd1) into master (57f20b7) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #2661   +/-   ##
=======================================
  Coverage   99.61%   99.61%           
=======================================
  Files         251      252    +1     
  Lines       20662    20726   +64     
=======================================
+ Hits        20582    20646   +64     
  Misses         80       80           

Impacted Files	Coverage Δ
pennylane/optimize/__init__.py	100.00% <100.00%> (ø)
pennylane/optimize/spsa.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57f20b7...a016cd1. Read the comment docs.

[MoritzWillmann]

Did you compare the optimizer to qiskit's ? I only compared it to noisyopt's implementation

Not yet, but I can do it smile

Hey @MoritzWillmann, please do this comparison too. I'm using both noisyopt and Qiskit currently and do not want any surprises when I switch too this implementation! 😆 😅

I yield similar results for a couple of different functions. I think you're good to go 😆

[antalszava]

Hi @ixime and @MoritzWillmann! This is looking amazing overall. 👏 🎆

The suggestions I have this time are smaller. Once they are addressed, I'll make another pass, but the overall addition here is very near approval.

Note: some conflicts will have to be resolved when merging master in.

[ixime]

I tried to solve the conflicts, but I don't have write access to solve them. My fork master is up to date and apparently I merged successfully my local branch with my up to date local master 🤷‍♀️ I am little lost here.. did I miss some step?

[josh146]

@ixime It should be fixed now :)

[ixime]

Hi @antalszava why some times the codecov report is fine and the next check is all wrong without changing a bit on the code, just the docstring?

[antalszava]

Looks good @ixime, @MoritzWillmann! 🎉 🎆 This is an amazing and high-quality addition, congrats 🙂 👏

Adding the Unitary HACK accepted label too.

Hi @antalszava why some times the codecov report is fine and the next check is all wrong without changing a bit on the code, just the docstring?

I think the issue came from a previous state where one test was failing and it didn't report to Codecov correctly. Now it should be okay 👍