Quantum dropout demo

[fran-scala]

Hi! This demo request refers to Issue #4929 for Pennylane, about implementing dropout for Quantum Neural Networks directly in Pennylane.

Title:

Dropout for Quantum Neural Networks

Summary:

In this demo we show how to exploit the quantum version of dropout technique to avoid the problem of
overfitting in deep Quantum Neural Networks (QNNs). What follows is based on the paper
“A General Approach to Dropout in Quantum Neural Networks” by F. Scala, et al.

Relevant references:

• [1] Scala, F., Ceschini, A., Panella, M., & Gerace, D. (2023). A General Approach to Dropout in Quantum Neural Networks. Adv. Quantum Technol., 2300220
• [2] Kiani,B. T., Lloyd, S., & Maity, R. (2020).Learning Unitaries by Gradient Descent. arXiv: 2001.11897
• [3] Larocca, M., Ju, N., García-Martín, D., Coles, P. J., & Cerezo, M. (2023). Theory of overparametrization in quantum neural networks. Nat. Comp. Science, 3, 542–551

Possible Drawbacks:

In this demo, to show the effectiveness of the technique, dropout is implemented by randomly setting some of the optimized parameters to 0 at each iteration. Actual dropout should be implemented by substituting a certain gate with the identity gate.

Related GitHub Issues:

Pennyalne Issue #4929

---

If you are writing a demonstration, please answer these questions to facilitate the marketing process.

• GOALS — Why are we working on this now?

We would like to implement dropout for QNNs directly in Pennylane, referring to paper [1]

• AUDIENCE — Who is this for?

This technique (hence this demo) is for all people interested in Quantum Machine Learning. Both researchers and enthusiasts may benefit by learning from this demo how to avoid overfitting when using overparametrized QNNs. We strongly believe that it will become a standard for QML just like its classical counterpart in ML.

• KEYWORDS — What words should be included in the marketing post?

"Quantum Neural Networks","QNN", "overfitting", "dropout", "regularization"

• Which of the following types of documentation is most similar to your file?
  (more details here)

• Tutorial
• Demo
• How-to

[github-actions]

Thank you for opening this pull request.

You can find the built site at this link.

Deployment Info:

• Pull Request ID: 1003
• Deployment SHA: df3fc57881a0de922d820f8afa39cc1df5f636ad
  (The Deployment SHA refers to the latest commit hash the docs were built from)

Note: It may take several minutes for updates to this pull request to be reflected on the deployed site.

[josh146]

thank you for this really nice demo contribution @fran-scala! The bot posted a link to the site preview above, but here is a direct link to the rendered demo: https://qml-build-previews.pennylane.ai/pull_request_build_preview/1003/qml/demos/tutorial_quantum_dropout/

I'll organize with the team for review on our end -- apologies, it might be a bit slow due to the christmas break!

[KetpuntoG]

A great demo! Thanks for working on it 😊
Here I leave you some comments.
Let me know if you have any questions

[josh146]

Hey @fran-scala! Just wondering if you saw the comments by @KetpuntoG :)

[fran-scala]

Hey @fran-scala! Just wondering if you saw the comments by @KetpuntoG :)

Hi @josh146! I saw the comments and I'm working on solving all the issues. This month was insane. I will commit the updated version ASAP.

[josh146]

No worries at all @fran-scala! Just wanted to check in :)

[fran-scala]

Hi @josh146 and @KetpuntoG ! I updated the demo. The only thing left to fix is how to entirely display the plot with training and test losses. The problem is that I'm putting the legend outside the box to not cover the plots. In my notebook, there is no display problem. It happens something similar when I try to save the image (but this can be solved by passing an extra parameter to the savefig function).

Let me know if you have any ideas on how to solve this and if the made changes are good.

[KetpuntoG]

Great job! I have seen that there are some rendering issues. I think with the extra "#" I've included, it should work :) Once the deployment works, I will contact the designer and marketing to proceed with publication. If you have a small sketch of how you would like the thumbnail, it would help us 😄

[fran-scala]

Great job! I have seen that there are some rendering issues. I think with the extra "#" I've included, it should work :) Once the deployment works, I will contact the designer and marketing to proceed with publication. If you have a small sketch of how you would like the thumbnail, it would help us 😄

Perfect! Many thanks for your help!

About the sketch, I'm not very good at drawing but I think that a good idea can be a circuit from which some gates literally "drop/jump out" from the circuit or, alternatively, something like the image here below (that is something I used at some conferences)

[KetpuntoG]

Great job! I have seen that there are some rendering issues. I think with the extra "#" I've included, it should work :) Once the deployment works, I will contact the designer and marketing to proceed with publication. If you have a small sketch of how you would like the thumbnail, it would help us 😄

Perfect! Many thanks for your help!

About the sketch, I'm not very good at drawing but I think that a good idea can be a circuit from which some gates literally "drop/jump out" from the circuit or, alternatively, something like the image here below (that is something I used at some conferences)

That helps, thanks! 🎨

[KetpuntoG]

Hey @fran-scala , just wanted to double check
Is this the expected output? or is the image cut?

[fran-scala]

Hey @fran-scala , just wanted to double check Is this the expected output? or is the image cut?

The output is now correct! But if you have any suggestions to make the plot clearer please tell me.

[KetpuntoG]

I found it different than normal to see the blue curve cut in the middle.
If you calculate it you could extend the y-axis up to maybe ~10^-4?
(if it will be too flat on the right side, ignore this comment)

[fran-scala]

I found it different than normal to see the blue curve cut in the middle. If you calculate it you could extend the y-axis up to maybe ~10^-4? (if it will be too flat on the right side, ignore this comment)

Yeah the point is that on the left side the blue curve goes to 0 extremely fast, and this leads to overfitting. But if we go to 10^-4 it would be difficult to see wll in the right panel. If it's not a big deal I would rather leave it like this.

By the way I think there are still rendering problems in the section "The circuit".

I saw the thumbnail, cool!

[KetpuntoG]

Hey ! for the publication it is possible that marketing may want to include your username in the post. Could you share with me your @username from Twitter (optional)?

[fran-scala]

Hey ! for the publication it is possible that marketing may want to include your username in the post. Could you share with me your @username from Twitter (optional)?

Unfortunately, I do not have a Twitter account but I would be happy to be tagged on LinkedIn (www.linkedin.com/in/fran-scala)

[KetpuntoG]

Great thanks! It will probably be visible on the web today but we will announce it in two weeks time

[KetpuntoG]

Great job 🚀

[fran-scala]

Great thanks! It will probably be visible on the web today but we will announce it in two weeks time

Fantastic! Thanks you all for the provided help and for what you do for the community!🤩

[KetpuntoG]

Hi @fran-scala ! Marketing has scheduled the post for March 28th :)

[fran-scala]

Hi @fran-scala ! Marketing has scheduled the post for March 28th :)

Sounds perfect! That day is my birthday so thanks a lot for the gift!🎉