Skip to content

Consolidate samples, run notebooks in build#1070

Merged
swernli merged 11 commits into
mainfrom
swernli/sample-notebooks
Jan 31, 2024
Merged

Consolidate samples, run notebooks in build#1070
swernli merged 11 commits into
mainfrom
swernli/sample-notebooks

Conversation

@swernli
Copy link
Copy Markdown
Collaborator

@swernli swernli commented Jan 24, 2024

This change consolidates our samples under the top level samples folder by moving the three notebooks that were under pip/samples.

It also adds a new build step controlled by --notebooks that uses a virtual python environment to run the notebooks and verify the cells execute successfully. It does not verify the contents of the output, that can come in a later PR.

This change consolidates our samples under the top level samples folder by moving the three notebooks that were under pip/samples.

It also adds a new build step controlled by `--notebooks` that uses a virtual python environment to run the notebooks and verify the cells execute successfully. It does not verify the contents of the output, that can come in a later PR.
@swernli
Copy link
Copy Markdown
Collaborator Author

swernli commented Jan 24, 2024

This helps address part of #1023

@github-actions
Copy link
Copy Markdown

Benchmark for aff6ccf

Click to view benchmark
Test Base PR %
Array append evaluation 863.0±7.69µs 863.1±9.38µs +0.01%
Array update evaluation 934.8±7.75µs 933.9±11.95µs -0.10%
Deutsch-Jozsa evaluation 31.3±0.20ms 31.4±0.36ms +0.32%
Large file parity evaluation 36.0±0.97ms 35.9±0.11ms -0.28%
Large input file 38.6±2.84ms 38.0±1.42ms -1.55%
Standard library 21.1±0.46ms 21.4±0.64ms +1.42%
Teleport evaluation 104.0±5.33µs 103.1±2.14µs -0.87%

@github-actions
Copy link
Copy Markdown

Benchmark for 47d1e09

Click to view benchmark
Test Base PR %
Array append evaluation 860.5±7.01µs 878.6±25.04µs +2.10%
Array update evaluation 934.5±6.04µs 956.9±26.79µs +2.40%
Deutsch-Jozsa evaluation 31.4±0.29ms 31.5±0.93ms +0.32%
Large file parity evaluation 36.1±0.13ms 36.4±1.25ms +0.83%
Large input file 37.7±0.56ms 38.2±1.56ms +1.33%
Standard library 20.8±0.20ms 21.0±1.14ms +0.96%
Teleport evaluation 102.4±2.07µs 104.4±2.02µs +1.95%

@github-actions
Copy link
Copy Markdown

Benchmark for 38991dd

Click to view benchmark
Test Base PR %
Array append evaluation 862.5±9.49µs 876.4±7.34µs +1.61%
Array update evaluation 934.1±7.11µs 950.2±5.41µs +1.72%
Deutsch-Jozsa evaluation 31.6±0.34ms 31.9±1.17ms +0.95%
Large file parity evaluation 36.0±0.14ms 36.2±0.11ms +0.56%
Large input file 40.2±1.18ms 40.2±1.24ms 0.00%
Standard library 21.6±0.73ms 21.9±0.76ms +1.39%
Teleport evaluation 103.3±1.91µs 103.8±1.84µs +0.48%

Copy link
Copy Markdown
Member

@minestarks minestarks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some thoughts about structuring the build scripts.

Comment thread build.py Outdated
Comment thread build.py Outdated
Comment thread build.py Outdated
Comment thread build.py Outdated
Comment thread build.py Outdated
@github-actions
Copy link
Copy Markdown

Benchmark for 915431f

Click to view benchmark
Test Base PR %
Array append evaluation 875.5±5.84µs 879.0±18.86µs +0.40%
Array update evaluation 950.7±9.50µs 952.1±14.94µs +0.15%
Deutsch-Jozsa evaluation 31.5±0.20ms 31.6±0.32ms +0.32%
Large file parity evaluation 36.2±0.15ms 36.4±1.86ms +0.55%
Large input file 40.5±1.64ms 38.6±0.87ms -4.69%
Standard library 22.1±1.02ms 21.0±0.49ms -4.98%
Teleport evaluation 106.3±1.63µs 106.3±2.23µs 0.00%

@github-actions
Copy link
Copy Markdown

Benchmark for 0d5a596

Click to view benchmark
Test Base PR %
Array append evaluation 862.8±10.08µs 870.5±7.56µs +0.89%
Array update evaluation 939.5±12.77µs 944.2±14.43µs +0.50%
Deutsch-Jozsa evaluation 30.9±0.57ms 31.3±0.29ms +1.29%
Large file parity evaluation 35.6±0.49ms 36.1±0.53ms +1.40%
Large input file 37.5±0.73ms 37.7±0.98ms +0.53%
Standard library 20.6±0.47ms 20.8±0.51ms +0.97%
Teleport evaluation 102.2±2.21µs 103.7±2.48µs +1.47%

github-merge-queue Bot pushed a commit that referenced this pull request Jan 26, 2024
When a `%%qsharp` cell throws a compiler or runtime error, the whole
should be marked as an error, just as when Python code throws an
exception.

Related: #1070 . When we test notebooks in automation, we want to catch
any failures in Q# cells.

VS Code before/after side by side:


![image](https://github.com/microsoft/qsharp/assets/16928427/b8938278-7535-4abf-be9b-b685e7865234)

Jupyter notebook before/after side by side:


![image](https://github.com/microsoft/qsharp/assets/16928427/2d669d0a-0b0f-43de-9bef-fa083dbe49a9)
@github-actions
Copy link
Copy Markdown

Benchmark for 052dbd9

Click to view benchmark
Test Base PR %
Array append evaluation 875.5±12.60µs 860.8±21.21µs -1.68%
Array update evaluation 949.0±4.75µs 932.3±4.21µs -1.76%
Deutsch-Jozsa evaluation 31.3±0.21ms 31.3±0.25ms 0.00%
Large file parity evaluation 36.1±0.17ms 36.0±0.11ms -0.28%
Large input file 39.7±1.13ms 41.3±0.84ms +4.03%
Standard library 21.7±1.08ms 21.6±0.82ms -0.46%
Teleport evaluation 105.0±2.26µs 103.7±1.70µs -1.24%

Copy link
Copy Markdown
Member

@minestarks minestarks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it, thank you!

@swernli swernli enabled auto-merge January 31, 2024 18:40
@swernli swernli added this pull request to the merge queue Jan 31, 2024
Merged via the queue into main with commit d81a83a Jan 31, 2024
@swernli swernli deleted the swernli/sample-notebooks branch January 31, 2024 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants