2024 reduce documentation build time after classical shadows added

[bdg221]

Fixes #2024

I am starting with the examples/rshadows_tutorial.md which takes on average over 7 minutes to run during the Sphinx documentation build.

Next, I will look to address the guide/shadows-1-intro.md which takes on average over 6.5 minutes to run.

To start with I will be taking any files that need to be remote pulled from a github repository and place those files locally under docs/source/resources.

If this still does not provide enough of a speed up, then I will look at other areas that could benefit the build time.

This PR will be used in testing the build times, so I do expect there to be a number of commits after the PR is opened.

License

• I license this contribution under the terms of the GNU GPL, version 3 and grant Unitary Fund the right to provide additional permissions as described in section 7 of the GNU GPL, version 3.

Before opening the PR, please ensure you have completed the following where appropriate.

• I added unit tests for new code.
• I used type hints in function signatures.
• I used Google-style docstrings for functions.
• I updated the documentation where relevant.
• Added myself / the copyright holder to the AUTHORS file

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (68b2b12) 98.30% compared to head (dadd8d4) 98.30%.
Report is 13 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2058   +/-   ##
=======================================
  Coverage   98.30%   98.30%           
=======================================
  Files          87       87           
  Lines        4118     4125    +7     
=======================================
+ Hits         4048     4055    +7     
  Misses         70       70           

see 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bdg221]

@purva-thakre I think you are correct with your comments on the issue. It makes much more sense that quantum processing of the circuits is taking the significant amount of time resulting.

[bdg221]

The new branch that I used included commits for a different issue/PR. I am happy to close this PR and open a new clean one to address this particular issue of the long documentation build time.

I did a hard reset on the branch and to clean it up. Also, I put the saved output into a repository on my github account. If/when we see the appropriate reduced time, then we can determine if there is a better place to put the files.

On my own system when building the docs locally, I found that after pulling the saved content for shadow_quantum_processing the time was reduced significantly. After those calls, the two remaining calls that take up the majority of the time are:

f_est = pauli_twirling_calibrate(
    k_calibration=1,
    qubits=qubits,
    executor=noisy_executor,
    num_total_measurements_calibration=50000,
)

and

cal_est_corrs = classical_post_processing(
    shadow_outcomes=shadow_measurement_output,
    use_calibration=True,
    calibration_results=f_est,
    state_reconstruction=True,
)

If we wanted to reduce the time for the Classical Shadows guide further, we could look to reduce num_total_measurements_calibration

[bdg221]

• only showing examples and guides that take longer than 1.5 minutes

By saving the shadow_quantum_processing results off and pulling them from github, the build time for guide/rshadows-1-intro.md dropped from the average of 6.62 minutes to 3.94 minutes.

examples/rshadows_tutorial.md already uses the saved results, but this call is also taking the majority of the time:

    f_est = pauli_twirling_calibrate(
        zero_state_shadow_outcomes=zero_state_shadow_output,
        k_calibration=k_calibration,
        locality=locality,
    )

This is called in a for loop for a total of 3 times with k_callibration = 10 and locality = 3. Then, the function is called again with k_callibration = 5 and locality = 2.

Would we want to look into saving the results of the pauli_twirling_callibrate to a file for guide/shadows-1-intro.md and exmaples/rshadows_tutorial.md? Does it matter if we use pre-calculated results as long as we provide the method for generating the results along with it? Is there a specific timing that we want to stay under for the different examples and guides when it comes to the builds?

Pinging @Misty-W @natestemen @andreamari for thoughts

[andreamari]

@bdg221 thanks! In my opinion, any action which reduces the build time is good. So for me it's ok to:

a) Check if it is possible to reduce the num_total_measurements_shadow without reducing too much the performances. I.e., the technique must show an clear improvement even if not a maximum improvement.

b) Move the pickle files that you have in your repo in the examples folder such that you can load them quickly without downloading from internet.

c) For me it's ok to also load the output of pauli_twirling_calibrate from file if it gives an advantage of at least 1 or 2 minutes.

We know that points b) and c) are not optimal for at least two reasons: they require saving data in the source code making the Mitiq repository a bit heavier and in practice we don't test some code of the docs by skipping it's execution.
However, given the long execution time, the advantages seem stronger than the disadvantages.

[bdg221]

I changed num_total_measurements_calibration from 50,000 to 5,000 as a test for guide/shadows-1-intro.md. For the PR build, it appears that the time dropped from ~3.94 minutes to ~3.05. However, I think the better approach would be loading saved results.

@natestemen @Misty-W I won't be able to make the Community Call tomorrow, but I would appreciate it if you could discuss this PR. I agree with @andreamari that loading the pauli_twirling_calibrate from a file would be beneficial. Does everybody agree on this approach? Also, as was brought up, where should we put these files so that we know they will exist in the future (rather than in somebody's personal github repo)? Should an existing directory be used or a new directory be created for them (ie. resources)?

Also, the Robust Shadow Estimation example uses the following explanation of using saved results. Are we okay with this language or would we want to provide more details? (Obviously changing the wording for pauli_twirling_calibrate.)

Whether to run the quantum measurement or directly use the results from the previous run. If True, the measurement will be run again. If False, the results from the previous run will be used.

Finally, I wanted to step back and check the doc build to see how long the whole process took and how long the examples and guides took to build. In the last build of this PR, the build/docs(pull_request) took ~66 minutes with ~61 minutes coming from just the examples and guides code execution. Below are the top 10 culprits of the examples and guides which account for over 48 minutes:

[Misty-W]

thanks @bdg221!

I propose creating a folder within the examples folder for approaches (b) and (c). Since you already implemented (a), that should result in smaller files, which should mitigate the size concern somewhat.

[bdg221]

After adding local files for pauli_twirling_calibrate() and shadow_quantum_processing(), the following improvements were seen:

rshadows_tutorial.md - 51.64 seconds (previously taken between 435 - 515 seconds)
shadows-1-intro.md - 104.81 seconds (previously taken between 182 seconds with some adjustments - 236 seconds originally)

If we are able to see similar reductions from the other long running examples/guides then I believe we can see a very noticeable reduction in build time.

Unless I hear otherwise, I will continue working on ways to use local files to reduce the build times.

[andreamari]

After adding local files for pauli_twirling_calibrate() and shadow_quantum_processing(), the following improvements were seen:

rshadows_tutorial.md - 51.64 seconds (previously taken between 435 - 515 seconds) shadows-1-intro.md - 104.81 seconds (previously taken between 182 seconds with some adjustments - 236 seconds originally)

Looks like good improvements! Thanks!

If we are able to see similar reductions from the other long running examples/guides then I believe we can see a very noticeable reduction in build time.

Unless I hear otherwise, I will continue working on ways to use local files to reduce the build times.

I would keep this PR within the topic of classical shadows. Mixing too many things in a single PR can be hard to review and it obfuscates what specific change produced some given specific improvement.

If you have ideas for further speed up improvements, unrelated to classical shadows, I would suggest to open new independent issues.

[andreamari]

LGTM! Thanks @bdg221 for the nice speed up!