Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce redundant tests & overheads for CI #319

Merged
merged 44 commits into from
Aug 3, 2022
Merged

Reduce redundant tests & overheads for CI #319

merged 44 commits into from
Aug 3, 2022

Conversation

mlxd
Copy link
Member

@mlxd mlxd commented Jul 27, 2022

Before submitting

Please complete the following checklist when submitting a PR:

  • All new features must include a unit test.
    If you've fixed a bug or added code that should be tested, add a test to the
    tests directory!

  • All new functions and code must be clearly commented and documented.
    If you do make documentation changes, make sure that the docs build and
    render correctly by running make docs.

  • Ensure that the test suite passes, by running make test.

  • Add a new entry to the .github/CHANGELOG.md file, summarizing the
    change, and including a link back to the PR.

  • Ensure that code is properly formatted by running make format.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


Context: This PR removes the need for intermediate Python version tests between the supported extrema versions (currently 3.7-3.10) for PRs. The full builder suite is still run on merge to master, and release. We also update to MacOS11 in tandem.

Description of the Change: See above.

Benefits: Reduced PR resource requirements and overhead for running CI checks.

Possible Drawbacks: Minimal chance of intermediate Python version issues that are reported on merge to master.

Related GitHub Issues:

@github-actions
Copy link
Contributor

Hello. You may have forgotten to update the changelog!
Please edit .github/CHANGELOG.md with:

  • A one-to-two sentence description of the change. You may include a small working example for new features.
  • A link back to this PR.
  • Your name (or GitHub username) in the contributors section.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 27, 2022

Test Report (Linux, Kokkos, and Kokkos Kernels) on Ubuntu

           1 files  ±0             1 suites  ±0   2s ⏱️ -1s
       975 tests ±0         975 ✔️ ±0  0 💤 ±0  0 ±0 
229 280 runs  ±0  229 280 ✔️ ±0  0 💤 ±0  0 ±0 

Results for commit d68aafe. ± Comparison against base commit f1d696f.

♻️ This comment has been updated with latest results.

@mlxd mlxd marked this pull request as ready for review July 28, 2022 19:00
@mlxd mlxd changed the title [WIP] Reduce redundant tests for CI Reduce redundant tests & overheads for CI Jul 28, 2022
@codecov
Copy link

codecov bot commented Jul 29, 2022

Codecov Report

Merging #319 (d68aafe) into master (f1d696f) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #319   +/-   ##
=======================================
  Coverage   99.79%   99.79%           
=======================================
  Files          40       40           
  Lines        3929     3929           
=======================================
  Hits         3921     3921           
  Misses          8        8           
Impacted Files Coverage Δ
pennylane_lightning/_version.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@maliasadi maliasadi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mlxd for putting this together! I may suggest to add pyver in a separate yml file and re-use it with wheels .yml instead of defining it in all of them. What do you think?

.github/workflows/benchmarks.yml Show resolved Hide resolved
Copy link
Contributor

@chaeyeunpark chaeyeunpark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one single comment.

@chaeyeunpark
Copy link
Contributor

Another potential issue I found is that Wheel:: workflows upload the built wheels under the same name for different python versions (as we use a matrix). This somehow merges all uploaded wheels into a single file now, but it is not a documented behavior and can be changed in the future actions/upload-artifact#24. Maybe it is better to upload files in different names but download them all in upload-pypi action (using https://github.com/actions/download-artifact#download-all-artifacts)?

@mlxd
Copy link
Member Author

mlxd commented Aug 3, 2022

Another potential issue I found is that Wheel:: workflows upload the built wheels under the same name for different python versions (as we use a matrix). This somehow merges all uploaded wheels into a single file now, but it is not a documented behavior and can be changed in the future actions/upload-artifact#24. Maybe it is better to upload files in different names but download them all in upload-pypi action (using https://github.com/actions/download-artifact#download-all-artifacts)?

I think I was relying on this to merge all wheels into a single Zip, as twine uploads accepts a Zip of all wheels as a single pass, and each wheel will be guaranteed a unique name from the builder and naming semantics. But you are correct that this is something we may want to update for how it is handled. I think we can handle this separately as they seem to still be debating the behaviour in the action, and it looks like nothing will change on this before the next release. Thoughts?

@chaeyeunpark
Copy link
Contributor

chaeyeunpark commented Aug 3, 2022

Another potential issue I found is that Wheel:: workflows upload the built wheels under the same name for different python versions (as we use a matrix). This somehow merges all uploaded wheels into a single file now, but it is not a documented behavior and can be changed in the future actions/upload-artifact#24. Maybe it is better to upload files in different names but download them all in upload-pypi action (using https://github.com/actions/download-artifact#download-all-artifacts)?

I think I was relying on this to merge all wheels into a single Zip, as twine uploads accepts a Zip of all wheels as a single pass, and each wheel will be guaranteed a unique name from the builder and naming semantics. But you are correct that this is something we may want to update for how it is handled. I think we can handle this separately as they seem to still be debating the behaviour in the action, and it looks like nothing will change on this before the next release. Thoughts?

👍

Copy link
Contributor

@chaeyeunpark chaeyeunpark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to approve this!

@mlxd
Copy link
Member Author

mlxd commented Aug 3, 2022

Thanks for the reviews all.

@mlxd mlxd merged commit 8098252 into master Aug 3, 2022
@mlxd mlxd deleted the reduce_test_set branch August 3, 2022 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants