Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip writing pyc files [ci] #98423

Merged
merged 1 commit into from Aug 16, 2023
Merged

Skip writing pyc files [ci] #98423

merged 1 commit into from Aug 16, 2023

Conversation

cdce8p
Copy link
Member

@cdce8p cdce8p commented Aug 14, 2023

Proposed change

Skip writing bytecode files for pytest runs - a small performance improvement. This is only really important for pytest as it compiles all test files as well. The other linters just read them as text files.
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE
https://docs.pytest.org/en/7.1.x/how-to/assert.html#assertion-rewriting-caches-files-on-disk

I also explored caching those to speed up pytest even more. However the .pyc files are regenerated based on the mtime of the source code which resets with each git checkout. Thus, caching them unfortunately doesn't really work.

Update
Reading on it some more, Python does actually support hash based .pyc checks via the --invalidation-mode option.
https://docs.python.org/3/reference/import.html#cached-bytecode-invalidation
https://docs.python.org/3/library/compileall.html#cmdoption-compileall-invalidation-mode

However, pytest certainly does not.
https://github.com/pytest-dev/pytest/blob/7.4.0/src/_pytest/assertion/rewrite.py#L387-L390

Even if it would, there is still the issue that the test suite is split into multiple groups. Therefore the bytecode files would likely need to be combined for caching to be effective. A lot of work. I might explore it in the future but for now simply disabling writing the bytecode files provides the most improvements.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@home-assistant home-assistant bot added cla-signed small-pr PRs with less than 30 lines. labels Aug 14, 2023
Copy link
Member

@frenck frenck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @cdce8p 👍

../Frenck

@frenck frenck merged commit 0bcc02e into home-assistant:dev Aug 16, 2023
34 checks passed
@cdce8p cdce8p deleted the ci-skip-pyc branch August 16, 2023 09:40
@github-actions github-actions bot locked and limited conversation to collaborators Aug 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed small-pr PRs with less than 30 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants