Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On older versions of Python, skip benchmarks that use features introduced in newer Python versions #283

Merged
merged 5 commits into from Apr 27, 2023

Conversation

AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Apr 26, 2023

All our benchmarks have a requires-python field in their pyproject.toml files, e.g.:

The requires-python field is added to the metadata of each benchmark as a python field here:

'requires-python': 'python',

for field, target in PEP_621_FIELDS.items():
if target is None:
target = field
if field == 'url':
repo = project.get('urls', {}).get('repository')
raise NotImplementedError
elif not resolved.get(target):
value = project.get(field)
if value is not None:
resolved[target] = value

We can use that metadata to create a new python property on Benchmark objects, which returns a packaging.specifiers.SpecifierSet instance. This property can then be easily used to filter out benchmarks that require a higher version of Python than the version of Python pyperformance is running on.

Fixes #281. Unblocks #280 and #268.

I haven't added a test for this -- I was unsure if it was necessary, and if it was necessary, where that test should go. I'm happy to add one if that would be helpful and there's an obvious place it could go, however!

I manually tested adding a benchmark that used new-in-Python-3.8 features in 1734919, and the CI passed fine (adding the same benchmark to main currently causes the CI to fail — see #280). Passing CI run on my GitHub fork: https://github.com/AlexWaygood/pyperformance/actions/runs/4813903519

Copy link
Member

@brandtbucher brandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm impressed how simple this is! Great job.

One opportunity to remove some branching:

pyperformance/_benchmark.py Outdated Show resolved Hide resolved
pyperformance/cli.py Outdated Show resolved Hide resolved
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
@AlexWaygood AlexWaygood merged commit 708265b into python:main Apr 27, 2023
11 checks passed
@AlexWaygood AlexWaygood deleted the new-feature-benchmarks branch April 27, 2023 02:28
@ericsnowcurrently
Copy link
Member

This is a great improvement. Thanks, @AlexWaygood!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Impossible to add a benchmark that exercises newer Python features
3 participants