Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

distutils-r1/python-utils-r1.eclass: Support EPYTEST_XDIST #33667

Closed
wants to merge 6 commits into from

Conversation

mgorny
Copy link
Member

@mgorny mgorny commented Nov 3, 2023

Add an EPYTEST_XDIST control variable that:

  1. Makes epytest enable xdist with the usual set of parameters (provided that we're actually doing >1 job).
  2. Makes distutils_enable_tests pytest add dev-python/pytext-xdist dep.

Also add EPYTEST_JOBS to control jobs independently of MAKEOPTS.

I suppose when this lands, we'd use a pkgcheck check for EPYTEST_XDIST being declared after d_e_t call.

@gentoo-bot
Copy link

Pull Request assignment

Submitter: @mgorny
Areas affected: ebuilds, eclasses
Packages affected: dev-python/numpy, dev-python/pyrate-limiter

dev-python/numpy: @gentoo/sci, @gentoo/python
dev-python/pyrate-limiter: @AndrewAmmerlaan, @gentoo/python

Linked bugs

No bugs to link found. If your pull request references any of the Gentoo bug reports, please add appropriate GLEP 66 tags to the commit message and request reassignment.

If you do not receive any reply to this pull request, please open or link a bug to attract the attention of maintainers.


In order to force reassignment and/or bug reference scan, please append [please reassign] to the pull request title.

Docs: Code of ConductCopyright policy (expl.) ● DevmanualGitHub PRsProxy-maint guide

@gentoo-bot gentoo-bot added assigned PR successfully assigned to the package maintainer(s). no bug found No Bug/Closes found in the commits. labels Nov 3, 2023
@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-11-03 19:42 UTC
Newest commit scanned: 2dfc226
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/95c9e1cb8c/output.html

Copy link
Member

@arthurzam arthurzam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Especially ${EPYTEST_JOBS} will help on devboxes such as catbus.sparc, where we have a huge amount of cores, that work well with make or ninja, but with pytest-xdist this job count doesn't work well (each core is very weak) - so now I can set it up with normal amount of jobs.

eclass/distutils-r1.eclass Outdated Show resolved Hide resolved
eclass/distutils-r1.eclass Show resolved Hide resolved
Copy link
Contributor

@ionenwks ionenwks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, had a peek at metadata too and not seeing anything off

Signed-off-by: Michał Górny <mgorny@gentoo.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
Add an `EPYTEST_XDIST` variable that can be used to enable running
the test suite in parallel via the dev-python/pytest-xdist plugin.
This also includes user-facing `EPYTEST_JOBS` to control the job count
independently of `MAKEOPTS`.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
Make `distutils_enable_tests pytest` automatically add
the `dev-python/pytest-xdist` dependency if `EPYTEST_XDIST` is set.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-11-04 06:17 UTC
Newest commit scanned: 795d291
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/025843b9b7/output.html

Copy link
Member

@thesamesam thesamesam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I got fed up of copying this everywhere.

>=dev-python/pytz-2019.3[${PYTHON_USEDEP}]
)
"

EPYTEST_XDIST=1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer it if we always kept eclass vars above inherit, but w/e.

if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then
test_deps+=" ${test_pkg}[${PYTHON_USEDEP}]"
test_deps+=" ${test_pkgs//'${PYTHON_USEDEP}'/${PYTHON_USEDEP}}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks a bit grim but if the metadata looks ok, good enough.

@gentoo-bot gentoo-bot closed this in 3c8764c Nov 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assigned PR successfully assigned to the package maintainer(s). no bug found No Bug/Closes found in the commits.
Projects
None yet
6 participants