-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support TBB on non-Linux, skip PyPy #73
Conversation
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
@conda-forge-admin, please rerender |
…da-forge-pinning 2021.03.13.19.08.27
I think this is working. Can I get another pair of eyes or two? |
@stuartarchibald might be interested in verifying, too? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the patch.
I think the original restrictions around TBB were from a time when Numba supported a lot more architectures than were widely supported in the ecosystem. For what conda-forge builds against, removal of these restrictions seems valid.
Numba doesn't work on PyPy so that restriction makes sense and the GH homepage redirect seems fine.
Thanks again!
The Drone aarch64 builds are randomly hanging. I'm going to restore the aarch64 limiting for now; macOS and Windows seem to be just fine. (might be a Drone issue, or might be TBB related). |
@conda-forge-admin, please rerender |
Ahh, looking in the history, it looks like Drone has been randomly hanging for a while, doesn't appear to have anything to do with the addition of TBB. I'll revert back to the original, tbb everywhere version. |
e9b3bec
to
2afc959
Compare
Yes, I think @jakirkham's analysis is correct. The higher versions of Python sometimes take 55 mins instead of 20 mins, which causes them to pass. 3.6 and 3.7 seem to be most vulnerable to failing with a 1 hour timeout. Is there some other infrastructure we can move to here? @chrisburr, do you know if there's an alternative to Drone like there is for Travis? Another idea could be to skip running the tests on 3.6 and 3.7 for Drone, I think that frees up several minutes. |
For context, I think Henry is referring to the discussion in issue ( #74 ) Am not aware of an alternative for cc @isuruf (in case you have other thoughts) |
We would only need to skip for Currently, it seems to only make it through 1/4 of the time or maybe less. I've restarted this about 6 times, always fails on one of 3.6, 3.7, and at least once failed with both. |
You can try cross compilation and running tests using QEMU, or full emulation. Those are the two choices right now. Skipping a python version wouldn't work, because it depends on which machine gets assigned in drone CI (They have different types of machines and some are slower) and even if this version passes just under 55 mins, a new version might not. |
Just want to highlight this option again. For example do we need to run all of the TBB tests there? Edit: Or could we just run them on Python 3.8 given that seems to work reliably and these tests are not likely too dependent on Python versions? |
It's possible Drone will update all their machines by the time the next version is released? Guessing the future of both the CI and the changes to Numba isn't very reliable. We can always change in the future if it fails on a new version. We are currently, if I understand correctly, only running 20% of the tests on aarch64, maybe a first step would be to reduce that fraction? I wonder if that's why it's variable (3.6 and 3.7 do pass sometimes), it depends on which 20% it selects. Maybe there's a long test or two that can always be excluded? |
numba-feedstock/recipe/run_test.sh Lines 40 to 41 in 9796deb
|
IIRC for a given environment the selection will be consistent across runs. |
I think the really heavy threading layer specific tests should already be skipped by virtue of Another option, instead of picking a percentage of "random" tests, is to "slice" through the test suite with the slice syntax as used here:
run_these_tests = list_of_all_tests[slice($TEST_START_INDEX,None,$TEST_COUNT)] and in the case of Numba's own CI, there's More info here: |
10% of the tests passed on the first try. It cut the "long" time for Python 3.7 to 42-ish minutes, and 3.9 to 39 minutes. For Python 3.6 and 3.8, it ran on the newer hardware, taking 12.5 and <12 minutes, respectively. Almost tempted to try 15%. I think I will.
It could very well be, this was running at 59 minutes when it was not timing out, so it was easily possible the differences causing pass/fail were CI load related. If these are consistent, I think it's fine as long as they easily pass. |
Haha, of course, that's the one time in the last 10 runs or so that py36 and py37 both get a fast node. 🤦 18 mins for 3.6 & 3.7, and 17 mins for 3.8. Looks like 3.9 got a slow node, and it took 53 minutes. Guessing around 55 mins for 3.6 and 3.7. I think that's pretty good, let's go with this for now, we can always refer back here and modify if things get more challenging in the future (or less, too). Thanks everyone! |
Sounds great! Thanks for working on this Henry 😄 |
Explicit skip for PyPy added, testing out including TBB (closes #43), and update the test line for TBB (closes #63). Also fixed the homepage link; GitHub will turn off the old redirects at some point.