-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs #14832
Conversation
|
|
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58 |
|
@github-actions crossbow submit -g conda |
Revision: a123df8 Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6 |
done by: `git add --renormalize -- dev/tasks/conda-recipes/r-arrow`
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here. Also: |
compilers are redundant as they are fixed per platform
@github-actions crossbow submit -g conda |
Revision: 39587c3 Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae |
this is especially relevant for the emulated tests on linux-aarch64
this avoids blowing up CI time by 30+ minutes due to running into unnecessary timeouts, by skipping earlier (if testbench not found)
@github-actions crossbow submit -g conda |
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix ( |
That's unavoidable without some surgery or splitting the feedstocks into I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect |
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top. |
* build for arrow 10.0.1 * MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41 * sync with upstream; be explicit about -std=c++17 * re-add ${R_ARGS} * MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35 * sync back changes from apache/arrow#14832 Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
I have manually re-run |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tried reviewing the different commits individually and looks good to me
Can we merge this as-is? This fixes our nightly CI. |
No objection. I'll merge this. |
Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
apache#34652) As well as other changes that happened on the feedstock since apache#14832 * Closes: apache#34669 Lead-authored-by: H. Vetinari <h.vetinari@gmx.com> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
apache#34652) As well as other changes that happened on the feedstock since apache#14832 * Closes: apache#34669 Lead-authored-by: H. Vetinari <h.vetinari@gmx.com> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
apache#34652) As well as other changes that happened on the feedstock since apache#14832 * Closes: apache#34669 Lead-authored-by: H. Vetinari <h.vetinari@gmx.com> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in
tasks.yml
).Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy
(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with
sparse
.