[R] Package Arrow 11.0.0 for R/CRAN #33702
Comments
|
Two nightly builds are failing: test-r-linux-valgrind (#33094) and test-r-rhub-debian-gcc-devel-lto-latest (#33701). The valgrind failures won't be solved for this release; however, I've run them locally with archery to make sure they are the same as they were before the last release (and they appear to be). I haven't been able to replicate those errors outside of that docker image and we haven't had any valgrind issues reported by CRAN. I'm working to see if the LTO error is transient or cache-related. |
|
The current CRAN check status indicates a few issues. I checked to make sure the package would compile under gcc-13 (unreleased, requires building gcc from source) and I wasn't able to get any build errors (#33635). The warning for suppressed diagnostics may be a problem; however, we can't solve it without actively attempting to circumvent a CRAN policy which would be bad (I think): #33637. |
|
The link-time optimization nightly failure is probably related to our CI config...I'll continue to investigate and if there's a fix that's needed for CRAN we can pick it into the packaging branch. |
kou
changed the title
|
gcc13 no longer shows up on https://cran.r-project.org/web/checks/check_results_arrow.html so we're good there. Just have to deal with clang 16 now :/ |
|
Is there an Arrow issue for clang16 yet? I didn't get to it today but I'm sure I can get a docker image together on Monday with clang16 and R. |
I haven't made one yet, just raised it on Zulip and the ML. I think @jonkeane tried to reproduce yesterday so he may have a dockerfile started. |
|
I made #33819 to track. LLVM provides a Dockerfile for a source build on Debian 10 so we should be able to reproduce. |
|
Re: maintainer change, CRAN policy says "Explain any change in the maintainer’s email address and if possible send confirmation from the previous address (by a separate email to CRAN-submissions@R-project.org) or explain why it is not possible." I will do this at the same time as the submission. |
|
Potential list of things we need to pick into the CRAN-specific release branch:
Neal mentioned removing the badges from the readme (but that's already on the checklist 🙂 ) |
|
@paleolimbot As discussed separately, mind just clarifying what we need to do to proceed here? |
|
The "just update boost" fix turned out to be more complicated than I expected and is a poor candidate for this point in the release cycle. The macOS install timeout thing is only an issue if when you run crossbow submit --group r you see that job fail (and re-running doesn't fix); the Boost fix is only an issue if our CRAN submission gets rejected. That leaves the LTO fix. I think the best way to proceed would be to pick the LTO fix, remove the badges, and carry on until we have a reason to pick anything else. |
|
@paleolimbot Would you mind running the revdepchecks? I remember last time I spent ages and didn't even get it done, due to some weird quirk in the difference between our respective setups, and we did discuss moving it to a Crossbow job at some point but haven't yet. |
|
Builth the package locally and I got the output below, so nothing unexpected there : |
|
The first one you can probably solve by doing The second one is showing up because you have The third one always shows up for us...we just have a really big shared library. |
|
Yep, re-ran after doing exactly that and the warnings disappeared. |
How so? I don't see problems mentioned on #33890 Up to y'all (ultimately Nic, since they'll be the one receiving the CRAN emails) if you want to submit without the boost upgrade. It's reasonable to submit without the upgrade, and if it still fails, respond that we ran on latest clang16 before submission and it passed so we believed the issue to be resolved. Downside is that I doubt we would get any extra time to fix, so I think it will still need to be resolved by Feb 16. |
|
See #33890 (comment) ...something is missing in the bundle I created via the trim-boost.sh script. If you have a workflow for debugging what it might be I'd love to know before I have to invent one! |
|
That fix needs completing and merging at some point anyway, and if it causes us to fail CRAN checks we'd need to submit it again with the change made, so it seems like the only real consequence of waiting for it to be done ahead of the CRAN release (given we're not 100% sure it'll cause a failure) is delaying the initial submission and adding more to @paleolimbot's to-do list in the short-term? If that's the case, I'm happy to submit to CRAN without it, if only so that we know sooner whether it matters or not. If any of my assumptions above don't hold though, let's reconsider. |
|
On a different note, we still have the issue of the URL check failing as there is a URL in our docs which currently exists on the dev docs but not the live docs yet. Is there anything to do about this other than wait for the arrow-site update before submitting to CRAN? |
|
Which link? Is it something that should be a relative URL? Otherwise sure, we can wait for the 11.0.0 docs to be published. |
|
FTR the docs update PR is apache/arrow-site#308 |
|
Thanks for linking to that, I didn't realise that that had just been done. Broken link is one of the new vignettes after the big docs refactor. |
|
Oh, I see, a relative URL would fix that. |
|
Actually...would it? It's in the README |
|
I think that is only 404'ing because the website is not updated for 11.0.0 yet: https://arrow.apache.org/docs/dev/r/articles/install_nightly.html |
|
|
|
Looks like the docs PR has merged. |
|
Great; I'm just waiting for the crossbow builds to run now + Dewey is doing revdepchecks, so after that it'll just be the Linux binary to upload + winbuilder/macbuilder checks and then off to CRAN. |
|
👍 sounds good, I'll email about the maintainer change now. |
It did fail; re-running now 🤞 |
|
Reverse dependencies are clear, both via archery and via my hacked-together local thing. Output from the hacked together local thing below...we have 4 more revdeps since the last version! ChangesCheck SummaryCRANLocalAll notes and errorsdisk.framereceptivitiduckdbfastverseparquetizepointblanksparklyrstrand |
|
Thanks @paleolimbot, will plough on through the list! |
|
Latest:
Once the crossbow checks are done and we're happy with the results, I'll submit the latest version to CRAN. |
|
I ran |
|
I got a failure with |
|
Follow-up tasks now completed so closing this. |
Describe the bug, including details regarding any error messages, version, and platform.
Packaging checklist for CRAN release
For a high-level overview of the release process see the
Apache Arrow Release Management Guide.
Before the release candidate is cut:
entitled
[R] CRAN packaging checklist for version X.X.Xand copy this checklist to the issue.
nightly tests and nightly packaging builds. These checks
replicate most of the checks that CRAN runs, so we need them all to be passing
or to understand that the failures may (though won't necessarily) result in a rejection from CRAN.
urlchecker::url_check()on the R directory at the release candidatecommit. Ignore any errors with badges as they will be removed in the CRAN release branch.
Wait for the release candidate to be cut:
Make pull requests into the autobrew and
rtools-packages repositories
used by the configure script on MacOS and Windows. These pull requests will
use the release candidate as the source.
the apache-arrow autobrew formula
to update the release version, SHA256 checksum of the release source file (which can be found in the same directory as the release source file), and any changes to dependencies and build steps that have changed in the
copy of the formula we have of that formula in the Arrow repo
the apache-arrow-static autobrew formula
to update the version, SHA, and any changes to dependencies and build steps that have changed in the
copy of the formula we have of that formula in the Arrow repo
autobrew script
to include any additions made to
r/tools/autobrew.
RTools PKGBUILD script
to reflect changes in
ci/PKGBUILD,
uncommenting the line that says "uncomment to test the rc".
Prepare and check the .tar.gz that will be released to CRAN.
git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -dmake build. This copies Arrow C++ into tools/cpp, prunes someunnecessary components, and runs
R CMD buildto generate the source tarball.Because this will install the package, you will need to ensure that the version
of Arrow C++ available to the configure script is the same as the version
that is vendored into the R package (e.g., you may need to unset
ARROW_HOME).devtools::check_built("arrow_X.X.X.tar.gz")locallymanual process;
however, in the future it may be a crossbow nightly job.
Wait for the official release...
on that commit.
were needed to fix CRAN-related submission issues identified in the above
steps.
urlchecker::url_check()on the R directoryWIP: [R] Verify CRAN release-10.0.1-rc0. Adda comment
@github-actions crossbow submit --group rto run all R crossbowjobs against the CRAN-specific release branch.
make build)Create new autobrew and r-windows PRs such that they use the release
instead of the release candidate; ensure linux binary packages are available:
https://apache.jfrog.io/ui/repos/tree/General/arrow/r
Check binary Arrow C++ distributions specific to the R package:
and confirm (with Nic, who will automatically receive an email about the results) that the check is clean.
This step cannot be completed before Jeroen has put the binaries in the MinGW repository, i.e. here, here, and here.
and confirm that the check is clean
install.packages("arrow_X.X.X.tar.gz")on Ubuntu and ensure that thehosted binaries are used
devtools::check_built("arrow_X.X.X.tar.gz")locally one more time (for luck)Submit!
CRAN submit page
Wait for CRAN...
ci/scripts/PKGBUILD,dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb,r/DESCRIPTION, andr/NEWS.mdpackaging process.
CRAN package page to reflect the
new version
Component(s)
R
The text was updated successfully, but these errors were encountered: