GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403

jonkeane · 2024-04-26T20:59:48Z

Rationale for this change

Keep up with the state of the world, ensure we are maintaining backwards compatibility.

Resolves #41402

What changes are included in this PR?

Bump to 4.4 as the release
Remove old 3.6 jobs now that we no longer support that; clean up code where we hardcode things fro 3.6 and below
Move many of our CI jobs to rhub's new containers. We were accidentally running stale R devel (from December 2023) because the other rhub images stopped being updated. (One exception to be done as a follow on: [R][CI] Use the new rhub/clang-asan image #41416)
Resolve a number of extended test failures

With this PR R extended tests should be all green with the exceptions of:

Two sanitizer jobs (test-fedora-r-clang-sanitizer, test-ubuntu-r-sanitizer) — which are being investigated / fixed in GH-41407: [C++] Use static method to fill scalar scratch space to prevent ub #41421
Valgrind — I'm running one last run with a new suppression file.
Binary jobs — these work but fail at upload, see GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403 (comment)
Windows R Release — failing on main, [R][CI] Windows job failing after R 4.4 release #41398

Are these changes tested?

By definition.

Are there any user-facing changes?

No.

GitHub Issue: [CI][R]: Update our backwards compatibility CI any other R 4.4 cleanups #41402

github-actions · 2024-04-26T21:00:13Z

⚠️ GitHub issue #41402 has been automatically assigned in GitHub to PR creator.

jonkeane · 2024-04-26T21:00:38Z

@github-actions crossbow submit test-r-arrow-backwards-compatibility

github-actions · 2024-04-26T21:02:53Z

Revision: 6b5dda3

Submitted crossbow builds: ursacomputing/crossbow @ actions-3ae27aae8b

Task	Status
test-r-arrow-backwards-compatibility

jonkeane · 2024-04-26T21:18:52Z

@github-actions crossbow submit -g r

github-actions · 2024-04-26T21:21:29Z

Revision: f31f859

Submitted crossbow builds: ursacomputing/crossbow @ actions-802b06fdc8

Task	Status
r-binary-packages
test-fedora-r-clang-sanitizer
test-r-arrow-backwards-compatibility
test-r-depsource-bundled
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-valgrind
test-r-minimal-build
test-r-offline-maximal
test-r-offline-minimal
test-r-rhub-debian-gcc-devel-lto-latest
test-r-rhub-debian-gcc-release-custom-ccache
test-r-rhub-ubuntu-gcc-release-latest
test-r-rocker-r-ver-latest
test-r-rstudio-r-base-4.1-opensuse153
test-r-rstudio-r-base-4.2-centos7-devtoolset-8
test-r-rstudio-r-base-4.2-focal
test-r-ubuntu-22.04
test-r-versions
test-ubuntu-r-sanitizer

assignUser

Nice, thanks for updating everything! The ci errors are mostly #41359 and likely unrelated (valgrind/ubsan). Not sure whats going on with the offline install but that's also failing on main.

assignUser · 2024-04-27T03:03:00Z

ci/docker/linux-apt-r.dockerfile

@@ -35,7 +35,7 @@ ENV LANG=C.UTF-8
 # Build R
 # [1] https://www.digitalocean.com/community/tutorials/how-to-install-r-on-ubuntu-18-04
 # [2] https://linuxize.com/post/how-to-install-r-on-ubuntu-18-04/#installing-r-packages-from-cran
-ARG r=3.6


dev/tasks/r/github.linux.versions.yml

assignUser · 2024-04-27T03:05:00Z

r/R/dplyr-funcs-type.R

@@ -140,7 +140,7 @@ register_bindings_type_cast <- function() {
             fix.empty.names = TRUE,
             stringsAsFactors = FALSE) {
      # we need a specific value of stringsAsFactors because the default was
-      # TRUE in R <= 3.6
+      # TRUE in R <= 3.6 and folks might still be cargoculting to stay in the past.


jonkeane · 2024-04-27T14:12:27Z

@github-actions crossbow submit test-r-offline-maximal

github-actions · 2024-04-27T14:14:45Z

Revision: f565fe6

Submitted crossbow builds: ursacomputing/crossbow @ actions-dbb88ab3ac

Task	Status
test-r-offline-maximal

jonkeane · 2024-04-27T14:27:50Z

@github-actions crossbow submit test-r-versions

github-actions · 2024-04-27T14:30:13Z

Revision: f565fe6

Submitted crossbow builds: ursacomputing/crossbow @ actions-eb08160d5b

Task	Status
test-r-versions

jonkeane · 2024-04-27T14:33:37Z

@github-actions crossbow submit test-r-offline-maximal

github-actions · 2024-04-27T14:35:54Z

Revision: 403c579

Submitted crossbow builds: ursacomputing/crossbow @ actions-4bdedd0dc6

Task	Status
test-r-offline-maximal

jonkeane · 2024-04-27T14:55:56Z

I fixed the maximal offline build, it was checking for docs and we were purposefully not biulding them. This looks to be a slight change in behavior of 4.4.0 where no docs at all and then testInstalledPackage fails. We could look into this separately and report upstream, but it's also pretty niche.

jonkeane · 2024-04-27T15:20:47Z

@github-actions crossbow submit test-r-linux-valgrind

github-actions · 2024-04-27T15:23:07Z

Revision: fcf9e74

Submitted crossbow builds: ursacomputing/crossbow @ actions-944b9b9e08

Task	Status
test-r-linux-valgrind

jonkeane · 2024-04-27T15:29:51Z

The valgrind cleanup I made probably will not fix the valgrind build (If I'm reading the output correctly, there's still a possible leak). But it will get rid of the silly failures where NAs are sometimes transformed into NaNs

https://stat.ethz.ch/pipermail/r-devel/2021-April/080683.html

So is an improvement (in that the failures are fewer), but might not be enough to have that be green.

jonkeane · 2024-04-27T16:46:34Z

Ooops, also caught that using the old rhub devel images are stuck in December:

* using R Under development (unstable) (2023-12-26 r85738)

I'll bump those to be the new "containers" approach: https://r-hub.github.io/containers/

jonkeane · 2024-04-27T17:03:21Z

@github-actions crossbow submit test-fedora-r-clang-sanitizer test-r-rhub-debian-gcc-devel-lto-latest

jonkeane · 2024-04-28T20:30:32Z

Should we also bump the minimum version of R in the DESCRIPTION? Based on the removal of the str2lang backport, we require at least 3.6 now? That said, str2lang is only used if you call quantile() in summarize(), so not practically an issue for most.

Yeah, I'll bump to 4.0 in the DESCRIPTION. I get we could have 3.6 there but I would rather pull out our random backports than have a lower (unsupported) version there.

jonkeane · 2024-04-28T20:37:01Z

@github-actions crossbow submit r-binary-packages test-r-linux-valgrind

github-actions · 2024-04-28T20:39:18Z

Revision: 8d91cab

Submitted crossbow builds: ursacomputing/crossbow @ actions-bb446802ac

Task	Status
r-binary-packages
test-r-linux-valgrind

jonkeane · 2024-04-28T22:43:28Z

@github-actions crossbow submit r-binary-packages

github-actions · 2024-04-28T22:45:45Z

Revision: 67941ba

Submitted crossbow builds: ursacomputing/crossbow @ actions-437d88c30f

Task	Status
r-binary-packages

dev/tasks/tasks.yml

assignUser · 2024-04-29T01:06:37Z

Wow, this balloned, thanks @jonkeane! A minor thing we can do in a follow up imo is dropping the centos7 builds as libarrow is dropping support for that too #41395 (or maybe we can do it in that PR).

assignUser · 2024-04-29T02:40:30Z

@github-actions crossbow submit r-binary-packages

@jonkeane I added the fix for the upload job just so we can potentially merge this if we want to get it into 16.1. Hope that's ok!

jonkeane · 2024-04-29T02:42:30Z

I'm happy for it to merge now. There are a few follow ons but I think those would be better separately!

Thanks for the fix for the binaries

github-actions · 2024-04-29T02:43:09Z

Revision: 8cad637

Submitted crossbow builds: ursacomputing/crossbow @ actions-c2f5ac8f88

Task	Status
r-binary-packages

assignUser · 2024-04-29T03:20:38Z

@jonkeane it looks like openssl is still not being picked up correctly so we probably need to look at that again. I think this is also caused by the change to the gh macos runners, I think I manually installed the cran openssl there... I will open an issue for a follow up.

….4 cleanups (#41403) ### Rationale for this change Keep up with the state of the world, ensure we are maintaining backwards compatibility. Resolves #41402 ### What changes are included in this PR? * Bump to 4.4 as the release * Remove old 3.6 jobs now that we no longer support that; clean up code where we hardcode things fro 3.6 and below * Move many of our CI jobs to [rhub's new containers](https://github.com/r-hub/containers). We were accidentally running stale R devel (from December 2023) because the other rhub images stopped being updated. (One exception to be done as a follow on: #41416) * Resolve a number of extended test failures With this PR R extended tests should be all green with the exceptions of: * Two sanitizer jobs (test-fedora-r-clang-sanitizer, test-ubuntu-r-sanitizer) — which are being investigated / fixed in #41421 * Valgrind — I'm running one last run with a new suppression file. * Binary jobs — these work but fail at upload, see #41403 (comment) * Windows R Release — failing on main, #41398 ### Are these changes tested? By definition. ### Are there any user-facing changes? No. * GitHub Issue: #41402 Lead-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

conbench-apache-arrow · 2024-04-29T16:55:17Z

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 6eb0b37.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

…er R 4.4 cleanups (apache#41403) ### Rationale for this change Keep up with the state of the world, ensure we are maintaining backwards compatibility. Resolves apache#41402 ### What changes are included in this PR? * Bump to 4.4 as the release * Remove old 3.6 jobs now that we no longer support that; clean up code where we hardcode things fro 3.6 and below * Move many of our CI jobs to [rhub's new containers](https://github.com/r-hub/containers). We were accidentally running stale R devel (from December 2023) because the other rhub images stopped being updated. (One exception to be done as a follow on: apache#41416) * Resolve a number of extended test failures With this PR R extended tests should be all green with the exceptions of: * Two sanitizer jobs (test-fedora-r-clang-sanitizer, test-ubuntu-r-sanitizer) — which are being investigated / fixed in apache#41421 * Valgrind — I'm running one last run with a new suppression file. * Binary jobs — these work but fail at upload, see apache#41403 (comment) * Windows R Release — failing on main, apache#41398 ### Are these changes tested? By definition. ### Are there any user-facing changes? No. * GitHub Issue: apache#41402 Lead-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

jonkeane requested review from assignUser, kou and raulcd as code owners April 26, 2024 20:59

github-actions bot added the awaiting committer review Awaiting committer review label Apr 26, 2024

kou changed the title ~~GH-41402: [CI] [R] Update our backwards compatibility CI any other R 4.4 cleanups~~ GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups Apr 26, 2024

jonkeane requested review from paleolimbot and thisisnic as code owners April 26, 2024 21:18

github-actions bot added the Component: R label Apr 26, 2024

assignUser approved these changes Apr 27, 2024

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Apr 27, 2024

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 27, 2024

github-actions bot added awaiting change review Awaiting change review awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Apr 28, 2024

PR comments

8d91cab

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 28, 2024

oops, copy pasta

67941ba

assignUser reviewed Apr 29, 2024

View reviewed changes

dev/tasks/tasks.yml Outdated Show resolved Hide resolved

assignUser mentioned this pull request Apr 29, 2024

GH-40735: [Packaging][CentOS] Drop support for CentOS 7 #41395

Closed

Fix artifact name

8cad637

assignUser mentioned this pull request Apr 29, 2024

[R][CI] CRAN-style openssl not being picked up #41426

Closed

jonkeane merged commit 6eb0b37 into apache:main Apr 29, 2024
57 of 60 checks passed

jonkeane removed the awaiting change review Awaiting change review label Apr 29, 2024

jonkeane mentioned this pull request Apr 29, 2024

[CI][R]: Update our backwards compatibility CI any other R 4.4 cleanups #41402

Closed

This was referenced Apr 29, 2024

[R][CI] rhub/container CI follow ons #41450

Closed

GH-41450: [R][CI] rhub/container follow ons #41451

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403

GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403

jonkeane commented Apr 26, 2024 •

edited

Loading

github-actions bot commented Apr 26, 2024

jonkeane commented Apr 26, 2024

github-actions bot commented Apr 26, 2024

jonkeane commented Apr 26, 2024

github-actions bot commented Apr 26, 2024

assignUser left a comment

assignUser Apr 27, 2024

assignUser Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 28, 2024

jonkeane commented Apr 28, 2024

github-actions bot commented Apr 28, 2024

jonkeane commented Apr 28, 2024

github-actions bot commented Apr 28, 2024

assignUser commented Apr 29, 2024

assignUser commented Apr 29, 2024

jonkeane commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

assignUser commented Apr 29, 2024 •

edited

Loading

conbench-apache-arrow bot commented Apr 29, 2024

GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403

GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups #41403

Conversation

jonkeane commented Apr 26, 2024 • edited Loading

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

github-actions bot commented Apr 26, 2024

jonkeane commented Apr 26, 2024

github-actions bot commented Apr 26, 2024

jonkeane commented Apr 26, 2024

github-actions bot commented Apr 26, 2024

assignUser left a comment

Choose a reason for hiding this comment

assignUser Apr 27, 2024

Choose a reason for hiding this comment

assignUser Apr 27, 2024

Choose a reason for hiding this comment

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

github-actions bot commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 27, 2024

jonkeane commented Apr 28, 2024

jonkeane commented Apr 28, 2024

github-actions bot commented Apr 28, 2024

jonkeane commented Apr 28, 2024

github-actions bot commented Apr 28, 2024

assignUser commented Apr 29, 2024

assignUser commented Apr 29, 2024

jonkeane commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

assignUser commented Apr 29, 2024 • edited Loading

conbench-apache-arrow bot commented Apr 29, 2024

jonkeane commented Apr 26, 2024 •

edited

Loading

assignUser commented Apr 29, 2024 •

edited

Loading