GH-41450: [R][CI] rhub/container follow ons #41451

jonkeane · 2024-04-29T22:03:27Z

More CI changes:

GitHub Issue: [R][CI] rhub/container CI follow ons #41450 (specifically use the rhub containers approach for clang sanitizer, remove some of our work arounds)
Remove CentOS 7 CI support for R

jonkeane · 2024-04-29T22:03:49Z

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

github-actions · 2024-04-29T22:03:52Z

⚠️ GitHub issue #41450 has been automatically assigned in GitHub to PR creator.

github-actions · 2024-04-29T22:06:05Z

Revision: 6536162

Submitted crossbow builds: ursacomputing/crossbow @ actions-9240543e1c

Task	Status
test-r-rhub-debian-gcc-devel-lto-latest

jonkeane · 2024-04-29T22:12:52Z

dev/tasks/r/github.packages.yml

+          - {image: "rstudio/r-base:4.1-focal"}
+          - {image: "rstudio/r-base:4.2-jammy"}
          - {image: "rstudio/r-base:4.3-noble"}


These probably shouldn't be all ubuntus. Any suggestions on other images we should use? centos8? RHEL? debian?

I think rockylinux8 gets at the centos/RHEL line.

jonkeane · 2024-04-30T13:03:43Z

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

github-actions · 2024-04-30T13:06:10Z

Revision: 020f8e1

Submitted crossbow builds: ursacomputing/crossbow @ actions-4a7b53ff9c

Task	Status
test-r-rhub-debian-gcc-devel-lto-latest

…d too

jonkeane · 2024-04-30T22:10:26Z

@github-actions crossbow submit test-r-clang-sanitizer

github-actions · 2024-04-30T22:13:13Z

Revision: 3185879

Submitted crossbow builds: ursacomputing/crossbow @ actions-09f0dd36e9

Task	Status
test-r-clang-sanitizer

jonkeane · 2024-04-30T22:17:10Z

@github-actions crossbow submit test-r-clang-sanitizer

github-actions · 2024-04-30T22:19:25Z

Revision: 9664a4c

Submitted crossbow builds: ursacomputing/crossbow @ actions-6816ab39a7

Task	Status
test-r-clang-sanitizer

jonkeane · 2024-04-30T23:40:26Z

@github-actions crossbow submit test-r-clang-sanitizer

github-actions · 2024-04-30T23:42:44Z

Revision: cb4ecea

Submitted crossbow builds: ursacomputing/crossbow @ actions-1cccdf27a2

Task	Status
test-r-clang-sanitizer

jonkeane · 2024-05-01T01:36:25Z

@github-actions crossbow submit test-r-clang-sanitizer

github-actions · 2024-05-01T01:38:41Z

Revision: 9843835

Submitted crossbow builds: ursacomputing/crossbow @ actions-2d9e138827

Task	Status
test-r-clang-sanitizer

nealrichardson · 2024-05-10T14:23:34Z

r/vignettes/install.Rmd

@@ -28,35 +28,6 @@ For `gcc`, this generally means version 7 or newer. Most contemporary Linux
 distributions have a new enough compiler; however, CentOS 7 is a notable
 exception, as it ships with gcc 4.8.

-If you are on CentOS 7, to build arrow you will need to install a newer `devtoolset`, and you'll need to update R's Makevars to define the `CXX17` variables. This script installs `devtoolset-8` and configures R to be able to use C++17:


The note in the paragraph above also mentions CentOS 7.

Maybe we should keep this discussion and say something like, we don't officially support CentOS 7 anymore, but if you want to try building on it, you'll need to:

nods ok yeah, I also like that it leaves a bit of a more obvious breadcrumb trail for folks debugging. I know it's in the git history, but I don't think folks would know to look here in that history necessarily to find out "what do I do with non-standard compiler setups?"

nealrichardson · 2024-05-10T14:24:38Z

r/vignettes/install.Rmd

@@ -517,10 +488,6 @@ The install script should work everywhere, so if libarrow fails to compile,
 please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues)
 so that we can improve the script.

-### Known installation issues


Similarly, maybe this should stick around in some form since we direct people here for troubleshooting.

nealrichardson · 2024-05-10T14:25:39Z

@github-actions crossbow submit -g r

github-actions · 2024-05-10T14:26:20Z

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/9033697943

nealrichardson · 2024-05-10T14:26:27Z

Remove CentOS 7 CI support for R

To be clear, it seems we are still building libarrow binaries on CentOS 7, at least as of this PR.

jonkeane · 2024-05-10T14:34:07Z

@github-actions crossbow submit -g r

github-actions · 2024-05-10T14:36:53Z

Revision: 9843835

Submitted crossbow builds: ursacomputing/crossbow @ actions-3ba4bec650

Task	Status
r-binary-packages
test-r-arrow-backwards-compatibility
test-r-clang-sanitizer
test-r-depsource-bundled
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-valgrind
test-r-minimal-build
test-r-offline-maximal
test-r-offline-minimal
test-r-rhub-debian-gcc-devel-lto-latest
test-r-rhub-debian-gcc-release-custom-ccache
test-r-rhub-ubuntu-release-latest
test-r-rocker-r-ver-latest
test-r-rstudio-r-base-4.1-opensuse153
test-r-rstudio-r-base-4.2-focal
test-r-ubuntu-22.04
test-r-versions
test-ubuntu-r-sanitizer

nealrichardson · 2024-05-10T15:42:41Z

@github-actions crossbow submit test-r-arrow-backwards-compatibility

github-actions · 2024-05-10T15:44:59Z

Revision: 9843835

Submitted crossbow builds: ursacomputing/crossbow @ actions-76bf294813

Task	Status
test-r-arrow-backwards-compatibility

assignUser · 2024-05-12T03:27:52Z

To be clear, it seems we are still building libarrow binaries on CentOS 7, at least as of this PR.

This brings up the whole support thing again... we use centos7 for the old glibc right? Do we just bump to the next non-eol alma/rocky version?

conbench-apache-arrow · 2024-05-12T15:21:41Z

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit 6d03215.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.

More CI changes: * GitHub Issue: apache#41450 (specifically use the rhub containers approach for clang sanitizer, remove some of our work arounds) * Remove CentOS 7 CI support for R Authored-by: Jonathan Keane <jkeane@gmail.com> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

kou · 2024-05-15T08:05:07Z

ci/scripts/java_jni_manylinux_build.sh

@@ -35,9 +35,6 @@ echo "=== Clear output directories and leftovers ==="
 rm -rf ${build_dir}

 echo "=== Building Arrow C++ libraries ==="
-devtoolset_version=$(rpm -qa "devtoolset-*-gcc" --queryformat %{VERSION} | \
-                       grep -o "^[0-9]*")
-devtoolset_include_cpp="/opt/rh/devtoolset-${devtoolset_version}/root/usr/include/c++/${devtoolset_version}"


@jonkeane Is it expected change?

Aaaah, I might have been overly aggressive in removing from this file.

I thought we only used the devtoolset_version on the R CentOS 7 builds and that other places we added that to were borrowing from those and no longer necessary. But if we need to add it back here that's totally fine. Sorry for not confirming that and the hassle.

No problem. :-)
Thanks for confirming this.

…41661) ### Rationale for this change Because #41451 removed devtoolset related flags unexpectedly. ### What changes are included in this PR? Restore devtoolset related flags. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #41660 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

More CI changes: * GitHub Issue: apache#41450 (specifically use the rhub containers approach for clang sanitizer, remove some of our work arounds) * Remove CentOS 7 CI support for R Authored-by: Jonathan Keane <jkeane@gmail.com> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

…AGS (apache#41661) ### Rationale for this change Because apache#41451 removed devtoolset related flags unexpectedly. ### What changes are included in this PR? Restore devtoolset related flags. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#41660 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

More CI changes: * GitHub Issue: apache#41450 (specifically use the rhub containers approach for clang sanitizer, remove some of our work arounds) * Remove CentOS 7 CI support for R Authored-by: Jonathan Keane <jkeane@gmail.com> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

jonkeane requested review from assignUser, kou and raulcd as code owners April 29, 2024 22:03

github-actions bot added Component: R awaiting committer review Awaiting committer review labels Apr 29, 2024

jonkeane requested review from paleolimbot and thisisnic as code owners April 29, 2024 22:11

jonkeane commented Apr 29, 2024

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Apr 29, 2024

jonkeane changed the title ~~GH-41450: [R][CI]~~ GH-41450: [R][CI] rhub/container follow ons Apr 29, 2024

jonkeane added 2 commits April 30, 2024 17:05

remove rhub/containers work around and test skip on devel that's fixe…

1f695df

…d too

Remove centos7 and devtoolset hacks for R

ad7b233

jonkeane force-pushed the CI_follow_on branch from 020f8e1 to ad7b233 Compare April 30, 2024 22:06

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 30, 2024

move to rhub/asan

3185879

s/RDsan/R/

9664a4c

Copy old rhub sanitizer options

cb4ecea

jonkeane requested a review from nealrichardson May 10, 2024 12:50

nealrichardson reviewed May 10, 2024

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels May 10, 2024

nealrichardson mentioned this pull request May 10, 2024

[Dev] Pull request review bot seems confused about who is a committer #41432

Open

jonkeane merged commit 6d03215 into apache:main May 12, 2024
60 of 65 checks passed

jonkeane removed the awaiting changes Awaiting changes label May 12, 2024

jonkeane mentioned this pull request May 12, 2024

[R][CI] rhub/container CI follow ons #41450

Closed

jonkeane deleted the CI_follow_on branch May 13, 2024 13:08

This was referenced May 15, 2024

[CI][Java] "AMD64 manylinux2014 Java JNI" is failed with nonexistent include path #41660

Closed

GH-41660: [CI][Java] Restore devtoolset relatead GANDIVA_CXX_FLAGS #41661

Merged

kou reviewed May 15, 2024

View reviewed changes

github-actions bot added the awaiting changes Awaiting changes label May 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-41450: [R][CI] rhub/container follow ons #41451

GH-41450: [R][CI] rhub/container follow ons #41451

jonkeane commented Apr 29, 2024 •

edited

Loading

jonkeane commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

jonkeane Apr 29, 2024

nealrichardson May 10, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented May 1, 2024

github-actions bot commented May 1, 2024

nealrichardson May 10, 2024

jonkeane May 10, 2024

nealrichardson May 10, 2024

nealrichardson commented May 10, 2024

github-actions bot commented May 10, 2024

nealrichardson commented May 10, 2024

jonkeane commented May 10, 2024

github-actions bot commented May 10, 2024

nealrichardson commented May 10, 2024

github-actions bot commented May 10, 2024

assignUser commented May 12, 2024

conbench-apache-arrow bot commented May 12, 2024

kou May 15, 2024

jonkeane May 15, 2024

kou May 15, 2024

GH-41450: [R][CI] rhub/container follow ons #41451

GH-41450: [R][CI] rhub/container follow ons #41451

Conversation

jonkeane commented Apr 29, 2024 • edited Loading

jonkeane commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented Apr 30, 2024

github-actions bot commented Apr 30, 2024

jonkeane commented May 1, 2024

github-actions bot commented May 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nealrichardson commented May 10, 2024

github-actions bot commented May 10, 2024

nealrichardson commented May 10, 2024

jonkeane commented May 10, 2024

github-actions bot commented May 10, 2024

nealrichardson commented May 10, 2024

github-actions bot commented May 10, 2024

assignUser commented May 12, 2024

conbench-apache-arrow bot commented May 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonkeane commented Apr 29, 2024 •

edited

Loading