Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[R] Build linux binaries on older image (like manylinux2014) #32292

Closed
asfimport opened this issue Jul 5, 2022 · 1 comment · Fixed by #34048
Closed

[R] Build linux binaries on older image (like manylinux2014) #32292

asfimport opened this issue Jul 5, 2022 · 1 comment · Fixed by #34048
Assignees
@kou
Labels
Component: Packaging Component: R Type: enhancement
Milestone
12.0.0

Comments

@asfimport
Copy link
Collaborator

asfimport commented Jul 5, 2022 •
edited
Loading

ARROW-16752 observed that even with newer compilers installed on centos 7, you can't use binaries built on ubuntu 18.04 because ubuntu 18.04 has glibc 2.27 but centos 7 only has 2.17. But if we built the binaries on centos 7 with devtoolset-7 or 8 or something, all features could compile and we'd work with older glibc.

Things built against older glibc are guaranteed to work with newer versions, and you can't just upgrade glibc because it would break the system. So for maximum compatibility, build with the oldest glibc. This strategy is like how they python manylinux standards work (IIUC).

Reporter: Neal Richardson / @nealrichardson

Related issues:

Note: This issue was originally created as ARROW-16976. Please see the migration documentation for further details.
@asfimport asfimport mentioned this issue Jan 11, 2023
Closed
@kou
Copy link
Member

kou commented Feb 4, 2023

I tried to use CentOS 7 binaries with Ubuntu 18.04 in #34020 (comment) but we can't do it:

https://github.com/ursacomputing/crossbow/actions/runs/4089420508/jobs/7052165676#step:12:265

 Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so':
  /usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZNK5arrow8DataType18ComputeFingerprintB5cxx11Ev

Because CentOS 7 + devtoolset doesn't use cxx11 ABI but Ubuntu 18.04 requires cxx11 ABI.

Can we close this?

@nealrichardson nealrichardson mentioned this issue Feb 5, 2023
Merged
kou added a commit to kou/arrow that referenced this issue Feb 6, 2023
@kou
apacheGH-32292: [R][Packaging] Use binaries build on CentOS 7 for Ubu…
9d58e2c 
…ntu < 22.04

Binaries build on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we
can't use them on Ubuntu 20.04 because Ubuntu 20.04 uses
`_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R
package should also uses `_GLIBCXX_USE_CXX11_ABI=0`.

This changes adds `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in
`arrow.pc`. our R package uses `Cflags` in `arrow.pc`. So users don't
need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.
@github-actions github-actions bot mentioned this issue Feb 6, 2023
Merged
kou added a commit to kou/arrow that referenced this issue Feb 9, 2023
@kou
apacheGH-32292: [R][Packaging] Use binaries build on CentOS 7 for Ubu…
3a11325 
…ntu < 22.04

Binaries build on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we
can't use them on Ubuntu 20.04 because Ubuntu 20.04 uses
`_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R
package should also uses `_GLIBCXX_USE_CXX11_ABI=0`.

This changes adds `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in
`arrow.pc`. our R package uses `Cflags` in `arrow.pc`. So users don't
need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.
kou added a commit to kou/arrow that referenced this issue Mar 18, 2023
@kou
apacheGH-32292: [R][Packaging] Use binaries build on CentOS 7 for Ubu…
20794a6 
…ntu < 22.04

Binaries build on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we
can't use them on Ubuntu 20.04 because Ubuntu 20.04 uses
`_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R
package should also uses `_GLIBCXX_USE_CXX11_ABI=0`.

This changes adds `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in
`arrow.pc`. our R package uses `Cflags` in `arrow.pc`. So users don't
need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.
kou added a commit to kou/arrow that referenced this issue Mar 21, 2023
@kou
apacheGH-32292: [R][Packaging] Use binaries build on CentOS 7 for Ubu…
93cd20d 
…ntu < 22.04

Binaries build on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we
can't use them on Ubuntu 20.04 because Ubuntu 20.04 uses
`_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R
package should also uses `_GLIBCXX_USE_CXX11_ABI=0`.

This changes adds `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in
`arrow.pc`. our R package uses `Cflags` in `arrow.pc`. So users don't
need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.
assignUser pushed a commit that referenced this issue Mar 22, 2023
@kou
GH-32292: [R][Packaging] Use binaries built on CentOS 7 for Ubuntu < …
9288275 
…22.04 (#34048)

### Rationale for this change

Binaries built on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we can't use them on Ubuntu 20.04 because
Ubuntu 20.04 uses `_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R package must use
`_GLIBCXX_USE_CXX11_ABI=0` too.

There is another problem for Ubuntu 20.04. CentOS 7 uses OpenSSL 1.0 but Ubuntu 20.04 uses OpenSSL 1.1.
OpenSSL 1.0 and 1.1 are incompatible. So we can't use binaries built on CentOS 7. We need binaries for OpenSSL
1.0 and OpenSSL 1.1. We can't use the same binaries on CentOS 7 (OpenSSL 1.0) and Ubuntu 20.04 (OpenSSL 1.1).

### What changes are included in this PR?

This changes add `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in `arrow.pc`. Our R package uses `Cflags` in
`arrow.pc`. So users don't need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.

This changes replace binaries built on Ubuntu 18.04 with binaries built on CentOS 7 with OpenSSL 1.1.
(The "ubuntu-18.04" binaries are replaced with the "linux-openssl-1.1" binaries.)

The "centos-7" binaries are renamed to the "linux-openssl-1.0" binaries.

The "ubuntu-22.04" binaries are renamed to the "linux-openssl-3.0" binaries.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.

* Closes: GH-33091
* Closes: #32292

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
@assignUser assignUser added this to the 12.0.0 milestone Mar 22, 2023
rtpsw pushed a commit to rtpsw/arrow that referenced this issue Mar 27, 2023
@kou @rtpsw
apacheGH-32292: [R][Packaging] Use binaries built on CentOS 7 for Ubu…
daf71fe 
…ntu < 22.04 (apache#34048)

### Rationale for this change

Binaries built on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we can't use them on Ubuntu 20.04 because
Ubuntu 20.04 uses `_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R package must use
`_GLIBCXX_USE_CXX11_ABI=0` too.

There is another problem for Ubuntu 20.04. CentOS 7 uses OpenSSL 1.0 but Ubuntu 20.04 uses OpenSSL 1.1.
OpenSSL 1.0 and 1.1 are incompatible. So we can't use binaries built on CentOS 7. We need binaries for OpenSSL
1.0 and OpenSSL 1.1. We can't use the same binaries on CentOS 7 (OpenSSL 1.0) and Ubuntu 20.04 (OpenSSL 1.1).

### What changes are included in this PR?

This changes add `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in `arrow.pc`. Our R package uses `Cflags` in
`arrow.pc`. So users don't need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.

This changes replace binaries built on Ubuntu 18.04 with binaries built on CentOS 7 with OpenSSL 1.1.
(The "ubuntu-18.04" binaries are replaced with the "linux-openssl-1.1" binaries.)

The "centos-7" binaries are renamed to the "linux-openssl-1.0" binaries.

The "ubuntu-22.04" binaries are renamed to the "linux-openssl-3.0" binaries.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.

* Closes: apacheGH-33091
* Closes: apache#32292

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
@kou @ArgusLi
apacheGH-32292: [R][Packaging] Use binaries built on CentOS 7 for Ubu…
b5fcdb5 
…ntu < 22.04 (apache#34048)

### Rationale for this change

Binaries built on CentOS 7 uses `_GLIBCXX_USE_CXX11_ABI=0`. So we can't use them on Ubuntu 20.04 because
Ubuntu 20.04 uses `_GLIBCXX_USE_CXX11_ABI=1`. If we use them on Ubuntu 20.04, our R package must use
`_GLIBCXX_USE_CXX11_ABI=0` too.

There is another problem for Ubuntu 20.04. CentOS 7 uses OpenSSL 1.0 but Ubuntu 20.04 uses OpenSSL 1.1.
OpenSSL 1.0 and 1.1 are incompatible. So we can't use binaries built on CentOS 7. We need binaries for OpenSSL
1.0 and OpenSSL 1.1. We can't use the same binaries on CentOS 7 (OpenSSL 1.0) and Ubuntu 20.04 (OpenSSL 1.1).

### What changes are included in this PR?

This changes add `-D_GLIBCXX_USE_CXX11_ABI=0` to `Cflags` in `arrow.pc`. Our R package uses `Cflags` in
`arrow.pc`. So users don't need to specify `-D_GLIBCXX_USE_CXX11_ABI=0` explicitly.

This changes replace binaries built on Ubuntu 18.04 with binaries built on CentOS 7 with OpenSSL 1.1.
(The "ubuntu-18.04" binaries are replaced with the "linux-openssl-1.1" binaries.)

The "centos-7" binaries are renamed to the "linux-openssl-1.0" binaries.

The "ubuntu-22.04" binaries are renamed to the "linux-openssl-3.0" binaries.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.

* Closes: apacheGH-33091
* Closes: apache#32292

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kou kou

Labels
Component: Packaging Component: R Type: enhancement
Projects
None yet
Milestone
12.0.0
Development

Successfully merging a pull request may close this issue.

GH-32292: [R][Packaging] Use binaries built on CentOS 7 for Ubuntu < 22.04 kou/arrow
3 participants
@kou @assignUser @asfimport