[R][Docs] Update documentation about macOS build system #37945
Assignees
Labels
Component: Documentation
Component: R
Priority: Blocker
Marks a blocker for the release
Type: enhancement
Milestone
Comments
paleolimbot
pushed a commit
that referenced
this issue
Oct 5, 2023
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
|
This is definitely a blocker for 14.0.0 yeah? |
paleolimbot
added a commit
that referenced
this issue
Oct 13, 2023
### Rationale for this change Several PRs over the last few months have update the build system to be more friendly for developers. During this process it has also come to light that we haven't supported the Windows development setup documented here since R 4.1 (released in spring 2021). I had to remove Windows from the test-r-devdocs job because the approach used there was not compatible with the `setup-r@ v2` action, and the job was failing with the `@ v1` action. ### What changes are included in this PR? - Updated the sections on using pre-built static libraries and bundled builds - Removed the Windows section regarding the bundled build. This section would need rewriting to support the last two minor releases of R but in the meantime I think it is mostly confusing. ### Are these changes tested? They are documentation changes. They are also slightly optimisitc: we can fix problems with the developer setup incrementally between releases, but it's more difficult to update our documentation. This PR documents the intended behaviour after #38236 . ### Are there any user-facing changes? No. * Closes: #37945 Lead-authored-by: Dewey Dunnington <dewey@voltrondata.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
raulcd
pushed a commit
that referenced
this issue
Oct 13, 2023
### Rationale for this change Several PRs over the last few months have update the build system to be more friendly for developers. During this process it has also come to light that we haven't supported the Windows development setup documented here since R 4.1 (released in spring 2021). I had to remove Windows from the test-r-devdocs job because the approach used there was not compatible with the `setup-r@ v2` action, and the job was failing with the `@ v1` action. ### What changes are included in this PR? - Updated the sections on using pre-built static libraries and bundled builds - Removed the Windows section regarding the bundled build. This section would need rewriting to support the last two minor releases of R but in the meantime I think it is mostly confusing. ### Are these changes tested? They are documentation changes. They are also slightly optimisitc: we can fix problems with the developer setup incrementally between releases, but it's more difficult to update our documentation. This PR documents the intended behaviour after #38236 . ### Are there any user-facing changes? No. * Closes: #37945 Lead-authored-by: Dewey Dunnington <dewey@voltrondata.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
JerAguilon
pushed a commit
to JerAguilon/arrow
that referenced
this issue
Oct 23, 2023
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
JerAguilon
pushed a commit
to JerAguilon/arrow
that referenced
this issue
Oct 23, 2023
### Rationale for this change Several PRs over the last few months have update the build system to be more friendly for developers. During this process it has also come to light that we haven't supported the Windows development setup documented here since R 4.1 (released in spring 2021). I had to remove Windows from the test-r-devdocs job because the approach used there was not compatible with the `setup-r@ v2` action, and the job was failing with the `@ v1` action. ### What changes are included in this PR? - Updated the sections on using pre-built static libraries and bundled builds - Removed the Windows section regarding the bundled build. This section would need rewriting to support the last two minor releases of R but in the meantime I think it is mostly confusing. ### Are these changes tested? They are documentation changes. They are also slightly optimisitc: we can fix problems with the developer setup incrementally between releases, but it's more difficult to update our documentation. This PR documents the intended behaviour after apache#38236 . ### Are there any user-facing changes? No. * Closes: apache#37945 Lead-authored-by: Dewey Dunnington <dewey@voltrondata.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this issue
Nov 13, 2023
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this issue
Nov 13, 2023
### Rationale for this change Several PRs over the last few months have update the build system to be more friendly for developers. During this process it has also come to light that we haven't supported the Windows development setup documented here since R 4.1 (released in spring 2021). I had to remove Windows from the test-r-devdocs job because the approach used there was not compatible with the `setup-r@ v2` action, and the job was failing with the `@ v1` action. ### What changes are included in this PR? - Updated the sections on using pre-built static libraries and bundled builds - Removed the Windows section regarding the bundled build. This section would need rewriting to support the last two minor releases of R but in the meantime I think it is mostly confusing. ### Are these changes tested? They are documentation changes. They are also slightly optimisitc: we can fix problems with the developer setup incrementally between releases, but it's more difficult to update our documentation. This PR documents the intended behaviour after apache#38236 . ### Are there any user-facing changes? No. * Closes: apache#37945 Lead-authored-by: Dewey Dunnington <dewey@voltrondata.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this issue
Feb 19, 2024
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this issue
Feb 19, 2024
### Rationale for this change Several PRs over the last few months have update the build system to be more friendly for developers. During this process it has also come to light that we haven't supported the Windows development setup documented here since R 4.1 (released in spring 2021). I had to remove Windows from the test-r-devdocs job because the approach used there was not compatible with the `setup-r@ v2` action, and the job was failing with the `@ v1` action. ### What changes are included in this PR? - Updated the sections on using pre-built static libraries and bundled builds - Removed the Windows section regarding the bundled build. This section would need rewriting to support the last two minor releases of R but in the meantime I think it is mostly confusing. ### Are these changes tested? They are documentation changes. They are also slightly optimisitc: we can fix problems with the developer setup incrementally between releases, but it's more difficult to update our documentation. This PR documents the intended behaviour after apache#38236 . ### Are there any user-facing changes? No. * Closes: apache#37945 Lead-authored-by: Dewey Dunnington <dewey@voltrondata.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
#37923 overhauls the macOS build system. We should make sure that the documentation reflects the changes once we honed in on the right default behavior and other details. This should include function docstrings (
install-arrow) and similar.Component(s)
Documentation, R
The text was updated successfully, but these errors were encountered: