ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] #12157

jonkeane · 2022-01-14T17:04:41Z

This is not totally resolved (I'm getting illegal op codes on the macos 11 job). And might not be the right approach regardless. The crux of this is that our (source) nightly builds bundle the apache-arrow.rb formula, and that formula works with autobrew/homebrew-core which isn't compatible with macos 11. (Of course, we need to keep apache-arrow.rb around and use autobrew/homebrew-core for macos 10.11/10.13 which cran uses!) The approach here is to also bundle (and test!) the formula that is in autobrew/homebrew-cran on a modern macos to ensure that that works.

These should only really matter for nightly (source) builds (or someone who otherwise tries to use these formulae) since releases will use the autobrew/scripts script without using brew at all.

I have confirmed that without the formula from autobrew/homebrew-cran, running on macos 11 runners fails with:

==> Installing thrift dependency: boost
==> Downloading https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
curl: (22) The requested URL returned error: 502 Bad Gateway
Error: Error: An exception occured within a child process:
  DownloadError: Failed to download resource "boost"
Download failed: https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2

==> Installing dependencies for apache-arrow: boost, thrift, zstd
==> Installing apache-arrow dependency: boost
/usr/bin/sandbox-exec -f /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/homebrew20220113-77625-5t1tdb.sb nice /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby -W0 -I /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/ruby-macho-2.0.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/plist-3.4.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/backports-3.11.4/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/activesupport-5.2.1/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/tzinfo-1.2.5/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/thread_safe-0.3.6/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/minitest-5.11.3/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/i18n-1.1.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/gems/2.3.0/gems/did_you_mean-1.0.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew -- /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/build.rb /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Taps/autobrew/homebrew-core/Formula/boost.rb --verbose
==> Downloading https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
/usr/bin/curl -q --show-error --insecure --user-agent Homebrew/1.X.Y\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 11.6.2\)\ curl/7.64.1 --fail --silent --location --remote-time --continue-at 0 --output /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/downloads/d08cab6533547504db34623190d93eb217086eb76b2f39ad79d3e9301b40b41e--boost_1_67_0.tar.bz2.incomplete https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
curl: (22) The requested URL returned error: 502 Bad Gateway
Error: Error: An exception occured within a child process:
  DownloadError: Failed to download resource "boost"
Download failed: https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2

created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-auth.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-cal.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-common.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-compression.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-event-stream.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-http.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-io.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-mqtt.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-s3.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-checksums.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-cognito-identity.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-config.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-core.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-identity-management.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-s3.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-sts.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-transfer.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-crt-cpp.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewtesting-resources.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewy.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewlz4.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewcrypto.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewssl.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewsnappy.a
------------------------- NOTE ---------------------------
There was an issue preparing the Arrow C++ libraries.
See https://arrow.apache.org/docs/r/articles/install.html
---------------------------------------------------------
ERROR: configuration failed for package ‘arrow’
* removing ‘/Users/runner/work/crossbow/crossbow/arrow/r/check/arrow.Rcheck/arrow’

1 error ✖ | 0 warnings ✔ | 0 notes ✔

Note: this also does not (yet) get the formulae in sync or add a check that they are still in sync with the homebrew / autobrew repos.

jonkeane · 2022-01-14T17:05:37Z

@github-actions crossbow submit homebrew-r-autobrew

github-actions · 2022-01-14T17:07:39Z

https://issues.apache.org/jira/browse/ARROW-12590

github-actions · 2022-01-14T17:07:40Z

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

github-actions · 2022-01-14T17:09:32Z

Revision: 5054ddc

Submitted crossbow builds: ursacomputing/crossbow @ actions-1397

Task	Status
homebrew-r-autobrew

jonkeane · 2022-02-04T15:35:36Z

@github-actions crossbow submit homebrew-r-autobrew

github-actions · 2022-02-04T15:36:54Z

Revision: b26b082

Submitted crossbow builds: ursacomputing/crossbow @ actions-1586

Task	Status
homebrew-r-autobrew

jonkeane · 2022-02-07T20:09:57Z

@github-actions crossbow submit homebrew-r-autobrew

github-actions · 2022-02-07T20:11:12Z

Revision: 1592565

Submitted crossbow builds: ursacomputing/crossbow @ actions-1601

Task	Status
homebrew-r-autobrew

…nt updates (#13769) Building on #12157 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

…nt updates (apache#13769) Building on apache#12157 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

paleolimbot · 2023-01-05T20:29:44Z

I came across this in a "browsing old PRs tagged with R" mission...is this still relevant?

jonkeane · 2023-01-05T21:38:32Z

this also does not (yet) get the formulae in sync or add a check that they are still in sync with the homebrew / autobrew repos.

Getting to a point where ^^^ happens would be very helpful for keeping up with changes downstream + making sure we don't break anything. And part of that would be to do many of the updates here (if they are still in the downstream sources...). But I don't think I'll be able to get back to this to do so, so I'll close it out — but we should still come back to that issue

github-actions bot added the Component: R label Jan 14, 2022

jonkeane added 7 commits February 4, 2022 09:33

run homebrew-r-autobrew on both macos versions

faf34e4

oops

301a5f0

first steps towards also testing autobrew with modern architectures

e2c0341

oops brew -> autobrew

533cbd1

try Homebrew/homebrew-core#80501

f47f01f

some prints

f7e2a5f

arrow_r_dev for more verbose logs

b26b082

jonkeane force-pushed the macos-autobrew branch from 5054ddc to b26b082 Compare February 4, 2022 15:35

jonkeane added 2 commits February 4, 2022 16:22

cleanup

d023d15

remove env runtime flag

1592565

assignUser mentioned this pull request Aug 1, 2022

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates #13769

Merged

jonkeane closed this Jan 5, 2023

asfimport mentioned this pull request Jan 5, 2023

[C++][R] Update copies of Homebrew files to reflect recent updates #28346

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] #12157

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] #12157

jonkeane commented Jan 14, 2022 •

edited

Loading

jonkeane commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

jonkeane commented Feb 4, 2022

github-actions bot commented Feb 4, 2022

jonkeane commented Feb 7, 2022

github-actions bot commented Feb 7, 2022

paleolimbot commented Jan 5, 2023

jonkeane commented Jan 5, 2023

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] #12157

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] #12157

Conversation

jonkeane commented Jan 14, 2022 • edited Loading

jonkeane commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

github-actions bot commented Jan 14, 2022

jonkeane commented Feb 4, 2022

github-actions bot commented Feb 4, 2022

jonkeane commented Feb 7, 2022

github-actions bot commented Feb 7, 2022

paleolimbot commented Jan 5, 2023

jonkeane commented Jan 5, 2023

jonkeane commented Jan 14, 2022 •

edited

Loading