Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++][R] Update copies of Homebrew files to reflect recent updates #28346

Closed
asfimport opened this issue Apr 28, 2021 · 10 comments
Closed

[C++][R] Update copies of Homebrew files to reflect recent updates #28346

asfimport opened this issue Apr 28, 2021 · 10 comments

Comments

@asfimport
Copy link
Collaborator

asfimport commented Apr 28, 2021

Our copies of the Homebrew formulae at https://github.com/apache/arrow/tree/master/dev/tasks/homebrew-formulae have drifted out of sync with what's currently in https://github.com/Homebrew/homebrew-core/tree/master/Formula and https://github.com/autobrew/homebrew-core/blob/master/Formula. Get them back in sync and consider automating some method of checking that they are in sync, e.g. by failing the homebrew-cpp and 
homebrew-r-autobrew nightly tests if our copies don't match what's in the Homebrew and autobrew repos (but only if there were changes there that weren't made in our repo, and not the inverse).

Update the instructions at 
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingHomebrewpackages as needed.

Reporter: Ian Cook / @ianmcook
Assignee: Jacob Wujciak / @assignUser

Related issues:

PRs and other links:

Note: This issue was originally created as ARROW-12590. Please see the migration documentation for further details.

@asfimport
Copy link
Collaborator Author

Ian Cook / @ianmcook:
Also consider updating the nightly tests to run on Big Sur or on multiple macOS versions.

@asfimport
Copy link
Collaborator Author

Jonathan Keane / @jonkeane:
From: autobrew/homebrew-core#89 (comment)

However the underlying problem here may be that we need to update the script (https://github.com/apache/arrow/blob/HEAD/r/tools/autobrew) in apache arrow that uses this repo to build a version of apache-arrow. The current repo is mostly intended to build libarrow for macos 10.11 and 10.13. For newer versions of macos we use this tap to build arrow: https://github.com/autobrew/homebrew-cran

Also, the CRAN package now uses a much simpler script to fetch a suitable binary of libarrow (https://github.com/autobrew/scripts/blob/master/apache-arrow), it no longer has to bootstrap a homebrew installation like we did early on.

@asfimport
Copy link
Collaborator Author

Neal Richardson / @nealrichardson:
It no longer has to bootstrap homebrew because that's done out of band, like the rtools-packages builds. So homebrew hasn't been removed, it's just been moved IIUC.

@asfimport
Copy link
Collaborator Author

Jonathan Keane / @jonkeane:
From Neal:

We could but we still need to test the homebrew/autobrew build. Also keep in mind that CRAN checks use 10.13.

Yes, we definitely still need to test this, I thought (though I *have not* dug into this) the new autobrew scripts did the right thing for each version (Jeroen has a few different repos and his comment made it seem like our setup was always trying to pull out of backports, when it should only do that on old macOSes, but pull out of one of the newer ones on newer macOSes)

@asfimport
Copy link
Collaborator Author

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

@asfimport
Copy link
Collaborator Author

Jacob Wujciak / @assignUser:
@jonkeane Just so I understand correctly: As far as I see we have added dependencies to our version of the formula that are missing from the upstream version on the other hand the upstream version has update the bottle tag/sha which is likely why we are having issues with that now. So these changes should clearly be synced in both directions.

There are some other changes that should obviously be excluded (download url/version + sha) but also some where I am unsure if we should also sync them down:

  • a patch step to change the mimalloc version in versions.txt
  • an addition to the test step (running the cpp tests?)

@asfimport
Copy link
Collaborator Author

Jonathan Keane / @jonkeane:
Agreed with syncing (and the original intent of this ticket was basically to find a way to detect if and when this happens in order to alert us about it). It is ok that the autobrew and the homebrew formulae are different (since in the newest versions of the autobrew setup, if we are on a modern enough system we just use brew).

If I'm remembering correctly, https://github.com/apache/arrow/pull/12157/files#diff-4b112dbca2ece7c78e15eb8aff3218e21dd6f4b1fab7cfc9182830488f68ca58R22-R30 was basically the operative code that fixes this. If I were you, I would take the commits on my branch there and create a new branch and push forward with that since it will let you run it in CI. Though the R tests will probably segfault with the simd issue in ARROW-15678. Maybe that's fine (since it's "only" a limited number of computers that this happens on — just so happens the GH runners are one of those, apparently) or maybe we'll need to actually resolve ARROW-15678?

@asfimport
Copy link
Collaborator Author

Jacob Wujciak / @assignUser:
Ok.

Though I thought that for [ARROW-15678] we had a workaround (setting -O2) in place that should prevent the segfault?

@asfimport
Copy link
Collaborator Author

Jonathan Keane / @jonkeane:
Yeah, that should work until the homer maintainers decide to pull it out

@asfimport
Copy link
Collaborator Author

Kouhei Sutou / @kou:
Issue resolved by pull request 13769
#13769

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants