Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Java][CI][Java-Jars][MacOS] C++ libraries for MacOS AARCH 64 #38076

Closed
davisusanibar opened this issue Oct 6, 2023 · 6 comments
Closed

[Java][CI][Java-Jars][MacOS] C++ libraries for MacOS AARCH 64 #38076

davisusanibar opened this issue Oct 6, 2023 · 6 comments
Assignees
@assignUser
Labels
Component: Continuous Integration Component: Java Priority: Blocker Marks a blocker for the release Type: bug
Milestone
14.0.0

Comments

@davisusanibar
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

There are errors on Java building C++ libraries for MacOS AARCH 64.

C++ libraries for MacOS AARCH 64: https://github.com/ursacomputing/crossbow/actions/runs/6403529915/job/17382218168
Machine name: 'm1mini02'
Error: No such keg: /opt/homebrew/Cellar/aws-sdk-cpp
Error: No such keg: /opt/homebrew/Cellar/llvm

There are more evidences of the error on other days on java-jars task at http://crossbow.voltrondata.com/

Component(s)

Continuous Integration, Java
@davisusanibar davisusanibar mentioned this issue Oct 6, 2023
Closed
@danepitkin danepitkin added the Priority: Blocker Marks a blocker for the release label Oct 6, 2023
@danepitkin danepitkin added this to the 14.0.0 milestone Oct 6, 2023
@danepitkin
Copy link
Contributor

danepitkin commented Oct 6, 2023 •
edited

Did these homebrew packages drop support for MacOS 10.13 by chance?

From the error log I see this

  env:
    MACOSX_DEPLOYMENT_TARGET: 10.13

..but this is for AARCH64.. could be irrelevant to the actual issue.

@danepitkin
Copy link
Contributor

danepitkin commented Oct 6, 2023 •
edited

Yeah, ignore what I said above.

The most recent MacOS version that support Apple Silicon was Big Sur (MacOS 11). Maybe the github runner m1mini02 is using this version and maybe support was recently dropped in homebrew for it? I haven't been able to verify yet.

@danepitkin
Copy link
Contributor

Confirmed, aws-sdk-cpp dropped Big Sur in Homebrew recently Homebrew/homebrew-core@1b84272

@danepitkin
Copy link
Contributor

And the m1mini02 is self-hosted https://github.com/apache/arrow/blob/main/dev/tasks/java-jars/github.yml#L82

@assignUser
Copy link
Member

Thanks for investigating. We will need to upgrade those machines.

@assignUser assignUser self-assigned this Oct 6, 2023
@davisusanibar
Copy link
Contributor Author

MacOS now run 13.6 and it is worked as expected at https://github.com/ursacomputing/crossbow/actions/runs/6403529915/job/17563133582

@LuciferYang LuciferYang mentioned this issue Nov 4, 2023
Closed
dongjoon-hyun pushed a commit to apache/spark that referenced this issue Nov 4, 2023
@LuciferYang @dongjoon-hyun
[SPARK-45781][BUILD] Upgrade Arrow to 14.0.0
749c79e 
### What changes were proposed in this pull request?
This pr upgrade Apache Arrow from 13.0.0 to 14.0.0.

### Why are the changes needed?
The Apache Arrow 14.0.0 release brings a number of enhancements and bug fixes.
‎
In terms of bug fixes, the release addresses several critical issues that were causing failures in integration jobs with Spark([GH-36332](apache/arrow#36332)) and problems with importing empty data arrays([GH-37056](apache/arrow#37056)). It also optimizes the process of appending variable length vectors([GH-37829](apache/arrow#37829)) and includes C++ libraries for MacOS AARCH 64 in Java-Jars([GH-38076](apache/arrow#38076)).
‎
The new features and improvements focus on enhancing the handling and manipulation of data. This includes the introduction of DefaultVectorComparators for large types([GH-25659](apache/arrow#25659)), support for extended expressions in ScannerBuilder([GH-34252](apache/arrow#34252)), and the exposure of the VectorAppender class([GH-37246](apache/arrow#37246)).
‎
The release also brings enhancements to the development and testing process, with the CI environment now using JDK 21([GH-36994](apache/arrow#36994)). In addition, the release introduces vector validation consistent with C++, ensuring consistency across different languages([GH-37702](apache/arrow#37702)).
‎
Furthermore, the usability of VarChar writers and binary writers has been improved with the addition of extra input methods([GH-37705](apache/arrow#37705)), and VarCharWriter now supports writing from `Text` and `String`([GH-37706](apache/arrow#37706)). The release also adds typed getters for StructVector, improving the ease of accessing data([GH-37863](apache/arrow#37863)).

The full release notes as follows:
- https://arrow.apache.org/release/14.0.0.html

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #43650 from LuciferYang/arrow-14.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@assignUser assignUser

Labels
Component: Continuous Integration Component: Java Priority: Blocker Marks a blocker for the release Type: bug
Projects
None yet
Milestone
14.0.0
Development

No branches or pull requests
3 participants
@davisusanibar @assignUser @danepitkin