Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llvmPackages_17: init #258614

Merged
merged 1 commit into from
Dec 8, 2023
Merged

llvmPackages_17: init #258614

merged 1 commit into from
Dec 8, 2023

Conversation

RossComputerGuy
Copy link
Member

Description of changes

Adds llvm v17, fixes #257654. Will be opened as a draft as work is being done.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@RossComputerGuy
Copy link
Member Author

So far, this is what I've gotten. Anything that isn't mentioned with an error is because I wasn't able to compile due to a dependency error.

Builds & Works

  • llvm
  • clang
  • compiler-rt
  • openmp
  • lld

Broken

  • libunwind
  • libcxxabi
  • libcxx
  • lldb

libunwind

       last 10 log lines:
       > 1 error generated.
       > [12/19] Building CXX object libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.o
       > FAILED: libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.o
       > /nix/store/6pc02mwflca13bhcr7wks65cvrhxfrpd-clang-wrapper-17.0.1/bin/clang++ -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBUNWIND_LINK_DL_LIB -D_LIBUNWIND_LINK_PTHREAD_LIB -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/build/libunwind-src-17.0.1/libunwind/include -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -std=c++11 -Werror=return-type -funwind-tables -nostdinc++ -D_DEBUG -UNDEBUG -D_LIBUNWIND_IS_NATIVE_ONLY -Wall -Wextra -Wnewline-eof -Wshadow -Wwrite-strings -Wno-unused-parameter -Wno-long-long -Werror=return-type -Wextra-semi -Wundef -Wunused-template -Wformat-nonliteral -Wno-user-defined-literals -Wno-covered-switch-default -Wno-suggest-override -Wno-error -pedantic -fno-rtti  -fstrict-aliasing -fno-exceptions -fno-rtti -MD -MT libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.o -MF libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.o.d -o libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.o -c /build/libunwind-src-17.0.1/libunwind/src/libunwind.cpp
       > In file included from /build/libunwind-src-17.0.1/libunwind/src/libunwind.cpp:12:
       > /build/libunwind-src-17.0.1/libunwind/include/libunwind.h:19:10: fatal error: 'stddef.h' file not found
       >    19 | #include <stddef.h>
       >       |          ^~~~~~~~~~
       > 1 error generated.
       > ninja: build stopped: subcommand failed.
       For full logs, run 'nix log /nix/store/wkc3pvlhd7ll2mi1645pn6nzjjlima7d-libunwind-17.0.1.drv'.

lldb

Tried using the git version of the resource patch, failed to apply so the patch may need to be updated.

       last 10 log lines:
       > -- Performing Test CXX_SUPPORTS_VLA_EXTENSION
       > -- Performing Test CXX_SUPPORTS_VLA_EXTENSION - Failed
       > -- LLDB version: 17.0.1
       > CMake Error at cmake/modules/LLDBConfig.cmake:306 (message):
       >   Expected directory for clang-resource headers not found:
       > Call Stack (most recent call first):
       >   CMakeLists.txt:37 (include)
       >
       >
       > -- Configuring incomplete, errors occurred!
       For full logs, run 'nix log /nix/store/q3rssicvjgr4ky84d42afmjbyxl18l05-lldb-17.0.1.drv'.

@alyssais
Copy link
Member

alyssais commented Oct 2, 2023

Just a note to say, please make sure llvmPackages_git is kept in sync with the new LLVM 17 package. Last time this wasn't done and it was a huge headache afterwards.

@RossComputerGuy
Copy link
Member Author

Alright, I'll update it once llvm 17 fully builds. @RaitoBezarius brought that up as well.

@DavidTruby
Copy link
Contributor

DavidTruby commented Oct 2, 2023

Anything I can do to help you with this? I work on LLVM (flang mostly) and use NixOS so would love to have this working!
I can at the very least test on AArch64 Linux but can also take a concrete look at one of the issues you aren't working on?
Edit: I notice you don't have MLIR or Flang in your works/broken list; I think it's worth adding these now that LLVM 17 is out as they're a lot more stable than they were in 16 and before.

@RossComputerGuy
Copy link
Member Author

I've been testing this out and getting it working on my Apple M1 Pro NixOS Asahi machine actually. Right now, I might need help finding or making certain patches that can get things working. I can make a separate commit after the init commit is done to add new things, already got word from someone else who's interested in having MLIR.

@DavidTruby
Copy link
Contributor

Yeah I wasn't suggesting we do everything at once just that MLIR and flang would be nice to have :)

@DavidTruby
Copy link
Contributor

FWIW I had a quick look and the libunwind issue isn't actually to do with libunwind; I can't use the clang built from this to build a C file that includes <stddef.h> either. I guess the wrapper isn't setting some of the include locations correctly?

@RossComputerGuy
Copy link
Member Author

Could be related to the gun install dirs patches? I removed them since patch didn't like them. Would have to update the patches.

@DavidTruby
Copy link
Contributor

That does sound like something that would affect it.

This wouldn't help LLVM17 as that's already released but as a more general point, I wonder if we could get some of the patches that are carried by NixOS into LLVM itself so we don't face the issue of having to update them all the time? I'm not really familiar with what the patches that NixOS carries do but LLVM is generally happy to have some linux distribution specific stuff if it's necessary (there's some Gentoo specific stuff for example)

@RossComputerGuy
Copy link
Member Author

RossComputerGuy commented Oct 3, 2023

Yeah, I agree. I think we should push for upstream patches that makes it easier for us to package new LLVM versions. It would be a much better long term solution for packaging and would make it easier.

@RossComputerGuy
Copy link
Member Author

I've fixed a few things, got clang working and libunwind builds now! Clang is working since I rebased the gnu install dirs patch. I'll look into rebasing the other gnu install dirs patches which should make everything working. But now I should be able to try this out with Zig compiler development now since the essentials build.

Builds & Works

  • llvm
  • clang
  • compiler-rt
  • openmp
  • lld
  • libunwind
  • lldb (fails to launch due to a linked lib error at rt, needs gnu install dirs patch?)

Broken

  • libcxxabi (bad patch)
  • libcxx (bad patch)

@alyssais
Copy link
Member

alyssais commented Oct 3, 2023

This wouldn't help LLVM17 as that's already released but as a more general point, I wonder if we could get some of the patches that are carried by NixOS into LLVM itself so we don't face the issue of having to update them all the time? I'm not really familiar with what the patches that NixOS carries do but LLVM is generally happy to have some linux distribution specific stuff if it's necessary (there's some Gentoo specific stuff for example)

AIUI @Ericson2314 has been in the process of upstreaming our LLVM patches for a long time, but progress has been slow.

@DavidTruby
Copy link
Contributor

I’m happy to help review on the LLVM side if there are patches going up :)

@Ericson2314
Copy link
Member

I have commit bit actually (unless they removed it for me being inactive) I have just been slow / doing other things. Feel free to stalk me on phabricator and see what things are still open.

@Ericson2314
Copy link
Member

The CMAKE_INSTALL_LIBDIR has been a PITA to upstream because lib64 with fedora and friends. IIRC the last things I was trying are quite diverged from the version of the patch that is in Nixpkgs.

@RossComputerGuy RossComputerGuy marked this pull request as ready for review December 7, 2023 01:28
@RossComputerGuy
Copy link
Member Author

@wegank I can boot into macOS on my MBP instead of NixOS and test it on aarch64-darwin. I don't have x86_64-darwin though.

@wegank
Copy link
Member

wegank commented Dec 7, 2023

Result of nixpkgs-review pr 258614 run on aarch64-darwin 1

2 packages failed to build:
  • llvmPackages_17.openmp
  • llvmPackages_17.openmp.dev
35 packages built:
  • clang-tools_17
  • clang_17 (llvmPackages_17.clang ,llvmPackages_17.libcxxClang)
  • lld_17 (llvmPackages_17.lld)
  • lld_17.dev (llvmPackages_17.lld.dev)
  • lld_17.lib (llvmPackages_17.lld.lib)
  • lldb_17 (llvmPackages_17.lldb)
  • lldb_17.dev (llvmPackages_17.lldb.dev)
  • lldb_17.lib (llvmPackages_17.lldb.lib)
  • llvmPackages_17.clang-manpages
  • llvmPackages_17.libclang (llvmPackages_17.clang-unwrapped)
  • llvmPackages_17.libclang.dev (llvmPackages_17.clang-unwrapped.dev)
  • llvmPackages_17.libclang.lib (llvmPackages_17.clang-unwrapped.lib)
  • llvmPackages_17.libclang.python (llvmPackages_17.clang-unwrapped.python)
  • llvmPackages_17.clangNoCompilerRt
  • llvmPackages_17.clangNoCompilerRtWithLibc
  • llvmPackages_17.clangNoLibc
  • llvmPackages_17.clangNoLibcxx
  • llvmPackages_17.clangUseLLVM
  • llvmPackages_17.compiler-rt (llvmPackages_17.compiler-rt-libc ,llvmPackages_17.compiler-rt-no-libc)
  • llvmPackages_17.compiler-rt.dev (llvmPackages_17.compiler-rt-libc.dev)
  • llvmPackages_17.libcxx
  • llvmPackages_17.libcxx.dev
  • llvmPackages_17.stdenv (llvmPackages_17.libcxxStdenv)
  • llvmPackages_17.libcxxabi
  • llvmPackages_17.libcxxabi.dev
  • llvm_17 (llvmPackages_17.llvm)
  • llvm_17.dev (llvmPackages_17.llvm.dev)
  • llvm_17.lib (llvmPackages_17.llvm.lib)
  • llvm_17.python (llvmPackages_17.llvm.python)
  • llvmPackages_17.libstdcxxClang
  • llvmPackages_17.libunwind
  • llvmPackages_17.libunwind.dev
  • llvmPackages_17.lldb-manpages
  • llvmPackages_17.llvm-manpages
  • tests.cc-wrapper.llvmTests.llvmPackages_17.clang (tests.cc-wrapper.llvmTests.llvmPackages_17.libcxx)

@wegank
Copy link
Member

wegank commented Dec 7, 2023

Result of nixpkgs-review pr 258614 run on x86_64-darwin 1

18 packages failed to build:
  • clang-tools_17
  • clang_17 (llvmPackages_17.clang ,llvmPackages_17.libcxxClang)
  • llvmPackages_17.clangNoLibc
  • llvmPackages_17.clangNoLibcxx
  • llvmPackages_17.clangUseLLVM
  • llvmPackages_17.compiler-rt (llvmPackages_17.compiler-rt-libc ,llvmPackages_17.compiler-rt-no-libc)
  • llvmPackages_17.compiler-rt.dev (llvmPackages_17.compiler-rt-libc.dev)
  • llvmPackages_17.libcxx
  • llvmPackages_17.libcxx.dev
  • llvmPackages_17.stdenv (llvmPackages_17.libcxxStdenv)
  • llvmPackages_17.libcxxabi
  • llvmPackages_17.libcxxabi.dev
  • llvmPackages_17.libstdcxxClang
  • llvmPackages_17.libunwind
  • llvmPackages_17.libunwind.dev
  • llvmPackages_17.openmp
  • llvmPackages_17.openmp.dev
  • tests.cc-wrapper.llvmTests.llvmPackages_17.clang (tests.cc-wrapper.llvmTests.llvmPackages_17.libcxx)
19 packages built:
  • lld_17 (llvmPackages_17.lld)
  • lld_17.dev (llvmPackages_17.lld.dev)
  • lld_17.lib (llvmPackages_17.lld.lib)
  • lldb_17 (llvmPackages_17.lldb)
  • lldb_17.dev (llvmPackages_17.lldb.dev)
  • lldb_17.lib (llvmPackages_17.lldb.lib)
  • llvmPackages_17.clang-manpages
  • llvmPackages_17.libclang (llvmPackages_17.clang-unwrapped)
  • llvmPackages_17.libclang.dev (llvmPackages_17.clang-unwrapped.dev)
  • llvmPackages_17.libclang.lib (llvmPackages_17.clang-unwrapped.lib)
  • llvmPackages_17.libclang.python (llvmPackages_17.clang-unwrapped.python)
  • llvmPackages_17.clangNoCompilerRt
  • llvmPackages_17.clangNoCompilerRtWithLibc
  • llvm_17 (llvmPackages_17.llvm)
  • llvm_17.dev (llvmPackages_17.llvm.dev)
  • llvm_17.lib (llvmPackages_17.llvm.lib)
  • llvm_17.python (llvmPackages_17.llvm.python)
  • llvmPackages_17.lldb-manpages
  • llvmPackages_17.llvm-manpages

@wegank
Copy link
Member

wegank commented Dec 7, 2023

x86_64-darwin builds hit llvm/llvm-project#64226, which isn't surprising since llvmPackages_17.libcxxabi needs macOS 10.13+. I propose marking llvmPackages_17.libcxxabi as broken for now and let @NixOS/darwin-maintainers look for fixes after this PR is merged.

@reckenrode
Copy link
Contributor

If we absolutely have to do it, we can revert the change to require 10.13 and hope nothing breaks. Alternatively, once a major package needs LLVM 17 (like Rust or Chromium will eventually), build libcxxabi and compiler-rt with the 11.0 SDK until @toonn’s SDK update is ready.

@RossComputerGuy
Copy link
Member Author

I propose marking llvmPackages_17.libcxxabi as broken for now

What's the best way to perform the check?

@wegank
Copy link
Member

wegank commented Dec 8, 2023

Result of nixpkgs-review pr 258614 run on aarch64-darwin 1

38 packages built:
  • clang-tools_17
  • clang_17
  • lld_17
  • lld_17.dev
  • lld_17.lib
  • lldb_17
  • lldb_17.dev
  • lldb_17.lib
  • llvmPackages_17.bintools (llvmPackages_17.bintoolsNoLibc)
  • llvmPackages_17.bintools-unwrapped
  • llvmPackages_17.clang-manpages
  • llvmPackages_17.clang-unwrapped
  • llvmPackages_17.clang-unwrapped.dev
  • llvmPackages_17.clang-unwrapped.lib
  • llvmPackages_17.clang-unwrapped.python
  • llvmPackages_17.clangNoCompilerRt
  • llvmPackages_17.clangNoCompilerRtWithLibc
  • llvmPackages_17.clangNoLibc
  • llvmPackages_17.clangNoLibcxx
  • llvmPackages_17.clangUseLLVM
  • llvmPackages_17.compiler-rt (llvmPackages_17.compiler-rt-no-libc)
  • llvmPackages_17.compiler-rt.dev (llvmPackages_17.compiler-rt-no-libc.dev)
  • llvmPackages_17.libcxx
  • llvmPackages_17.libcxx.dev
  • llvmPackages_17.stdenv (llvmPackages_17.libcxxStdenv)
  • llvmPackages_17.libcxxabi
  • llvmPackages_17.libcxxabi.dev
  • llvmPackages_17.libllvm
  • llvmPackages_17.libllvm.dev
  • llvmPackages_17.libllvm.lib
  • llvmPackages_17.libllvm.python
  • llvmPackages_17.libstdcxxClang
  • llvmPackages_17.libunwind
  • llvmPackages_17.libunwind.dev
  • llvmPackages_17.lldb-manpages
  • llvmPackages_17.llvm-manpages
  • llvmPackages_17.openmp
  • llvmPackages_17.openmp.dev

@rrbutani rrbutani requested review from rrbutani and removed request for rrbutani December 8, 2023 03:48
@wegank
Copy link
Member

wegank commented Dec 8, 2023

Result of nixpkgs-review pr 258614 run on x86_64-linux 1

43 packages built:
  • clang-tools_17
  • clang_17 (llvmPackages_17.clang ,llvmPackages_17.libstdcxxClang)
  • lld_17 (llvmPackages_17.lld)
  • lld_17.dev (llvmPackages_17.lld.dev)
  • lld_17.lib (llvmPackages_17.lld.lib)
  • lldb_17 (llvmPackages_17.lldb)
  • lldb_17.dev (llvmPackages_17.lldb.dev)
  • lldb_17.lib (llvmPackages_17.lldb.lib)
  • llvmPackages_17.bintools
  • llvmPackages_17.bintools-unwrapped
  • llvmPackages_17.bintoolsNoLibc
  • llvmPackages_17.clang-manpages
  • llvmPackages_17.libclang (llvmPackages_17.clang-unwrapped)
  • llvmPackages_17.libclang.dev (llvmPackages_17.clang-unwrapped.dev)
  • llvmPackages_17.libclang.lib (llvmPackages_17.clang-unwrapped.lib)
  • llvmPackages_17.libclang.python (llvmPackages_17.clang-unwrapped.python)
  • llvmPackages_17.clangNoCompilerRt
  • llvmPackages_17.clangNoCompilerRtWithLibc
  • llvmPackages_17.clangNoLibc
  • llvmPackages_17.clangNoLibcxx
  • llvmPackages_17.clangUseLLVM
  • llvmPackages_17.compiler-rt (llvmPackages_17.compiler-rt-libc ,llvmPackages_17.compiler-rt-no-libc)
  • llvmPackages_17.compiler-rt.dev (llvmPackages_17.compiler-rt-libc.dev)
  • llvmPackages_17.libcxx
  • llvmPackages_17.libcxx.dev
  • llvmPackages_17.libcxxClang
  • llvmPackages_17.libcxxStdenv
  • llvmPackages_17.libcxxabi
  • llvmPackages_17.libcxxabi.dev
  • llvm_17 (llvmPackages_17.llvm)
  • llvm_17.dev (llvmPackages_17.llvm.dev)
  • llvm_17.lib (llvmPackages_17.llvm.lib)
  • llvm_17.python (llvmPackages_17.llvm.python)
  • llvmPackages_17.libunwind
  • llvmPackages_17.libunwind.dev
  • llvmPackages_17.lldb-manpages
  • llvmPackages_17.llvm-manpages
  • llvmPackages_17.openmp
  • llvmPackages_17.openmp.dev
  • llvmPackages_17.stdenv
  • tests.cc-wrapper.llvmTests.llvmPackages_17.clang
  • tests.cc-wrapper.llvmTests.llvmPackages_17.libcxx
  • tests.cc-wrapper.supported

@wegank
Copy link
Member

wegank commented Dec 8, 2023

Result of nixpkgs-review pr 258614 run on x86_64-darwin 1

24 packages marked as broken and skipped:
  • clang-tools_17
  • clang_17
  • llvmPackages_17.clang
  • llvmPackages_17.clangNoLibc
  • llvmPackages_17.clangNoLibcxx
  • llvmPackages_17.clangUseLLVM
  • llvmPackages_17.compiler-rt
  • llvmPackages_17.compiler-rt-libc
  • llvmPackages_17.compiler-rt-libc.dev
  • llvmPackages_17.compiler-rt-no-libc
  • llvmPackages_17.compiler-rt-no-libc.dev
  • llvmPackages_17.compiler-rt.dev
  • llvmPackages_17.libcxx
  • llvmPackages_17.libcxx.dev
  • llvmPackages_17.libcxxClang
  • llvmPackages_17.libcxxStdenv
  • llvmPackages_17.libcxxabi
  • llvmPackages_17.libcxxabi.dev
  • llvmPackages_17.libstdcxxClang
  • llvmPackages_17.libunwind
  • llvmPackages_17.libunwind.dev
  • llvmPackages_17.stdenv
  • tests.cc-wrapper.llvmTests.llvmPackages_17.clang
  • tests.cc-wrapper.llvmTests.llvmPackages_17.libcxx
21 packages built:
  • lld_17 (llvmPackages_17.lld)
  • lld_17.dev (llvmPackages_17.lld.dev)
  • lld_17.lib (llvmPackages_17.lld.lib)
  • lldb_17 (llvmPackages_17.lldb)
  • lldb_17.dev (llvmPackages_17.lldb.dev)
  • lldb_17.lib (llvmPackages_17.lldb.lib)
  • llvmPackages_17.clang-manpages
  • llvmPackages_17.libclang (llvmPackages_17.clang-unwrapped)
  • llvmPackages_17.libclang.dev (llvmPackages_17.clang-unwrapped.dev)
  • llvmPackages_17.libclang.lib (llvmPackages_17.clang-unwrapped.lib)
  • llvmPackages_17.libclang.python (llvmPackages_17.clang-unwrapped.python)
  • llvmPackages_17.clangNoCompilerRt
  • llvmPackages_17.clangNoCompilerRtWithLibc
  • llvm_17 (llvmPackages_17.llvm)
  • llvm_17.dev (llvmPackages_17.llvm.dev)
  • llvm_17.lib (llvmPackages_17.llvm.lib)
  • llvm_17.python (llvmPackages_17.llvm.python)
  • llvmPackages_17.lldb-manpages
  • llvmPackages_17.llvm-manpages
  • llvmPackages_17.openmp
  • llvmPackages_17.openmp.dev

@wegank
Copy link
Member

wegank commented Dec 8, 2023

@ofborg build llvmPackages_17.clang llvmPackages_17.clang.passthru.tests

@wegank wegank merged commit 1be4693 into NixOS:master Dec 8, 2023
21 of 23 checks passed
Copy link
Contributor

github-actions bot commented Dec 8, 2023

Successfully created backport PR for release-23.11:

Copy link
Contributor

Backport failed for release-23.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-23.11
git worktree add -d .worktree/backport-258614-to-release-23.11 origin/release-23.11
cd .worktree/backport-258614-to-release-23.11
git switch --create backport-258614-to-release-23.11
git cherry-pick -x d6d5c8a14e326588aca3b86dafc9aa7c6c26c09a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Package request: llvmPackages_17