Skip to content

Fix building with clang and lld - #13046

Closed
Peter-Levine wants to merge 2 commits into
openzfs:masterfrom
Peter-Levine:fix-clang
Closed

Fix building with clang and lld#13046
Peter-Levine wants to merge 2 commits into
openzfs:masterfrom
Peter-Levine:fix-clang

Conversation

@Peter-Levine

@Peter-Levine Peter-Levine commented Jan 30, 2022

Copy link
Copy Markdown
Contributor

Explicitly pass CC and LD when building modules to prevent kbuild from defaulting to gcc and ld.bfd.

Motivation and Context

Fixes 10107

Description

When building sys-fs/zfs-kmod-9999 on Gentoo Linux, configure fails with:

configure: error: 
        *** Unable to build an empty module.

config.log shows:

configure:16388: clang -c -march=native -O2 -pipe -Werror -Wbool-compare  conftest.c >&5
error: unknown warning option '-Wbool-compare' [-Werror,-Wunknown-warning-option]

build/conftest/build.log shows:

make: Entering directory '/usr/src/linux-5.16.3-gentoo'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: clang version 13.0.0
  You are using:           x86_64-pc-linux-gnu-gcc (Gentoo Hardened 11.2.1_p20220115 p4) 11.2.1 20220115
  CC [M]  /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/build/conftest/conftest.o
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-Qunused-arguments’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mretpoline-external-thunk’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mno-global-merge’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-ftrivial-auto-var-init=zero’
make[1]: *** [scripts/Makefile.build:287: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/build/conftest/conftest.o] Error 1
make[1]: Target '__build' not remade because of errors.
make: *** [Makefile:1846: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/build/conftest] Error 2
make: Target 'modules' not remade because of errors.
make: Leaving directory '/usr/src/linux-5.16.3-gentoo'

The offending code is from config/kernel.m4:

dnl #
dnl # ZFS_LINUX_COMPILE
dnl #
dnl # $1 - build dir
dnl # $2 - test command
dnl # $3 - pass command
dnl # $4 - fail command
dnl # $5 - set KBUILD_MODPOST_NOFINAL='yes'
dnl # $6 - set KBUILD_MODPOST_WARN='yes'
dnl #
dnl # Used internally by ZFS_LINUX_TEST_{COMPILE,MODPOST}
dnl #
AC_DEFUN([ZFS_LINUX_COMPILE], [
	AC_TRY_COMMAND([
	    KBUILD_MODPOST_NOFINAL="$5" KBUILD_MODPOST_WARN="$6"
	    make modules -k -j$TEST_JOBS -C $LINUX_OBJ $ARCH_UM
	    M=$PWD/$1 >$1/build.log 2>&1])
	AS_IF([AC_TRY_COMMAND([$2])], [$3], [$4])
])

kbuild doesn't parse CC, LD, et al., from the environment but expects the to be explicitly passed to make.

Similarly, the build fails to compile, with

x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-Qunused-arguments’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-Qunused-arguments’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-Qunused-arguments’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mretpoline-external-thunk’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mretpoline-external-thunk’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mretpoline-external-thunk’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mno-global-merge’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mno-global-merge’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mno-global-merge’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-ftrivial-auto-var-init=zero’
make[5]: *** [scripts/Makefile.build:287: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/module/avl/avl.o] Error 1
make[4]: *** [scripts/Makefile.build:549: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/module/avl] Error 2
make[4]: *** Waiting for unfinished jobs....
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-ftrivial-auto-var-init=zero’
make[5]: *** [scripts/Makefile.build:287: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/module/spl/../os/linux/spl/spl-atomic.o] Error 1
make[4]: *** [scripts/Makefile.build:549: /var/tmp/portage/sys-fs/zfs-kmod-9999/work/zfs-kmod-9999/module/spl] Error 2

The offending code is from module/Makefile.in:

modules-Linux:
	list='$(SUBDIR_TARGETS)'; for td in $$list; do $(MAKE) -C $$td; done
	$(MAKE) -C @LINUX_OBJ@ M="$$PWD" @KERNEL_MAKE@ CONFIG_ZFS=m modules

Again, this is resolved by explicitly passing CC and LD.

How Has This Been Tested?

Tested using /etc/portage/patches/sys-fs/zfs-kmod-9999/zfs-kmod-9999-clang.patch:

--- a/config/kernel.m4
+++ b/config/kernel.m4
@@ -604,7 +604,7 @@
 	AC_TRY_COMMAND([
 	    KBUILD_MODPOST_NOFINAL="$5" KBUILD_MODPOST_WARN="$6"
 	    make modules -k -j$TEST_JOBS -C $LINUX_OBJ $ARCH_UM
-	    M=$PWD/$1 >$1/build.log 2>&1])
+	    CC="$CC" LD="$LD" M=$PWD/$1 >$1/build.log 2>&1])
 	AS_IF([AC_TRY_COMMAND([$2])], [$3], [$4])
 ])

and /etc/portage/patches/sys-fs/zfs-kmod-9999/zfs-kmod-9999-clang2.patch:

--- zfs-kmod-9999/module/Makefile.in.old	2022-01-30 01:51:53.977190277 -0500
+++ zfs-kmod-9999/module/Makefile.in	2022-01-30 02:19:35.827239919 -0500
@@ -52,7 +52,7 @@
 
 modules-Linux:
 	list='$(SUBDIR_TARGETS)'; for td in $$list; do $(MAKE) -C $$td; done
-	$(MAKE) -C @LINUX_OBJ@ M="$$PWD" @KERNEL_MAKE@ CONFIG_ZFS=m modules
+	$(MAKE) -C @LINUX_OBJ@ CC="$(CC)" LD="$(LD)" M="$$PWD" @KERNEL_MAKE@ CONFIG_ZFS=m modules
 
 modules-FreeBSD:
 	+$(FMAKE)

then ebuild /var/db/repos/local/sys-fs/zfs-kmod/zfs-kmod-9999.ebuild merge

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

szubersk
szubersk suggested changes Jan 31, 2022
View reviewed changes
Comment thread module/Makefile.in
@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Feb 2, 2022
szubersk
szubersk suggested changes Feb 3, 2022
View reviewed changes
Comment thread config/kernel.m4
Comment thread module/Makefile.in Outdated
@szubersk

szubersk commented Feb 3, 2022

Copy link
Copy Markdown
Contributor

Good change overall! This patch will help experimenting with clang builds on Linux.

Peter-Levine
Peter-Levine commented Feb 4, 2022
View reviewed changes

@Peter-Levine Peter-Levine left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Works on my end with both gcc/ld.bfd and clang/ld.lld when building against a kernel built with the same toolchain.

Comment thread module/Makefile.in
szubersk
szubersk reviewed Feb 4, 2022
View reviewed changes
Comment thread config/kernel.m4 Outdated
@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Feb 4, 2022
@szubersk

szubersk commented Feb 7, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

@Peter-Levine I just tried out this patch with 5.15.21 LLVM=1 built kernel. What bothers me is that while make works fine, make pkg-kmod doesn't. KERNEL_* variables are not injected into rpmbuild and GCC is still enforced there: the build fails.
It's still worth it to include this PR in master branch as it unblocks people down the stream. Clang-compiled modules cannot be used for ZTS yet, though.

@behlendorf

behlendorf commented Feb 7, 2022

Copy link
Copy Markdown
Contributor

@Peter-Levine would you also mind rebasing this PR on the latest version of the master branch, then force updating the PR. This should get us a cleaner CI run to make sure all is well before merging it.

@szubersk

szubersk commented Feb 8, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

@Peter-Levine Please consider incorporating szubersk@d21dba8 into this PR. I just tested that commit and compilation passes for clang-built 5.15.21 kernel.

@Peter-Levine
Add support for $KERNEL_{CC,LD,LLVM} variables
842c932 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Signed-off-by: Peter Levine <plevine457@gmail.com>
@Peter-Levine
propagate KERNEL_* to *.spec
450e1dd 
Authored by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
@Peter-Levine

Peter-Levine commented Feb 8, 2022

Copy link
Copy Markdown
Contributor Author

@szubersk PR is incorporated.
@behlendorf Rebased and pushed.

@behlendorf behlendorf closed this in b66140c Feb 9, 2022
behlendorf pushed a commit that referenced this pull request Feb 9, 2022
@szubersk @behlendorf
Propagate KERNEL_* to *.spec
4eea717 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes #13046
@szubersk

szubersk commented Feb 13, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fix #10107
Fix #10267

tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 15, 2022
@Peter-Levine @tonyhutter
Add support for $KERNEL_{CC,LD,LLVM} variables
7e2877d 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 15, 2022
@szubersk @tonyhutter
Propagate KERNEL_* to *.spec
10ec129 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 16, 2022
@Peter-Levine @tonyhutter
Add support for $KERNEL_{CC,LD,LLVM} variables
f62ce3e 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 16, 2022
@szubersk @tonyhutter
Propagate KERNEL_* to *.spec
7adb87f 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 17, 2022
@Peter-Levine @tonyhutter
Add support for $KERNEL_{CC,LD,LLVM} variables
c7fcf00 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Feb 17, 2022
@szubersk @tonyhutter
Propagate KERNEL_* to *.spec
2681f8a 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
gentoo-bot pushed a commit to gentoo/gentoo that referenced this pull request Mar 11, 2022
@thesamesam
sys-fs/zfs-kmod: add Clang logic
7b55c82 
The needed patch made its way upstream but we need
a small amount of logic in the ebuild to accommodate
it fully.

Bug: openzfs/zfs#13046
Closes: https://bugs.gentoo.org/814194
Thanks-to: Peter Levine <plevine457@gmail.com>
Signed-off-by: Sam James <sam@gentoo.org>
nicman23 pushed a commit to nicman23/zfs that referenced this pull request Aug 22, 2022
@Peter-Levine @nicman23
Add support for $KERNEL_{CC,LD,LLVM} variables
cdb858e 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
nicman23 pushed a commit to nicman23/zfs that referenced this pull request Aug 22, 2022
@szubersk @nicman23
Propagate KERNEL_* to *.spec
3ef0097 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
nicman23 pushed a commit to nicman23/zfs that referenced this pull request Aug 22, 2022
@Peter-Levine @nicman23
Add support for $KERNEL_{CC,LD,LLVM} variables
9cbc5a8 
Currently, $(CC), $(LD), and $(LLVM) variables aren't passed to kbuild
while building modules.  This causes modules to build with the default
GNU GCC toolchain and prevents experimenting with other toolchains such
as CLANG/LLVM.  It can also lead to build failure if the CFLAGS/LDFLAGS
passed are incompatible with gcc/ld.

Pass $KERNEL_CC, $KERNEL_LD, and $KERNEL_LLVM as $(CC), $(LD), and
$(LLVM), respectively, to kbuild for each that is defined in the
environment.  This should take care of the majority of alternative
toolchain use cases.

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
nicman23 pushed a commit to nicman23/zfs that referenced this pull request Aug 22, 2022
@szubersk @nicman23
Propagate KERNEL_* to *.spec
2d58776 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Peter Levine <plevine457@gmail.com>
Closes openzfs#13046
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@behlendorf behlendorf behlendorf approved these changes
+1 more reviewer
@szubersk szubersk szubersk approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Status: Accepted Ready to integrate (reviewed, tested)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Peter-Levine @szubersk @behlendorf