In-tree kernel support #851
Labels
Type: Building
Indicates an issue related to building binaries
Type: Feature
Feature request or new feature
Milestone
Comments
|
The thread for this feature on zfs-discuss: |
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
In zfs, each module Makefile contains a MODULE variable which contains the name of the module, and the following declarations reference this variable. In spl, there is a MODULES variable which is never used. Rename it to MODULE and use it like in zfs. This improves consistency between the two build systems. Signed-off-by: Prakash Surya <surya1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
Currently, when configure --with-config is used, selective compilation is only effective for the simple "make" case. Package builders (e.g. make rpm) still build everything (utils and modules). This patch fixes that. This patch also drops the duplicate rpm-modules build target. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Prakash Surya <surya1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
Commit 3160d4f changed the set of conditions under which spl_mutex_spin_max would be implemented as a function by changing an #if in sys/mutex.h. The corresponding implementation file spl-mutex.c, however, has not been updated to reflect the change. This results in undefined reference errors on spl_mutex_spin_max under the following condition: ((!CONFIG_SMP || CONFIG_DEBUG_MUTEXES) && HAVE_MUTEX_OWNER && HAVE_TASK_CURR) This patch fixes the issue by using the same #if in sys/mutex.h and spl-mutex.c. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
This patch adds a new option to configure: --enable-linux-builtin. When this option is used, the following happens: - Compilation of kernel modules is disabled. - A failure to find UTS_RELEASE is followed by a suggestion to run "make prepare" on the kernel source tree. This patch also adds a new test which tries to compile an empty module as a basic toolchain sanity test. If it fails and the option was specified, the error is followed by a suggestion to run "make scripts" on the kernel source tree. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
Currently, when building a test case, we're compiling an entire Linux module from beginning to end. This includes the MODPOST stage, which generates a "conftest.mod.c" file with some boilerplate module declaration code. This poses a problem when configuring for built-in on kernels which have loadable module support disabled. In this case conftest.mod.c is referencing disabled code, resulting in a compilation failure, thus breaking the tests. This patch fixes the issue by faking the modpost stage when the --enable-linux-builtin option is provided. It does so by forcing the modpost command to be /bin/true, and using an empty conftest.mod.c file. The test module still compiles fine, although the result isn't loadable, but we don't really care at this point. Note it is important to preserve the modpost stage when building out of tree. This allows for the posibility of configure checks to leverage this phase to identify GPL-only symbols. Signed-off-by: Prakash Surya <surya1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
This patch adds a new autoconf function: SPL_LINUX_TRY_COMPILE_SYMBOL. This new function does the following: - Call LINUX_TRY_COMPILE with the specified parameters. - If unsuccessful, return false. - If successful and we're configuring with --enable-linux-builtin, return true. - Else, call CHECK_SYMBOL_EXPORT with the specified parameters and return the result. All calls to CHECK_SYMBOL_EXPORT are converted to LINUX_TRY_COMPILE_SYMBOL so that the tests work even when configuring for builtin on a kernel which doesn't have loadable module support, or hasn't been built yet. The only exception are: - AC_GET_VMALLOC_INFO, because we don't even have a public header to include in the test case, but that's okay considering this symbol can be ignored just fine. - SPL_AC_DEVICE_CREATE, which is legacy API for 2.6.18 kernels. Since kernels this old are no longer supported it should arguably just be removed entirely from the build system. Note that we're also checking for the correct prototype with an actual call, which was not the case with CHECK_SYMBOL_EXPORT. However, for "complicated" test cases like with multiple symbol versions (e.g. vfs_fsync), we stick with the original behavior and only check for the function's existence. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
This commit introduces a "copy-builtin" script designed to prepare a kernel source tree for building SPL as a builtin module. The script makes a full copy of all needed files, thus making the kernel source tree fully independent of the spl source package. To achieve that, some compilation flags (-include, -I) have been moved to module/Makefile. This Makefile is only used when compiling external modules; when compiling builtin modules, a Kbuild file generated by the configure-builtin script is used instead. This makes sure Makefiles inside the kernel source tree does not contain references to the spl source package. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
Currently, the SPL tries to determine the hostid at module load. The hostid is usually determined by running the userland program "hostid" during module initialization. Unfortunately, when the module initializes, it may be way too soon to be able to run any userland programs. This is especially true when the module is compiled directly inside the kernel (built-in); in that case, the SPL would try to run hostid when the kernel is still initializing, which of course is doomed to fail. This patch fixes the issue by deferring hostid generation until something actually needs the hostid (that is, when zone_get_hostid() is called), thus switching to a "on-initialization" model to a "on-demand" (lazy loading) model. ZFS only needs the hostid when some pool operations are requested, and this always happens way after the kernel has finished initialization, thus solving the problem. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs/zfs#851
behlendorf
added a commit
to behlendorf/spl
that referenced
this issue
Jul 26, 2012
Support in-tree builtin module building.
These commits add support for compiling the SPL module as a built-in
kernel module by copying the module code into the kernel source tree.
Here's the procedure:
- Create your kernel configuration (`.config` file) as usual. This
has to be done first so that SPL's configure script is able to
detect kernel features correctly.
- Run `make prepare scripts` inside the kernel source tree.
- Run `./configure --enable-linux-builtin --with-linux=/usr/src/linux-...`
inside the SPL directory.
- Run `./copy-builtin /usr/src/linux-...` inside the SPL directory.
- In the kernel source tree, enable the `CONFIG_SPL` option
(e.g. using `make menuconfig`).
- Build the kernel as usual.
SPL module parameters can be set at boot time using the following syntax
on the kernel command line: `spl.parameter_name=parameter_value`.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs/zfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
Currently, when configure --with-config is used, selective compilation is only effective for the simple "make" case. Package builders (e.g. make rpm) still build everything (utils and modules). This patch fixes that. Signed-off-by: Prakash Surya <surya1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
This patch adds a new option to configure: --enable-linux-builtin. When this option is used, the following happens: - Compilation of kernel modules is disabled. - A failure to find UTS_RELEASE is followed by a suggestion to run "make prepare" on the kernel source tree. This patch also adds a new test which tries to compile an empty module as a basic toolchain sanity test. If it fails and the option was specified, the error is followed by a suggestion to run "make scripts" on the kernel source tree. Signed-off-by: Prakash Surya <surya1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
Currently, when building a test case, we're compiling an entire Linux module from beginning to end. This includes the MODPOST stage, which generates a "conftest.mod.c" file with some boilerplate module declaration code. This poses a problem when configuring for built-in on kernels which have loadable module support disabled. In this case conftest.mod.c is referencing disabled code, resulting in a compilation failure, thus breaking the tests. This patch fixes the issue by faking the modpost stage when the --enable-linux-builtin option is provided. It does so by forcing the modpost command to be /bin/true, and using an empty conftest.mod.c file. The test module still compiles fine, although the result isn't loadable, but we don't really care at this point. Note it is important to preserve the modpost stage when building out of tree. The ZFS_AC_KERNEL_BLK_END_REQUEST, ZFS_AC_KERNEL_BLK_QUEUE_FLUSH, and ZFS_AC_KERNEL_BLK_RQ_BYTES configure checks all depend on it to identify GPL-only symbols. Signed-off-by: Prakash Surya <surya1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
This patch adds a new autoconf function: ZFS_LINUX_TRY_COMPILE_SYMBOL. This new function does the following: - Call LINUX_TRY_COMPILE with the specified parameters. - If unsuccessful, return false. - If successful and we're configuring with --enable-linux-builtin, return true. - Else, call CHECK_SYMBOL_EXPORT with the specified parameters and return the result. All calls to CHECK_SYMBOL_EXPORT are converted to LINUX_TRY_COMPILE_SYMBOL so that the tests work even when configuring for builtin on a kernel which doesn't have loadable module support, or hasn't been built yet. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
This commit introduces a "copy-builtin" script designed to prepare a kernel source tree for building ZFS as a builtin module. The script makes a full copy of all needed files, thus making the kernel source tree fully independent of the zfs source package. To achieve that, some compilation flags (-include, -I) have been moved to module/Makefile. This Makefile is only used when compiling external modules; when compiling builtin modules, a Kbuild file generated by the configure-builtin script is used instead. This makes sure Makefiles inside the kernel source tree does not contain references to the zfs source package. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
When libzfs checks if the module is loaded or not, it currently reads /proc/modules and searches for a line matching the module name. Unfortunately, if the module is included in the kernel itself (built-in module), then /proc/modules won't list it, so libzfs will wrongly conclude that the module is not loaded, thus making all ZFS userspace tools unusable. Fortunately, all loaded modules appear as directories in /sys/module, even built-in ones. Thus we can use /sys/module in lieu of /proc/modules to fix the issue. As a bonus, the code for checking becomes much simpler. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue openzfs#851
behlendorf
added a commit
to behlendorf/zfs
that referenced
this issue
Jul 26, 2012
Support in-tree builtin module building.
These commits add support for compiling the ZFS module as a built-in
kernel module by copying the module code into the kernel source tree.
Here's the procedure:
- Create your kernel configuration (`.config` file) as usual. This
has to be done first so that ZFS's configure script is able to
detect kernel features correctly.
- Run `make prepare scripts` inside the kernel source tree.
- Run `./configure --enable-linux-builtin --with-linux=/usr/src/linux-...`
inside the ZFS directory.
- Run `./copy-builtin /usr/src/linux-...` inside the ZFS directory.
- In the kernel source tree, enable the `CONFIG_ZFS` option (e.g. using
`make menuconfig`). Note that this option depends on `CONFIG_SPL`
(see openzfs/spl@7440380).
- Build the kernel as usual.
ZFS module parameters can be set at boot time using the following syntax
on the kernel command line: `zfs.parameter_name=parameter_value`.
Note that you also need to rebuild the userspace tools (see
openzfs/zfs@f09398c).
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#851
|
For those interested in this change, please give it a try. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Extend the build infrastructure to facilitate the process of patching the spl/zfs code directly in the kernel. This will allow for kernels to be build with zfs support and module loading disabled.
The text was updated successfully, but these errors were encountered: