Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add driverdog to link and load kernel modules at runtime #1867

Merged
merged 2 commits into from Jan 11, 2022

Conversation

arnaldo2792
Copy link
Contributor

Issue number:
N / A

Description of changes:
Related to #1799 , that PR is quite big. I'm taking some of the commits in that PR as their individual PR.

3b4cd1da sources: add driver dog

driverdog is a tool to link kernel modules at runtime, given a
configuration. The configuration must be provided per driver vendor, and
it is created at runtime by the tmpfilesd daemon, using a file
from the %{_cross_factorydir} directory.
e2b2b235 packages: add binutils

Binutils provides the ld linker, required to link proprietary kernel
modules at runtime.

We can't ship kernel modules linked against GPL code (i. e. proprietary kernel modules), doing so violates the GPL license under which the kernel is released. Instead, Bottlerocket images with proprietary kernel modules will link and load the kernel modules at runtime using driverdog.

binutils is built as part of the release package, but it isn't included in any variant. Instead, packages that depend on it will include it in their spec file, as follows:

Requires: %{_cross_os}binutils

Testing done:
In the new variant created in #1799:

Shipped a drivers configuration file that looks as follows, along with required objects in the OS image:

[nvidia-tesla]
lib-modules-path = "kernel/drivers/extra/video/nvidia/tesla"
objects-source = "__NVIDIA_MODULES__" # this is set to the directory where the objects used to like the kernel modules are

[nvidia-tesla.object-files."nvidia.o"]
link-objects = ["nv-interface.o", "nv-kernel.o"]

[nvidia-tesla.kernel-modules."nvidia.ko"]
link-objects = ["nvidia.o", "nvidia.mod.o"]

[nvidia-tesla.object-files."nvidia-modeset.o"]
link-objects = ["nv-modeset-interface.o", "nv-modeset-kernel.o"]

[nvidia-tesla.kernel-modules."nvidia-modeset.ko"]
link-objects = ["nvidia-modeset.o", "nvidia-modeset.mod.o"]

[nvidia-tesla.kernel-modules."nvidia-uvm.ko"]
link-objects = ["nvidia-uvm.o", "nvidia-uvm.mod.o"]

Validated the kernel modules were linked and loaded:

bash-5.0# systemctl status link-kernel-modules
● link-kernel-modules.service - Link additional kernel modules
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/link-kernel-modules.service; enabled; vendor preset: enabled)
     Active: active (exited) since Mon 2021-12-13 20:10:53 UTC; 3min 27s ago
   Main PID: 2554 (code=exited, status=0/SUCCESS)

Dec 13 20:10:53 localhost systemd[1]: Starting Link additional kernel modules...
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Linked object 'nvidia.o'
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Stripped object 'nvidia.o'
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Linked object 'nvidia-modeset.o'
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Stripped object 'nvidia-modeset.o'
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Linked nvidia-modeset.ko
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Linked nvidia.ko
Dec 13 20:10:53 localhost driverdog[2554]: 20:10:53 [INFO] Linked nvidia-uvm.ko
Dec 13 20:10:53 localhost systemd[1]: Finished Link additional kernel modules.


bash-5.0# systemctl status load-kernel-modules
● load-kernel-modules.service - Load additional kernel modules
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/load-kernel-modules.service; enabled; vendor preset: enabled)
     Active: active (exited) since Mon 2021-12-13 20:10:55 UTC; 3min 31s ago
   Main PID: 2610 (code=exited, status=0/SUCCESS)

Dec 13 20:10:53 localhost systemd[1]: Starting Load additional kernel modules...
Dec 13 20:10:55 localhost driverdog[2610]: 20:10:55 [INFO] Updated modules dependencies
Dec 13 20:10:55 localhost driverdog[2610]: 20:10:55 [INFO] Loaded kernel modules
Dec 13 20:10:55 localhost systemd[1]: Finished Load additional kernel modules.


bash-5.0# lsmod | grep nvidia
nvidia_uvm           1138688  0
nvidia_modeset       1155072  0
nvidia              34754560  18 nvidia_uvm,nvidia_modeset
drm                   606208  1 nvidia
i2c_core               98304  2 nvidia,drm

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

ca-certificates = { path = "../ca-certificates" }
binutils = { path = "../binutils" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: should keep this sorted.

Suggested change
ca-certificates = { path = "../ca-certificates" }
binutils = { path = "../binutils" }
binutils = { path = "../binutils" }
ca-certificates = { path = "../ca-certificates" }

RefuseManualStart=true
RefuseManualStop=true
After=link-kernel-modules.service
Wants=link-kernel-modules.service
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could strengthen this dependency -

Suggested change
Wants=link-kernel-modules.service
Requires=link-kernel-modules.service

Comment on lines 109 to 120
// If the target module set was given, link the kernel modules in it
if let Some(target) = target {
let driver_config = modules_sets
.get(&target)
.context(error::MissingModuleSet { target })?;
link_modules(&driver_config, &kernel_version)?
}

// Link all the modules sets if no target module was given
for driver_config in modules_sets.values() {
link_modules(&driver_config, &kernel_version)?;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we always link all module sets, even if we pass the option for a specific set. The load modules function has the same issue.

Copy link
Member

@webern webern Dec 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, missing else block? (Edit: Resolved)

packages/os/link-kernel-modules.service Show resolved Hide resolved
Comment on lines 109 to 120
// If the target module set was given, link the kernel modules in it
if let Some(target) = target {
let driver_config = modules_sets
.get(&target)
.context(error::MissingModuleSet { target })?;
link_modules(&driver_config, &kernel_version)?
}

// Link all the modules sets if no target module was given
for driver_config in modules_sets.values() {
link_modules(&driver_config, &kernel_version)?;
}
Copy link
Member

@webern webern Dec 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, missing else block? (Edit: Resolved)

sources/driverdog/src/main.rs Outdated Show resolved Hide resolved
Binutils provides the ld linker, required to link proprietary kernel
modules at runtime.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
@arnaldo2792
Copy link
Contributor Author

Forced push includes rebase to resolve conflicts

driverdog is a tool to link kernel modules at runtime, given a
configuration. The configuration must be provided per driver vendor, and
it is created at runtime by the tmpfilesd daemon, using a file
from the %{_cross_factorydir} directory.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
@arnaldo2792
Copy link
Contributor Author

Forced push includes:

  • Add missing else statement
  • Add error message when the kernel module isn't UTF-8
  • Add some comments explaining why link-kernel-modules and load-kernel-modules shouldn't be manually restarted

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌺

@arnaldo2792 arnaldo2792 merged commit d07091e into bottlerocket-os:develop Jan 11, 2022
@arnaldo2792 arnaldo2792 deleted the driverdog branch January 14, 2022 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants