Skip to content
This repository has been archived by the owner. It is now read-only.

Building custom kernel modules fails in 1897.0.0 #2505

Closed
eranreshef opened this issue Sep 20, 2018 · 10 comments
Closed

Building custom kernel modules fails in 1897.0.0 #2505

eranreshef opened this issue Sep 20, 2018 · 10 comments

Comments

@eranreshef
Copy link

@eranreshef eranreshef commented Sep 20, 2018

Issue Report

Bug

Container Linux Version

1897.0.0

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1897.0.0
VERSION_ID=1897.0.0
BUILD_ID=2018-09-10-2308
PRETTY_NAME="Container Linux by CoreOS 1897.0.0 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"

Environment

aws

Expected Behavior

I have an automated process for building a custom kernel module (wireguard) for coreos.
The process ran smoothly on previous versions (alpha, beta and stable).

Actual Behavior

When I executed the module's make command in 1897.0.0 I got:

ERROR: Kernel configuration is invalid.
include/generated/autoconf.h or include/config/auto.conf are missing.
Run 'make oldconfig && make prepare' on kernel src to fix it.

Reproduction Steps

  1. Follow the custom kernel module installation process - https://coreos.com/os/docs/latest/kernel-modules.html
  2. Follow wireguard's compilation process -
    https://www.wireguard.com/install/#compiling-from-source
@dm0-
Copy link

@dm0- dm0- commented Sep 20, 2018

It's working for me. Can you paste your scripts?

I ran this in the developer container, and everything finished successfully:

emerge-gitclone
emerge -gKv coreos-sources
gzip -cd /proc/config.gz > /usr/src/linux/.config
make -C /usr/src/linux modules_prepare
git clone --depth=1 https://git.zx2c4.com/WireGuard
make -C WireGuard/src

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Sep 22, 2018

emerge-gitclone
emerge -gKv coreos-sources
gzip -cd /proc/config.gz > /usr/src/linux/.config
make -C /usr/src/linux modules_prepare

WIREGUARD_VERSION=0.0.20180918
filename=WireGuard-$WIREGUARD_VERSION
wget https://git.zx2c4.com/WireGuard/snapshot/$filename.tar.xz
tar xf $filename.tar.xz
make -C /$filename/src

Thanks for helping out.
I'm going on a short holiday and won't be available in the next couple of days, please don't hurry to close the issue if I don't respond.

@dm0-
Copy link

@dm0- dm0- commented Sep 26, 2018

I pasted all of those commands directly, and everything succeeded. Make sure your developer container version matches the host OS.

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Oct 3, 2018

I download the dev container according to the instructions:

. /usr/share/coreos/release
. /usr/share/coreos/update.conf
url="http://${GROUP:-stable}.release.core-os.net/$COREOS_RELEASE_BOARD/$COREOS_RELEASE_VERSION/coreos_developer_container.bin.bz2"

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Oct 4, 2018

This script is running on the coreOS machine:

#!/bin/sh
set -exuo pipefail

modules=/opt/modules
mkdir -p ${modules} ${modules}.wd
mount -o "lowerdir=/lib/modules,upperdir=${modules},workdir=${modules}.wd" -t overlay overlay /lib/modules
systemctl enable lib-modules.mount

. /usr/share/coreos/release
. /usr/share/coreos/update.conf
url="http://${GROUP:-stable}.release.core-os.net/$COREOS_RELEASE_BOARD/$COREOS_RELEASE_VERSION/coreos_developer_container.bin.bz2"
curl -L "$url" | bzip2 -d > coreos_developer_container.bin

# the coreos developer container is lacking of file system space since its downloading all the coreos sources to /usr/src
# we resolved the lack of space issue by creating an overlay on that dir
src_dir=$(pwd)/coreos_src
mkdir -p ${src_dir}
systemd-nspawn --bind=/lib/modules --bind=/tmp --overlay=${src_dir}::/usr/src --image=coreos_developer_container.bin /tmp/installer.sh

modprobe wireguard

and /tmp/installer.sh is:

#!/bin/bash
set -exuo pipefail
export WIREGUARD_VERSION=0.0.20180918

emerge-gitclone
emerge -gKv coreos-sources
gzip -cd /proc/config.gz > /usr/src/linux/.config
make -C /usr/src/linux modules_prepare

# wireguard specific steps
emerge libmnl
filename=WireGuard-$WIREGUARD_VERSION
wget https://git.zx2c4.com/WireGuard/snapshot/$filename.tar.xz
tar xf $filename.tar.xz
make -C /$filename/src
make install -C /$filename/src

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Oct 17, 2018

ping... anyone there?

@dm0-
Copy link

@dm0- dm0- commented Oct 17, 2018

Try adding KERNELDIR=/usr/src/linux to WireGuard's make commands.

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Oct 17, 2018

Wow! thanks! that did solve the issue!
Can you elaborate a bit more about the issue and the solution?
Is it a bug in my scripts?

@dm0-
Copy link

@dm0- dm0- commented Oct 18, 2018

The reason you saw the problem and I didn't is because your script mounts /lib/modules from the host's production file system, and I didn't add that to the nspawn command when I tried.

Since an upgrade of the package manager a few weeks ago, it seems that they made a change in behavior that contradicts the documentation which resulted in kernel headers starting to be dropped from production images. They were still included in the developer container's file system, so our automated tests on it still passed.

I've added a workaround for that so it should work again in the next release.

@eranreshef
Copy link
Author

@eranreshef eranreshef commented Oct 18, 2018

Thanks for the explanation!

@dm0- dm0- closed this in #3449 Oct 18, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants