version check fails on RHEL / CentOS 7.x

[sspans]

Installing the openvswitch datapath modules via DKMS fails on redhat/centos 7 because the original modules are tagged with a checksum in the srcversion field:

# modinfo /lib/modules/3.10.0-514.el7.x86_64/kernel/net/openvswitch/openvswitch.ko
filename:       /lib/modules/3.10.0-514.el7.x86_64/kernel/net/openvswitch/openvswitch.ko
license:        GPL
description:    Open vSwitch switching datapath
rhelversion:    7.3
srcversion:     B31AE95554C9D9A0067F935
depends:        nf_conntrack,nf_nat,libcrc32c,nf_nat_ipv6,nf_nat_ipv4,nf_defrag_ipv6
intree:         Y
vermagic:       3.10.0-514.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        D4:88:63:A7:C1:6F:CC:27:41:23:E6:29:8F:74:F0:57:AF:19:FC:54
sig_hashalgo:   sha256

Resulting failure:

#dkms autoinstall openvswitch/2.6.2

openvswitch:
Running module version sanity check.
Error! Module version 2.6.2 for openvswitch.ko
is not newer than what is already found in kernel 3.10.0-514.el7.x86_64 (B31AE95554C9D9A0067F935).
You may override by specifying --force.

Quick hack which resolves the issue for me:

--- /usr/sbin/dkms.orig	2016-12-20 11:31:59.368461247 +0100
+++ /usr/sbin/dkms	2016-12-20 11:32:03.059461265 +0100
@@ -723,7 +723,7 @@
             res[2]=${vals[2]}
             ;;
         srcversion:)
-        res[1]=${vals[1]}
+        #res[1]=${vals[1]}
         ;;
     esac
     done < <(modinfo $1)

[yuzaipiaofei]

`#dkms autoinstall openvswitch/2.6.2

openvswitch:
Running module version sanity check.
Error! Module version 2.6.2 for openvswitch.ko
is not newer than what is already found in kernel 3.10.0-514.el7.x86_64 (B31AE95554C9D9A0067F935).
You may override by specifying --force.`

I want to know which kernel version you are using to build the openvswitch.ko module ?

[sspans]

The default kernel from RHEL 7.3 version: 3.10.0-514.el7.x86_64

However this issue seems to hit every RHEL7 / CentOS 7 kernel, they all have similar modinfo output.

[scaronni]

Same issue as here: #5

[sspans]

It would make sense to make dkms handle 'broken' modules, given that this affects the mainline kernels from RHEL/CentOS. POLA and all.

[yuzaipiaofei]

@sspans @scaronni
Could you help to provide the broken modules source code to test ？

[yuzaipiaofei]

I have the RHEL7.3 build environment to reproduce the installing error，if the issue can be reproduced and make sure it is a dkms bug ,we will find out the workable solution to fix it .

[superm1]

@yuzaipiaofei I believe this should be the source https://github.com/openvswitch/ovs/tree/master/datapath

Here's the DKMS package spec file:
https://github.com/openvswitch/ovs/blob/75e2077e0c43224bcca92746b28b01a4936fc101/rhel/openvswitch-dkms.spec.in

@sspans You might be able to provide something a little more direct about the exact version you are failing with, but it sounds like this should be a generic failure.

[sspans]

The failure is triggered by the upstream openvswitch kernel module - it's not dependend on the dkms source tarball. A sample of affected kernels would be:

kernel-3.10.0-514.2.2.el7.x86_64
kernel-3.10.0-327.el7.x86_64
kernel-3.10.0-327.36.3.el7.x86_64

Modinfo which triggers the issue looks like this:

> modinfo /var/lib/dkms/openvswitch/2.6.2/3.10.0-514.2.2.el7.x86_64/x86_64/module/openvswitch.ko
filename:       /var/lib/dkms/openvswitch/2.6.2/3.10.0-514.2.2.el7.x86_64/x86_64/module/openvswitch.ko
version:        2.6.2
license:        GPL
description:    Open vSwitch switching datapath
rhelversion:    7.3
srcversion:     3A8FCA763C880FAD0F8AD6A
depends:        nf_conntrack,nf_nat,nf_defrag_ipv6,udp_tunnel,libcrc32c,nf_nat_ipv6,nf_nat_ipv4
vermagic:       3.10.0-514.2.2.el7.x86_64 SMP mod_unload modversions

[sspans]

@superm1 please see issue #5 for a longer description. There are upstream modules in distribution kernels which trigger DKMS failures because the modules lack a version field.

Fixing the upstream modules is a noble goal, but would not help for the kernels already out there.
The fix to DKMS mentionedi in issue #5 is trivial and would solve the issue.

[jared-dominguez]

@sspans I already stated why the patch in #5 will not be accepted and gave acceptable alternatives, including other changes to DKMS (even though openvswitch really should fix their module). Also, for what it's worth, distributions update kernels more frequently than DKMS, so I don't follow your argument about kernels already in the field, especially since the bug is in openvswitch anyway.

[sspans]

I'm pretty sure this will not be changed on the rhel/centos side.

What would be an acceptable change to DKMS to deal with this?

[yuzaipiaofei]

Before change i also can get below install error
[root@localhost src]# dkms install openvswitch -v 1.4

openvswitch:
Running module version sanity check.
Error! Module version B31AE95554C9D9A0067F935 for openvswitch.ko
is not newer than what is already found in kernel 3.10.0-514.el7.x86_64 (B31AE95554C9D9A0067F935).
You may override by specifying --force.
Adding any weak-modules

depmod...

Backing up initramfs-3.10.0-514.el7.x86_64.img to /boot/initramfs-3.10.0-514.el7.x86_64.img.old-dkms
Making new initramfs-3.10.0-514.el7.x86_64.img
(If next boot fails, revert to initramfs-3.10.0-514.el7.x86_64.img.old-dkms image)
dracut..........

DKMS: install completed.

[yuzaipiaofei]

apply patch code :

diff --git a/dkms b/dkms
index 65286de..a66af89 100644
--- a/dkms
+++ b/dkms
@@ -803,7 +803,7 @@ check_version_sanity()
return 1
fi

• if [[ $kernels_info && $dkms_info && ! ( $(VER $dkms_info) > $(VER $kernels_info) ) && ! $force ]]; then

• if [[ $kernels_info && $dkms_info && ! ( $(VER $dkms_info) -ge $(VER $kernels_info) ) && ! $force ]]; then
  error $"Module version $dkms_info for ${4}$module_suffix"
  $"is not newer than what is already found in kernel $1 ($kernels_info)."
  $"You may override by specifying --force."
  @@ -3681,7 +3681,7 @@ while (($# > 0)); do
  exec >/dev/null 2>&1
  ;;
  --version|-V)

[yuzaipiaofei]

Then install the module again :
[root@localhost src]# dkms install openvswitch -v 1.4

openvswitch:
Running module version sanity check.

• Original module
  • Found /lib/modules/3.10.0-514.el7.x86_64/updates/openvswitch.ko
  • Storing in /var/lib/dkms/openvswitch/original_module/3.10.0-514.el7.x86_64/x86_64/
  • Archiving for uninstallation purposes
• Installation
  • Installing to /lib/modules/3.10.0-514.el7.x86_64/extra/
    Adding any weak-modules

depmod...

Backing up initramfs-3.10.0-514.el7.x86_64.img to /boot/initramfs-3.10.0-514.el7.x86_64.img.old-dkms
Making new initramfs-3.10.0-514.el7.x86_64.img
(If next boot fails, revert to initramfs-3.10.0-514.el7.x86_64.img.old-dkms image)
dracut..........

DKMS: install completed.

[yuzaipiaofei]

In my opinion，if the kernel module is built by user or developer, maybe the module source codes are same or changed.

1. If the module codes are same as the module codes built and installed in kernel . i think dkms should allow user to install it again .
2. if the codes are changed and built with module version crc ,or somehow the code built without essential kernel module version . Dkms should also allow the user to install their modules without previous error warning .

This is my suggestion to fix the issue.

Mario & Jared . How do you think this solution @superm1 @jared-dominguez
？

[jared-dominguez]

The semantics for "--force" shouldn't be changed since it already is in use. A new parameter is preferable. Perhaps "--force-unversioned"? It should output a warning message too.

The new behavior being described here should be restricted to using this --force-unversioned parameter.

[sspans]

If this also would be available as a configuration option then our issue would be resolved.
Most invocations of dkms are via the automated kernel upgrades, so a way to configure this behaviour for the openvswitch module would be needed.

[jared-dominguez]

@sspans Yes, sorry, that was implied, but again only if a warning message is output. And again, openvswitch really should fix their bug as that's the real issue.

[yuzaipiaofei]

After i review the issue again, i found that this issue root cause is the openvswitch dkms package source code version is same as the RHEL7.3 GA kernel module source codes. DKMS will compare the module info and found the new built openvswitch module info is same as the installed one .

Current DKMS sanity check policy will refuse the duplicated installation .
If you still want to install your module , you can use "--force" to install it.
So i do not think we need add more codes for this issue .
Sorry about that .

[sspans]

Please reopen this issue, you seem to have misunderstood the issue.

There are two versions of the openvswitch kernel module, the in-tree version (as shipped by redhat - with no versioning) and the out of tree version (from https://github.com/openvswitch/ovs). The latter has functionality which isn't included in the in-tree version.

With dkms as-is it isn't possible to install the out of tree - versioned ovs module, because the in-tree module doesn't have a version number.

Please considder merging one of the suggested patches.

[celesteking]

Can we get this fixed finally? Comparing on md5 hash is stupid.

[tonyhutter]

I've posted a fix (including a reproducer script) for this issue in #60. Please take a look at it when you get a chance.

[scaronni]

Sounds good! Many thanks

[GabeSJones]

I don't think the posted fix addresses the original reporter's problem. I am facing a similar predicament. We have made modifications to an in-tree driver in recent 4.1x kernels. We'd like for these changes to be available out-of-tree to users of distributions with much older kernels, so we anticipate distributing them via a DKMS RPM or DEB package.

The code has never used MODULE_VERSION. We want to change that upstream, but none of the existing releases of this driver are versioned, so any current distribution that has this driver installed has an unversioned module. Attempting to install a DKMS package with a versioned module results in the error that the original reporter posted, because check_version_sanity() ends up comparing the version of the new module with the srcversion hash of the old module.

If, e..g, the existing module has no version, and the new module is version 4.20, you end up with something like:

kernels_info = (D461CC2083F4A806A36EBD1)
dkms_info = (4.20 89CAAC69BAB21FECAA04D17)

The following ends up comparing 4.20 with D461CC2083F4A806A36EBD1, which seems invalid:

if [[ $kernels_info && $dkms_info && ! ( $(VER $dkms_info) > $(VER $kernels_info) ) && ! $force ]]; then

Further, in the case where neither module is versioned, the code above compares the two srcversion hashes against each other with something other than equality, which doesn't make any sense. As a result, if I leave the new module unversioned as well, the install may or may not work depending upon whether the srcversion hash on the new module happens to compare favorably to the one on the existing module. The code above only works consistently if both the existing module and the module being installed both have version numbers.