Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uninstall/update should rmmod (old) kernel module #418

Closed
majorsimon opened this issue Sep 12, 2018 · 5 comments · Fixed by #459
Closed

Uninstall/update should rmmod (old) kernel module #418

majorsimon opened this issue Sep 12, 2018 · 5 comments · Fixed by #459

Comments

@majorsimon
Copy link

During a recent update from v0.11.1 to v.0.12.1, we were hit with segmentation fault issues. After investigation, it appears that the old kernel module had not been unloaded. An execution of rmmod falco_probe && modprobe falco_probe && systemctl restart falco resolved the issue.
@majorsimon
Copy link
Author

from a sudo yum update falco

Resolving Dependencies
--> Running transaction check
---> Package falco.x86_64 0:0.11.1-1 will be updated
---> Package falco.x86_64 0:0.12.1-1 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================================================================
 Package                   Arch                       Version                         Repository                  Size
=======================================================================================================================
Updating:
 falco                     x86_64                     0.12.1-1                        draios                     2.4 M
Transaction Summary
=======================================================================================================================
Upgrade  1 Package
Total download size: 2.4 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
falco-0.12.1-x86_64.rpm                                                                         | 2.4 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : falco-0.12.1-1.x86_64                                                                               1/2 
Creating symlink /var/lib/dkms/falco/0.12.1/source ->
                 /usr/src/falco-0.12.1
DKMS: add completed.
Kernel preparation unnecessary for this kernel.  Skipping...
Building module:
cleaning build area...
make -j2 KERNELRELEASE=3.10.0-862.11.6.el7.x86_64 -C /lib/modules/3.10.0-862.11.6.el7.x86_64/build M=/var/lib/dkms/falco/0.12.1/build..........
cleaning build area...
DKMS: build completed.
falco-probe.ko.xz:
Running module version sanity check.
 - Original module
   - This kernel never originally had a module by this name
 - Installation
   - Installing to /lib/modules/3.10.0-862.11.6.el7.x86_64/extra/
Adding any weak-modules
depmod......
DKMS: install completed.
-------- Uninstall Beginning --------
Module:  falco
Version: 0.11.1
Kernel:  3.10.0-862.11.6.el7.x86_64 (x86_64)
-------------------------------------
Status: This module version was INACTIVE for this kernel.
depmod......
DKMS: uninstall completed.
------------------------------
Deleting module version: 0.11.1
completely from the DKMS tree.
------------------------------
Done.
  Cleanup    : falco-0.11.1-1.x86_64                                                                               2/2 
  Verifying  : falco-0.12.1-1.x86_64                                                                               1/2 
  Verifying  : falco-0.11.1-1.x86_64                                                                               2/2 
Updated:
  falco.x86_64 0:0.12.1-1                                                                                              
Complete!

@majorsimon
Copy link
Author

majorsimon commented Sep 12, 2018 •
edited
Loading 

$ sudo falco 
Wed Sep 12 13:56:21 2018: Falco initialized with configuration file /etc/falco/falco.yaml
Wed Sep 12 13:56:21 2018: Loading rules from file /etc/falco/falco_rules.yaml:
Segmentation fault
Segmentation fault

/var/log/messages

Sep 12 13:56:22 staging-01 kernel: falco_probe: adding new consumer ffff93b6d76d2f70
Sep 12 13:56:22 staging-01 kernel: falco_probe: initializing ring buffer for CPU 0
Sep 12 13:56:22 staging-01 kernel: falco_probe: CPU buffer initialized, size=8388608
Sep 12 13:56:22 staging-01 kernel: falco_probe: initializing ring buffer for CPU 1
Sep 12 13:56:22 staging-01 kernel: falco_probe: CPU buffer initialized, size=8388608
Sep 12 13:56:22 staging-01 kernel: falco_probe: starting capture
Sep 12 13:56:23 staging-01 kernel: falco[5900]: segfault at 7f29c05500d2 ip 000000000058ad88 sp 00007fff242ebe20 error 4 in falco[400000+695000]
Sep 12 13:56:23 staging-01 kernel: falco_probe: deallocating consumer ffff93b6d76d2f70
Sep 12 13:56:23 staging-01 kernel: falco_probe: no more consumers, stopping capture

@majorsimon
Copy link
Author

The same has occurred on another similar system:

[root@simon-01 ~]# tree /var/lib/dkms/falco/
/var/lib/dkms/falco/
├── 0.11.1
│   ├── 3.10.0-862.11.6.el7.x86_64
│   │   └── x86_64
│   │       ├── log
│   │       │   └── make.log
│   │       └── module
│   │           └── falco-probe.ko.xz
│   └── source -> /usr/src/falco-0.11.1
└── kernel-3.10.0-862.11.6.el7.x86_64-x86_64 -> 0.11.1/3.10.0-862.11.6.el7.x86_64/x86_64

7 directories, 2 files
[root@simon-01 ~]# lsmod | grep falco 
falco_probe           601973  2 
[root@simon-01 ~]# dmesg | grep falco
[ 1215.053579] falco_probe: loading out-of-tree module taints kernel.
[ 1215.060394] falco_probe: module verification failed: signature and/or required key missing - tainting kernel
[ 1215.072475] falco_probe: driver loading, falco-probe 0.11.1
[ 1215.083156] falco_probe: adding new consumer ffff994e62495ee0
[ 1215.089941] falco_probe: initializing ring buffer for CPU 0
[ 1215.105219] falco_probe: CPU buffer initialized, size=8388608
[ 1215.111222] falco_probe: initializing ring buffer for CPU 1
[ 1215.129832] falco_probe: CPU buffer initialized, size=8388608
[ 1215.138757] falco_probe: starting capture
[root@simon-01 ~]# yum update falco 
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.umd.edu
 * epel: mirror.us.leaseweb.net
 * extras: mirror.mojohost.com
 * updates: mirror.vtti.vt.edu
Resolving Dependencies
--> Running transaction check
---> Package falco.x86_64 0:0.11.1-1 will be updated
---> Package falco.x86_64 0:0.12.1-1 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================
 Package                   Arch                       Version                         Repository                  Size
=======================================================================================================================
Updating:
 falco                     x86_64                     0.12.1-1                        draios                     2.4 M

Transaction Summary
=======================================================================================================================
Upgrade  1 Package

Total download size: 2.4 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
falco-0.12.1-x86_64.rpm                                                                         | 2.4 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : falco-0.12.1-1.x86_64                                                                               1/2 

Creating symlink /var/lib/dkms/falco/0.12.1/source ->
                 /usr/src/falco-0.12.1

DKMS: add completed.

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
make -j2 KERNELRELEASE=3.10.0-862.11.6.el7.x86_64 -C /lib/modules/3.10.0-862.11.6.el7.x86_64/build M=/var/lib/dkms/falco/0.12.1/build....
cleaning build area...

DKMS: build completed.

falco-probe.ko.xz:
Running module version sanity check.
 - Original module
   - This kernel never originally had a module by this name
 - Installation
   - Installing to /lib/modules/3.10.0-862.11.6.el7.x86_64/extra/
Adding any weak-modules

depmod....

DKMS: install completed.

-------- Uninstall Beginning --------
Module:  falco
Version: 0.11.1
Kernel:  3.10.0-862.11.6.el7.x86_64 (x86_64)
-------------------------------------

Status: This module version was INACTIVE for this kernel.
depmod....

DKMS: uninstall completed.

------------------------------
Deleting module version: 0.11.1
completely from the DKMS tree.
------------------------------
Done.
  Cleanup    : falco-0.11.1-1.x86_64                                                                               2/2 
  Verifying  : falco-0.12.1-1.x86_64                                                                               1/2 
  Verifying  : falco-0.11.1-1.x86_64                                                                               2/2 

Updated:
  falco.x86_64 0:0.12.1-1                                                                                              

Complete!
[root@simon-01 ~]# dmesg | grep falco 
[ 1215.053579] falco_probe: loading out-of-tree module taints kernel.
[ 1215.060394] falco_probe: module verification failed: signature and/or required key missing - tainting kernel
[ 1215.072475] falco_probe: driver loading, falco-probe 0.11.1
[ 1215.083156] falco_probe: adding new consumer ffff994e62495ee0
[ 1215.089941] falco_probe: initializing ring buffer for CPU 0
[ 1215.105219] falco_probe: CPU buffer initialized, size=8388608
[ 1215.111222] falco_probe: initializing ring buffer for CPU 1
[ 1215.129832] falco_probe: CPU buffer initialized, size=8388608
[ 1215.138757] falco_probe: starting capture
[232783.262219] falco_probe: deallocating consumer ffff994e62495ee0
[232783.275174] falco_probe: no more consumers, stopping capture
[232784.115890] falco_probe: adding new consumer ffff994e36e60fd0
[232784.123117] falco_probe: initializing ring buffer for CPU 0
[232784.137070] falco_probe: CPU buffer initialized, size=8388608
[232784.143774] falco_probe: initializing ring buffer for CPU 1
[232784.157843] falco_probe: CPU buffer initialized, size=8388608
[232784.164238] falco_probe: starting capture
[232784.692123] falco[2830]: segfault at 7f79e8c450d2 ip 000000000058ad88 sp 00007fffe51c0630 error 4 in falco[400000+695000]
[232784.708514] falco_probe: deallocating consumer ffff994e36e60fd0
[232784.720017] falco_probe: no more consumers, stopping capture

@majorsimon
Copy link
Author

System info:

[root@simon-01 ~]# cat /etc/*elease
CentOS Linux release 7.5.1804 (Core) 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.5.1804 (Core) 
CentOS Linux release 7.5.1804 (Core) 
[root@simon-01 ~]# uname -r 
3.10.0-862.11.6.el7.x86_64

@majorsimon
Copy link
Author

majorsimon commented Sep 13, 2018 •
edited
Loading

Issue has occurred on a debian machine also, so doesn't appear to be OS-dependent:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  falco
1 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
Need to get 0 B/2,920 kB of archives.
After this operation, 129 kB of additional disk space will be used.
(Reading database ... 69333 files and directories currently installed.)
Preparing to unpack .../falco_0.12.1_amd64.deb ...

-------- Uninstall Beginning --------
Module:  falco
Version: 0.11.1
Kernel:  4.9.0-6-amd64 (x86_64)
-------------------------------------

Status: Before uninstall, this module version was ACTIVE on this kernel.

falco-probe.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.9.0-6-amd64/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.

depmod......

DKMS: uninstall completed.

------------------------------
Deleting module version: 0.11.1
completely from the DKMS tree.
------------------------------
Done.
Unpacking falco (0.12.1) over (0.11.1) ...
Setting up falco (0.12.1) ...
Loading new falco-0.12.1 DKMS files...
Building for 4.9.0-6-amd64
Building initial module for 4.9.0-6-amd64
Done.

falco-probe:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.9.0-6-amd64/updates/dkms/

depmod...

DKMS: install completed.
Processing triggers for systemd (232-25+deb9u1) ...
simon@staging-client-debian:~$ sudo dmesg | grep falco 
[2057270.424184] falco_probe: deallocating consumer ffff9e48b6ccf040
[2057270.430867] falco_probe: no more consumers, stopping capture
[2057270.475827] audit: type=1130 audit(1536828834.327:291938): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=falco comm="systemd" exe="/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[2057270.501791] audit: type=1131 audit(1536828834.351:291939): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=falco comm="systemd" exe="/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

simon@staging-client-debian:~$ sudo falco 
Thu Sep 13 08:54:39 2018: Falco initialized with configuration file /etc/falco/falco.yaml
Thu Sep 13 08:54:39 2018: Loading rules from file /etc/falco/falco_rules.yaml:
Segmentation fault
Segmentation fault
simon@staging-client-debian:~$ sudo rmmod falco-probe && sudo modprobe falco-probe && sudo falco 
Thu Sep 13 08:55:03 2018: Falco initialized with configuration file /etc/falco/falco.yaml
Thu Sep 13 08:55:03 2018: Loading rules from file /etc/falco/falco_rules.yaml:
^CEvents detected: 0
Rule counts by severity:
Triggered rules by rule name:
simon@staging-client-debian:~$ cat /etc/*elease 
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

@majorsimon majorsimon changed the title Uninstall should rmmod kernel module Uninstall/update should rmmod (old) kernel module Sep 17, 2018
mstemm added a commit that referenced this issue Nov 6, 2018
@mstemm
Load/unload kernel module on start/stop
41c588f 
When falco is started, load the kernel module. (The falco binary also
will do a modprobe if it can't open the inspector, as a backup).

When falco is stopped, unload the kernel module.

This fixes #418.
This was referenced Nov 6, 2018
Merged
Closed
mstemm added a commit that referenced this issue Nov 6, 2018
@mstemm
Load/unload kernel module on start/stop (#459)
32f8e30 
* Load/unload kernel module on start/stop

When falco is started, load the kernel module. (The falco binary also
will do a modprobe if it can't open the inspector, as a backup).

When falco is stopped, unload the kernel module.

This fixes #418.

* Put script execute line in right place.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Load/unload kernel module on start/stop falcosecurity/falco
1 participant
@majorsimon