Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Falco probe loader build in Linuxkit (docker4mac, docker4win) #657

Closed
fntlnz opened this issue Jun 10, 2019 · 13 comments
Closed

Falco probe loader build in Linuxkit (docker4mac, docker4win) #657

fntlnz opened this issue Jun 10, 2019 · 13 comments

Comments

@fntlnz
Copy link
Member

@fntlnz fntlnz commented Jun 10, 2019

@janbeerden in #656 reported that he is not able to run Falco on a Kind (Kubernetes in Docker) cluster.

Then, Jan reported his execution log, and it was immediately clear that what he was seeing was because there's no prebuilt module for Linuxkit kernels (usually used in Docker for Mac and Windows installations)

Execution log
* Setting up /usr/src links from host
* Unloading falco-probe, if present
* Running dkms install for falco
Error! echo
Your kernel headers for kernel 4.9.125-linuxkit cannot be found at
/lib/modules/4.9.125-linuxkit/build or /lib/modules/4.9.125-linuxkit/source.
* Running dkms build failed, couldn't find /var/lib/dkms/falco/0.15.1/build/make.log
* Trying to load a system falco-probe, if present
* Trying to find precompiled falco-probe for 4.9.125-linuxkit
Found kernel config at /proc/config.gz
* Trying to download precompiled module from https://s3.amazonaws.com/download.draios.com/stable/sysdig-probe-binaries/falco-probe-0.15.1-x86_64-4.9.125-linuxkit-598c2d809165b47f38383b83b6ff4798.ko
curl: (22) The requested URL returned error: 404 Not Found
Download failed, consider compiling your own falco-probe and loading it or getting in touch with the sysdig community
Mon Jun 10 20:30:26 2019: Falco initialized with configuration file /etc/falco/falco.yaml
Mon Jun 10 20:30:26 2019: Loading rules from file /etc/falco/falco_rules.yaml:
Mon Jun 10 20:30:27 2019: Loading rules from file /etc/falco/falco_rules.local.yaml:
Mon Jun 10 20:30:27 2019: Loading rules from file /etc/falco/k8s_audit_rules.yaml:
Mon Jun 10 20:30:28 2019: Unable to load the driver. Exiting.
Mon Jun 10 20:30:28 2019: Runtime error: error opening device /host/dev/falco0. Make sure you have root credentials and that the falco-probe module is loaded.. Exiting.

Those Linuxkit distros also lack a package manager or a way to install or even obtain kernel headers so the only viable option seems to be to figure it out on CI and prebuild the kernel module for those.

@fntlnz
Copy link
Member Author

@fntlnz fntlnz commented Jun 11, 2019

/kind feature

Loading

@fntlnz fntlnz changed the title Falco probe loader prebuilt module for Linuxkit Falco probe loader build in Linuxkit Jun 27, 2019
@fntlnz
Copy link
Member Author

@fntlnz fntlnz commented Jun 27, 2019

@mfdii pointed out that on linuxkit (docker4win and docker4mac) we can build headers using a docker container build using a Dockerfile like this

FROM linuxkit/kernel:4.9.125 AS ksrc
FROM alpine:3.4
ARG FALCOVER=0.13.1
ARG SYSDIGVER=0.24.2

COPY --from=ksrc /kernel-dev.tar /

RUN apk add --no-cache --update wget ca-certificates \
    build-base gcc abuild binutils \
    bc \
    cmake \
    git \
    autoconf && \
  export KERNELVER=`uname -r  | cut -d '-' -f 1`  && \
  export KERNELDIR=/usr/src/linux-headers-4.9.125-linuxkit/ && \
  tar xf /kernel-dev.tar && \
  cd $KERNELDIR && \
  zcat /proc/1/root/proc/config.gz > .config && \
  make olddefconfig && \
  mkdir -p /falco/build && \
  mkdir /src && \
  cd /src && \
  wget https://github.com/falcosecurity/falco/archive/$FALCOVER.tar.gz && \
  tar zxf $FALCOVER.tar.gz && \
  wget https://github.com/draios/sysdig/archive/$SYSDIGVER.tar.gz && \
  tar zxf $SYSDIGVER.tar.gz && \
  mv sysdig-$SYSDIGVER sysdig && \ 
  cd /falco/build && \
  cmake /src/falco-$FALCOVER && \
  make driver && \
  rm -rf /src && \
  apk del wget ca-certificates \
    build-base gcc abuild binutils \
    bc \
    cmake \
    git \
    autoconf

CMD ["insmod","/falco/build/driver/falco-probe.ko"] 
docker build -t falco-docker4win:latest .

Here's how to start it:

$ docker run -it --rm --privileged falco-docker4win:latest
$ docker run -e "SYSDIG_SKIP_LOAD=1" -i -t --name falco --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro falcosecurity/falco

I think that this one should be built-in the probe loader entrypoint when detected that we are on Linuxkit or something like that? Wdyt?

An alternative would just be to include this in the install documentation for such environments so that users are aware.

Loading

@fntlnz fntlnz changed the title Falco probe loader build in Linuxkit Falco probe loader build in Linuxkit (docker4mac, docker4win) Jun 27, 2019
@leodido
Copy link
Member

@leodido leodido commented Jun 27, 2019

Let's start with including it into the installation docs.

Loading

@fntlnz
Copy link
Member Author

@fntlnz fntlnz commented Jun 27, 2019

/kind documentation

Loading

@dghoshal-lbl
Copy link

@dghoshal-lbl dghoshal-lbl commented Jul 10, 2019

I tried a similar Dockerfile to build sysdig on my docker4mac (linuxkit) and it failed because there is nothing on /lib/modules for the specific linux headers. Can someone please let me know how to get the modules and build sysdig correctly on linuxkit?

Loading

@fntlnz
Copy link
Member Author

@fntlnz fntlnz commented Jul 11, 2019

Did you try exactly the one I posted here @dghoshal-lbl ? #657 (comment)

Loading

@dghoshal-lbl
Copy link

@dghoshal-lbl dghoshal-lbl commented Jul 11, 2019

Not for falco. I am trying the same for sysdig on linuxkit (docker4mac). The build fails because there's no /lib/modules.

Loading

@dghoshal-lbl
Copy link

@dghoshal-lbl dghoshal-lbl commented Jul 12, 2019

@fntlnz Any idea how to build the required kernel modules for sysdig on linuxkit (docker4mac)?

Loading

@mfdii
Copy link
Member

@mfdii mfdii commented Aug 16, 2019

FROM alpine:3.4
ARG SYSDIGVER=0.24.2

COPY --from=ksrc /kernel-dev.tar /

RUN apk add --no-cache --update wget ca-certificates \
    build-base gcc abuild binutils \
    bc \
    cmake \
    git \
    autoconf && \
  export KERNELVER=`uname -r  | cut -d '-' -f 1`  && \
  export KERNELDIR=/usr/src/linux-headers-4.9.125-linuxkit/ && \
  tar xf /kernel-dev.tar && \
  cd $KERNELDIR && \
  zcat /proc/1/root/proc/config.gz > .config && \
  make olddefconfig && \
  mkdir -p /sysdig/build && \
  mkdir /src && \
  cd /src && \
  wget https://github.com/draios/sysdig/archive/$SYSDIGVER.tar.gz && \
  tar zxf $SYSDIGVER.tar.gz && \
  mv sysdig-$SYSDIGVER sysdig && \ 
  cd /sysdig/build && \
  cmake /src/sysdig && \
  make driver && \
  rm -rf /src && \
  apk del wget ca-certificates \
    build-base gcc abuild binutils \
    bc \
    cmake \
    git \
    autoconf

CMD ["insmod","/sysdig/build/driver/sysdig-probe.ko"] ```

The above should work but I haven't tested. 

Closing this as it's for sysdig not Falco. 

Loading

@mfdii mfdii closed this Aug 16, 2019
@Issif
Copy link
Member

@Issif Issif commented Aug 30, 2019

@mfdii helped me a lot of months ago with same issue, here my results if can help : https://github.com/Issif/falco-docker4windows/tree/4.9.184

Loading

@saiharshitachava
Copy link

@saiharshitachava saiharshitachava commented Mar 4, 2020

Im trying to use the same thing..But it fails with permission denied
Sending build context to Docker daemon 3.072kB
Step 1/7 : FROM linuxkit/kernel:4.9.125 AS ksrc
---> f99021f74d5e
Step 2/7 : FROM alpine:3.4
---> b7c5ffe56db7
Step 3/7 : ARG FALCOVER=0.13.1
---> Using cache
---> ac0b7eefa5db
Step 4/7 : ARG SYSDIGVER=0.24.2
---> Using cache
---> 7a26d1a0db0c
Step 5/7 : COPY --from=ksrc /kernel-dev.tar /
---> Using cache
---> f178c2f684a4
Step 6/7 : RUN apk add --no-cache --update wget ca-certificates build-base gcc abuild binutils bc cmake git autoconf && export KERNELVER=uname -r | cut -d '-' -f 1 && export KERNELDIR=/usr/src/linux-headers-4.9.125-linuxkit/ && tar xf /kernel-dev.tar && cd $KERNELDIR && zcat /proc/1/root/proc/config.gz > .config && make olddefconfig && mkdir -p /falco/build && mkdir /src && cd /src && wget https://github.com/falcosecurity/falco/archive/$FALCOVER.tar.gz && tar zxf $FALCOVER.tar.gz && wget https://github.com/draios/sysdig/archive/$SYSDIGVER.tar.gz && tar zxf $SYSDIGVER.tar.gz && mv sysdig-$SYSDIGVER sysdig && cd /falco/build && cmake /src/falco-$FALCOVER && make driver && rm -rf /src && apk del wget ca-certificates build-base gcc abuild binutils bc cmake git autoconf
---> Running in 5a2c3e9b8778
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.4/main: Permission denied
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz: Permission denied
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.4/community: Permission denied
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz: Permission denied
ERROR: unsatisfiable constraints:
abuild (missing):
required by: world[abuild]
autoconf (missing):
required by: world[autoconf]
bc (missing):
required by: world[bc]
binutils (missing):
required by: world[binutils]
build-base (missing):
required by: world[build-base]
ca-certificates (missing):
required by: world[ca-certificates]
cmake (missing):
required by: world[cmake]
gcc (missing):
required by: world[gcc]
git (missing):
required by: world[git]
wget (missing):
required by: world[wget]
The command '/bin/sh -c apk add --no-cache --update wget ca-certificates build-base gcc abuild binutils bc cmake git autoconf && export KERNELVER=uname -r | cut -d '-' -f 1 && export KERNELDIR=/usr/src/linux-headers-4.9.125-linuxkit/ && tar xf /kernel-dev.tar && cd $KERNELDIR && zcat /proc/1/root/proc/config.gz > .config && make olddefconfig && mkdir -p /falco/build && mkdir /src && cd /src && wget https://github.com/falcosecurity/falco/archive/$FALCOVER.tar.gz && tar zxf $FALCOVER.tar.gz && wget https://github.com/draios/sysdig/archive/$SYSDIGVER.tar.gz && tar zxf $SYSDIGVER.tar.gz && mv sysdig-$SYSDIGVER sysdig && cd /falco/build && cmake /src/falco-$FALCOVER && make driver && rm -rf /src && apk del wget ca-certificates build-base gcc abuild binutils bc cmake git autoconf' returned a non-zero code: 10

Loading

@saiharshitachava
Copy link

@saiharshitachava saiharshitachava commented Mar 4, 2020

Never mind it was a proxy issue and the image is now built and Im trying to use it as initcontainer and somehow this error pops up..

insmod: can't insert '/falco/build/driver/falco-probe.ko': invalid module format

and I cant run falco either

Loading

@hazcod
Copy link

@hazcod hazcod commented Jul 20, 2021

How does this translate for Docker for Mac Kubernetes via the Helm installation?

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants