new(cmd,pkg): added new `local` build processor to build drivers using local kernel sources/gcc/clang #306

FedeDP · 2023-11-21T14:02:48Z

What type of PR is this?

/kind feature

Any specific area of the project related to this PR?

/area cmd
/area pkg

What this PR does / why we need it:

This PR introduces a new local build processor that will just build chosen driver version against local kernel sources using local gcc/clang.
Therefore, to work, it needs kernel-headers installed (eg: /lib/modules/6.6.1-arch1-1/build) and clang/gcc on the machine.
It does not use docker, of course.

Moreover, it supports building kmod through dkms, and supports directly using a provided preconfigured src-dir.
The bigger aim is to completely implement driver building logic that is today part of falco-driver-loader script; this would allow us to properly use driverkit inside falcoctl to implement the drivertype.Build method (eg: https://github.com/falcosecurity/falcoctl/blob/main/pkg/driver/type/kmod.go#L183).
IMHO driverkit should be the only project responsible of building drivers.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

new: added a local build processor to build local kernel using local kernel sources/gcc/clang

FedeDP · 2023-11-21T14:03:49Z

cmd/local.go

+		},
+	}
+	// Add root flags, but not the ones unneeded
+	unusedFlagsSet := map[string]struct{}{


These flags aren't needed for this command, therefore they are removed.
I think we might want to do the same eg: for images cmd.

FedeDP · 2023-11-21T14:04:59Z

cmd/local.go

+}
+
+// Partially overrides rootCmd.persistentPreRunFunc setting some defaults before config init/validation stage.
+func persistentPreRunFunc(rootCommand *RootCmd, rootOpts *RootOptions) func(c *cobra.Command, args []string) error {


This gets called in place of rootCmd.PersistentPreRunE since we shadowed it.
It does just set a bunch of default values (that are not used anyway by the local build processor, but are logged) and calls the rootCmd.PersistentPreRunE (ie: persistentValidateFunc)

FedeDP · 2023-11-21T14:08:09Z

pkg/driverbuilder/builder/builders.go

+	// nor registered in byTarget, just use "arch".
+	// NOTE: it won't be used anyway, as driverbuilder/local.go
+	// manually creates a "local" builder.
+	if target.String() == "local" {


This is a workaround to let local cmd set local target and passing the root options validation step.

Note: local builder is not mapped inside the byTarget map because i did not want it to be visible by the user when running eg: docker builder processor, since it makes no sense.
So, the new local builder is hidden and manually instantiated by local builder processor.

FedeDP · 2023-11-21T14:08:46Z

pkg/driverbuilder/builder/local.go

+var localTemplate string
+
+type LocalBuilder struct {
+	GccPath string


GccPath is the full path to a discovered Gcc, and will be set by local build processor.

Since there might be multiple gcc present, we tried all of them until we can build the module.

FedeDP · 2023-11-21T14:09:24Z

pkg/driverbuilder/builder/templates/local.sh

There is no kernel headers download step in the local script.

FedeDP · 2023-11-21T14:09:43Z

pkg/driverbuilder/local.go

+	kr := b.KernelReleaseFromBuildConfig()
+
+	// create a builder based on the choosen build type
+	v := &builder.LocalBuilder{}


Manually instantiated; see https://github.com/falcosecurity/driverkit/pull/306/files#r1400657216

FedeDP · 2023-11-21T14:10:10Z

pkg/driverbuilder/local.go

+		gccs = []string{"gcc"}
+	}
+
+	for _, gcc := range gccs {


For each discovered gcc, try to build module and probe.

FedeDP · 2023-11-21T14:11:07Z

pkg/driverbuilder/local.go

+		}
+		// If we received an error, perhaps we must just rebuilt the kmod.
+		// Check if we were able to build anything.
+		if _, err = os.Stat(builder.ModuleFullPath); !os.IsNotExist(err) {


If any of module or probe could be built, disable its build for the subsequent step.
Ie: if we have multiple GCCs available, and the first one is not able to build the kmod, but the system clang was able to build the probe, disable its build.

FedeDP · 2023-11-21T14:12:31Z

Example output:

./_output/bin/driverkit local --output-probe /tmp/probe.o --output-module /tmp/mod.ko -l debug
level=DEBUG msg="running without a configuration file"
level=DEBUG msg="running with options" output-module=/tmp/mod.ko output-probe=/tmp/probe.o driverversion=master kernelrelease=6.6.1-arch1-1 kernelversion=1 target=local arch=amd64 kernelurls=[] repo-org=falcosecurity repo-name=libs
level=INFO msg="driver building, it will take a few seconds" processor=local
level=DEBUG msg="doing a new local build"
make -C /lib/modules/6.6.1-arch1-1/build M=/tmp/driver modules
make[1]: ingresso nella directory «/usr/lib/modules/6.6.1-arch1-1/build»
  CC [M]  /tmp/driver/main.o
  CC [M]  /tmp/driver/dynamic_params_table.o
  CC [M]  /tmp/driver/fillers_table.o
  CC [M]  /tmp/driver/flags_table.o
  CC [M]  /tmp/driver/ppm_events.o
  CC [M]  /tmp/driver/ppm_fillers.o
  CC [M]  /tmp/driver/event_table.o
  CC [M]  /tmp/driver/syscall_table64.o
  CC [M]  /tmp/driver/ppm_cputime.o
  CC [M]  /tmp/driver/ppm_tp.o
  CC [M]  /tmp/driver/syscall_ia32_64_map.o
  LD [M]  /tmp/driver/falco.o
  MODPOST /tmp/driver/Module.symvers
  CC [M]  /tmp/driver/falco.mod.o
  LD [M]  /tmp/driver/falco.ko
  BTF [M] /tmp/driver/falco.ko
make[1]: uscita dalla directory «/usr/lib/modules/6.6.1-arch1-1/build»
filename:       /tmp/driver/module.ko
schema_version: 2.12.4
api_version:    6.0.0
build_commit:   master
version:        master
author:         the Falco authors
license:        Dual MIT/GPL
srcversion:     1748D869C7785E1E4F96844
depends:
retpoline:      Y
name:           falco
vermagic:       6.6.1-arch1-1 SMP preempt mod_unload
parm:           g_buffer_bytes_dim:This is the dimension of a single per-CPU buffer in bytes. Please note: this buffer will be mapped twice in the process virtual memory, so pay attention to its size.
parm:           max_consumers:Maximum number of consumers that can simultaneously open the devices (uint)
parm:           verbose:Enable verbose logging (bool)
make -C /lib/modules/6.6.1-arch1-1/build M=$PWD
make[1]: ingresso nella directory «/usr/lib/modules/6.6.1-arch1-1/build»
clang -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/compiler-version.h -include ./include/linux/kconfig.h \
        -D__KERNEL__ -fmacro-prefix-map=./= \
         \
         \
        -D__KERNEL__ \
        -D__BPF_TRACING__ \
        -Wno-gnu-variable-sized-type-not-at-end \
        -Wno-address-of-packed-member \
        -fno-jump-tables \
        -fno-stack-protector \
        -Wno-tautological-compare \
        -Wno-unknown-attributes \
        -O2 -g -emit-llvm -c /tmp/driver/bpf/probe.c -o /tmp/driver/bpf/probe.ll
llc -march=bpf -filetype=obj -o /tmp/driver/bpf/probe.o /tmp/driver/bpf/probe.ll
  MODPOST /tmp/driver/bpf/Module.symvers
make[1]: uscita dalla directory «/usr/lib/modules/6.6.1-arch1-1/build»
-rw-r--r-- 1 federico federico 6399480 21 nov 15.12 probe.o
level=INFO msg="kernel module available" path=/tmp/mod.ko
level=INFO msg="eBPF probe available" path=/tmp/probe.o

ls -la /tmp/mod.ko
-rw-r--r-- 1 federico federico 1169360 21 nov 15.11 /tmp/mod.ko

ls -la /tmp/probe.o
-rw-r--r-- 1 federico federico 6399480 21 nov 15.12 /tmp/probe.o

FedeDP · 2023-11-21T15:16:04Z

/hold for discussion, eventually :)

…ocal kernel sources / gccs / clang. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

…options in local builder. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

dwindsor · 2023-11-23T17:53:09Z

LGTM! Incorporating driverkit into falcoctl is a great idea.

Is falcoctl the only intended user of the local build processor? IOW, will we ever expect an end user to actually compile w/ local headers?

dwindsor · 2023-11-23T17:49:37Z

cmd/local.go

+		PersistentPreRunE: persistentPreRunFunc(rootCommand, rootOpts),
+		Run: func(c *cobra.Command, args []string) {
+			slog.With("processor", c.Name()).Info("driver building, it will take a few seconds")
+			if !configOptions.DryRun {


Just a stylistic nit: you could remove a level of indentation in this closure with

if configOptions.DryRun { return }

poiana · 2023-11-23T17:53:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dwindsor, FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [FedeDP,dwindsor]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

FedeDP · 2023-11-24T08:00:20Z

Is falcoctl the only intended user of the local build processor? IOW, will we ever expect an end user to actually compile w/ local headers?

Well, since eg: Arch builder is hit by #303, i was able to build the drivers on my machine using the new local build processor :D
But yep, i think the user will be mostly falcoctl!

FedeDP · 2023-11-24T08:00:51Z

/unhold

poiana added kind/feature New feature or request dco-signoff: yes area/cmd area/pkg labels Nov 21, 2023

poiana requested review from dwindsor and Lowaiz November 21, 2023 14:02

poiana added the approved label Nov 21, 2023

FedeDP changed the title ~~new(cmd,pkg): added new local build processor to build local kernel using local kernel sources/gcc/clang~~ new(cmd,pkg): added new local build processor to build drivers using local kernel sources/gcc/clang Nov 21, 2023

poiana added the size/XL label Nov 21, 2023

FedeDP commented Nov 21, 2023

View reviewed changes

pkg/driverbuilder/builder/templates/local.sh Outdated

Copy link

Contributor Author

FedeDP Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no kernel headers download step in the local script.

FedeDP commented Nov 21, 2023

View reviewed changes

FedeDP force-pushed the new/local branch from 2c2be6d to 1e5b173 Compare November 21, 2023 14:11

FedeDP force-pushed the new/local branch from 1e5b173 to c88725f Compare November 21, 2023 14:14

poiana added the do-not-merge/hold label Nov 21, 2023

new(cmd,pkg): added new local command to build local kernel using l…

879e1b2

…ocal kernel sources / gccs / clang. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP force-pushed the new/local branch from c88725f to 879e1b2 Compare November 22, 2023 08:17

new(cmd,pkg/driverbuilder): support env,src-dir and dkms build …

ae9069b

…options in local builder. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

dwindsor approved these changes Nov 23, 2023

View reviewed changes

poiana assigned dwindsor Nov 23, 2023

poiana added the lgtm label Nov 23, 2023

poiana removed the do-not-merge/hold label Nov 24, 2023

poiana merged commit 2c7d6f0 into master Nov 24, 2023
4 checks passed

poiana deleted the new/local branch November 24, 2023 08:01

FedeDP mentioned this pull request Nov 24, 2023

new(pkg): use driverkit local build processor instead of implementing drivers build falcosecurity/falcoctl#356

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new(cmd,pkg): added new `local` build processor to build drivers using local kernel sources/gcc/clang #306

new(cmd,pkg): added new `local` build processor to build drivers using local kernel sources/gcc/clang #306

FedeDP commented Nov 21, 2023 •

edited

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP Nov 21, 2023

FedeDP commented Nov 21, 2023

FedeDP commented Nov 21, 2023

dwindsor commented Nov 23, 2023

dwindsor Nov 23, 2023

poiana commented Nov 23, 2023

FedeDP commented Nov 24, 2023

FedeDP commented Nov 24, 2023

new(cmd,pkg): added new local build processor to build drivers using local kernel sources/gcc/clang #306

new(cmd,pkg): added new local build processor to build drivers using local kernel sources/gcc/clang #306

Conversation

FedeDP commented Nov 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FedeDP commented Nov 21, 2023

FedeDP commented Nov 21, 2023

dwindsor commented Nov 23, 2023

Choose a reason for hiding this comment

poiana commented Nov 23, 2023

FedeDP commented Nov 24, 2023

FedeDP commented Nov 24, 2023

new(cmd,pkg): added new `local` build processor to build drivers using local kernel sources/gcc/clang #306

new(cmd,pkg): added new `local` build processor to build drivers using local kernel sources/gcc/clang #306

FedeDP commented Nov 21, 2023 •

edited