openbsd: add syz-manager with enabled PF configuration

[blackgnezdo]

We don't test enough configurations and certainly miss bugs in PF as our default test machine configuration has it disabled. We should add a distinct config where PF is enabled. This is achieved by replacing /etc/rc.conf.local file. Replicating create-openbsd-vmm-worker.sh sounds suboptimal. A slightly different mechanism is possible though.

We already support different kernel config files that kernel builds consume. They are referenced in the syz-ci configuration file kept outside of repo. I propose we add an option to keep an additional "file system overlay" in the syzkaller source tree. The overlay will simply be a set of files to copy over the files in the default image. We already copy the kernel into the images, it is simple to add a few more files. The only missing part is plumbing the per-config overlay location through syz-bot.

What do you think @mptre & @dvyukov?

[dvyukov]

The userspaceDir can contain whatever you (pkg/build/openbsd.go) want and it's already plumbed through syzbot.
There are also cmdlineFile and sysctlFile which don't use yet. Once of then may contain, say, PF=1.

[mptre]

I like the general idea.

The userspaceDir can contain whatever you (pkg/build/openbsd.go) want and it's already plumbed through syzbot. There are also cmdlineFile and sysctlFile which don't use yet. Once of then may contain, say, `PF=1`.

Are such files visible in crash reports?

[dvyukov]

Only the kernel config is saved with crashes.

[blackgnezdo]

The configuration would be named and versioned with syzkaller. The sha1 will determine the configuration unambiguously. I agree it's not direct though.

[blackgnezdo]

Thanks @dvyukov, the userspaceDir idea looks pretty viable. I'll need some symlink games to point into the source tree but it's all easy enough. If you have any preferences about where I should put these overlays in syzkaller repo - let me know.

[dvyukov]

What exactly will be in that overlay?
If we store whole files, we will need to store lots of irrelevant (default) configs, right?
If we are talking only about /etc/rc.conf.local, then it seems to match precisely the designation of sysctl file. So perhaps we could do the same as linux in its image build:

syzkaller/tools/create-gce-image.sh

Line 131 in 1c0e457

cat $SYZ_SYSCTL_FILE | sudo tee -a disk.mnt/etc/sysctl.conf

https://github.com/google/syzkaller/blob/1c0e457aaaae0f885242810950c8a7c3508eabbb/dashboard/config/upstream.sysctl

[blackgnezdo]

I don't know how to abstract the interesting subset of OpenBSD configurations neatly. sysctl.conf is but one of the configuration mechanisms. rc.conf.local is another one and PF policy is one more. They all have different configuration file formats that aren't easy to unify into anything more coherent than a bunch of files.

As for storing whole files vs something more surgical, the extreme I want to avoid is that of abstracting the differences too far. The way it is now is:

   upstream default <- worker image default

I want to extend it one more step:

    upstream default <- worker image default
                     <- overlay

The settings in worker image default are already "diffs" to upstream because the upstream values are overridden. I want overlay to be at the same level rather than being diffs to worker image default. It appears more direct this way.

[dvyukov]

Let's try this.