dashboard/config/openbsd: increase the number of procs

[a-nogikh]

Currently OpenBSD instances are underutilized because of using only 2
syz-executor processes per VM. Change it to 6 to match the number of
procs on other syzbot instances.

[a-nogikh]

Hmm, I deployed it and it began to continuously crash as

executor 5: exit status 67
SYZFAIL: tun_id out of range
tun_id=5 (errno 2: No such file or directory)
loop exited with status 67

I'll set it to 4 for now (this is the MAX_TUN value). What can go wrong if we increase MAX_TUN?

[codecov]

Codecov Report

Merging #2942 (2453714) into master (51cf8ae) will not change coverage.
The diff coverage is n/a.

[mptre]

On Thu, Dec 23, 2021 at 02:50:07AM -0800, Aleksandr Nogikh wrote: Hmm, I deployed it and it began to continuously crash as ``` executor 5: exit status 67 SYZFAIL: tun_id out of range tun_id=5 (errno 2: No such file or directory) loop exited with status 67 ``` I'll set it to 4 for now (this is the `MAX_TUN` value). What can go wrong if we increase `MAX_TUN`?

Only 4 tun devices are created by default. Something like this needs to be added to tools/create-openbsd-vmm-worker.sh: cd /dev && for i in `jot - 0 10`; do sh MAKEDEV tun$i; done

[a-nogikh]

I've also added a commit than does what you say, thanks!
Now rebuilding the OpenBSD image with the updated script. If everything's fine, I'll merge this PR.

[a-nogikh]

Hmm, no, in its current state it did not create those tun devices at boot time. Though when I execute that command manually, it does what it should do.

I'm afraid I won't be able to finish and deploy it now as I'm already running out of time. I'll be back from vacation in mid Jan and then will complete it.

[mptre]

Since the heredoc is treated as a literal, the escaped \$ in hostname \$(cat /etc/myname) must be dropped. Otherwise it's a syntax error and probably why the subsequent MAKEDEV invocation does not run.

[a-nogikh]

Yes, that was exactly the reason. Thanks for noting!

[blackgnezdo]

I'm late to this party, but it appears we got the name wrong. The actual devices we use are /dev/tap%d. Hence https://syzkaller.appspot.com/bug?extid=5ff17a738451a0a08d4a
I'll send a quick 2 char fix :)

[a-nogikh]

I'm late to this party, but it appears we got the name wrong. The actual devices we use are /dev/tap%d. Hence https://syzkaller.appspot.com/bug?extid=5ff17a738451a0a08d4a I'll send a quick 2 char fix :)

Yes, I noticed that after deployment and have regenerated the image after a local fix.

What makes me wonder is that the total number of executed calls per day has not increased. When I log in to the VMs, they still don't seem to be overloaded most of the time - everything just like it was when I recorded traces from them.

It's as if there are alternating periods when all procs are mostly sleeping and when they're all super active at the same time and compete hard for the CPU time...

[a-nogikh]

FTR

Hmm, looked at Linux instances - same picture.

This all is just a hypothesis, but I think this might have to do with the design of syz-fuzzer. It uses a single work queue and it apparently shifts its targets slowly - especially given that most of the executions are not completely new programs but playing with the old ones (mutating etc). And programs have completely different real vs CPU time ratio - some calls are blocking and syzkaller sleeps until they time out, some calls cause syzkaller to await extra coverage, etc. So there must be long waves of slow programs (when exec total barely grows and CPU is mostly idle) and short waves of fast programs (when CPU is very busy). The problem is that these waves cover all procs at the same time.

@dvyukov what do you think?

Ideally we could start more procs when the system is mostly idle (actually most of the time) and freeze some of them when we face a bottleneck. Or maybe split procs into several work queues, e.g. one queue per 2-4 procs..