Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: optimize FreeBSD image boot time #22894

Closed
bradfitz opened this issue Nov 28, 2017 · 11 comments

Comments

Projects
None yet
3 participants
@bradfitz
Copy link
Member

commented Nov 28, 2017

I noticed while testing my images for #19303 and #22854 that our FreeBSD images have a 10 second boot prompt countdown. (I watched it count down with cute art on "serial" over the remote serial API)

Let's remove that 10 second countdown and get FreeBSD booting faster. 10 seconds is a notable percentage of the time our VMs live.

/cc @paulzhol

@paulzhol

This comment has been minimized.

Copy link
Member

commented Nov 28, 2017

The images need to have /boot/loader.conf appended with autoboot_delay=0 and maybe beastie_disable=YES lines. I'll can send a CL tonight.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2017

Thanks!

@paulzhol paulzhol self-assigned this Nov 28, 2017

@gopherbot

This comment has been minimized.

Copy link

commented Nov 28, 2017

Change https://golang.org/cl/80335 mentions this issue: env/freebsd-amd64: disable boot prompt and logo

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2017

While running make.bash with this latest code on 11.1, I noticed a kernel crash:

===> Creating users
Creating user 'git_daemon' with uid '964'.
Extracting git-2.14.1:  98%panic: ufs_dirbad: /: bad dir ino 1048795 at offset 0
: mangled entry
cpuid = 1
KDB: stack backtrace:
#0 0xffffffff80aada97 at kdb_backtrace+0x67
#1 0xffffffff80a6bb76 at vpanic+0x186
#2 0xffffffff80a6b9e3 at panic+0x43
#3 0xffffffff80d3e0c9 at ufs_lookup_ino+0xe89
#4 0xffffffff8104b963 at VOP_CACHEDLOOKUP_APV+0x83
#5 0xffffffff80b13786 at vfs_cache_lookup+0xd6
#6 0xffffffff8104b853 at VOP_LOOKUP_APV+0x83
#7 0xffffffff80b1d0c1 at lookup+0x701
#8 0xffffffff80b1c576 at namei+0x486
#9 0xffffffff80b3772c at vn_open_cred+0xcc
#10 0xffffffff80b30e5e at kern_openat+0x1fe
#11 0xffffffff80ee0394 at amd64_syscall+0x6c4
#12 0xffffffff80ec392b at Xfast_syscall+0xfb
Uptime: 7m29s
Automatic reboot in 15 seconds - press a key on the console to abort

And then it rebooted:

cd0: Serial Number QM00003
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: 0MB (185 2048 byte sectors)
GEOM: vtbd0: the secondary GPT header is not in the last LBA.
random: unblocking device.
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/gpt/rootfs [rw]...
WARNING: / was not properly dismounted
Setting hostuuid: 73dce969-d480-11e7-99d3-b15cb8c9d93f.
Setting hostid: 0x281f1cb5.
Entropy harvesting: interrupts ethernet point_to_point swi.
warning: total configured swap (262144 pages) exceeds maximum recommended amount
(204672 pages).
warning: increase kern.maxswzone or reduce amount of swap.
Starting file system checks:
/dev/gpt/rootfs: UNALLOCATED  I=1765670  OWNER=root MODE=100644
/dev/gpt/rootfs: SIZE=0 MTIME=Nov 28 21:09 2017
/dev/gpt/rootfs: FILE=/var/log/utx.lastlogin

/dev/gpt/rootfs: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY.
Automatic file system check failed; help!
ERROR: ABORTING BOOT (sending SIGTERM to parent)!
Nov 28 21:09:45 init: /bin/sh on /etc/rc terminated abnormally, going to single
user mode
Enter full pathname of shell or RETURN for /bin/sh:

Fun.

Want to file an upstream FreeBSD kernel bug?

@paulzhol

This comment has been minimized.

Copy link
Member

commented Nov 28, 2017

yikes! does it reproduce? You're running this inside a GCE linux VM right? (When I tried locally I flipped the --enable-kvm flag to speed things up, and it was fine).

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2017

Yes, I'm in a GCE Debian Jessie VM, running your make.bash unmodified. (so qemu with whatever options you specified)

I've seen it twice now: in both 11.1, and 10.3.

I'm in a GNU Screen session, if that matters, and both times I saw it happened might've been while I was flipping between virtual terminals. (Does screen send some signal or keypess when I do that which is confusing qemu?)

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2017

But I just re-ran ./make.bash 11.1 and it succeeded now.

So it's not 100% of the time.

@paulzhol

This comment has been minimized.

Copy link
Member

commented Nov 28, 2017

I'm in a GNU Screen session, if that matters, and both times I saw it happened might've been while I was flipping between virtual terminals. (Does screen send some signal or keypess when I do that which is confusing qemu?)

with screen and qemu is running with -curses under expect there could be something terminal related.

But this

===> Creating users
Creating user 'git_daemon' with uid '964'.

came from a regular pkg install of git. I'll try to reproduce, and get a kernel dump if I can.

@gopherbot

This comment has been minimized.

Copy link

commented Nov 28, 2017

Change https://golang.org/cl/80304 mentions this issue: dashboard: rebuild FreeBSD VM to avoid 10 second boot prompt

gopherbot pushed a commit to golang/build that referenced this issue Nov 28, 2017

dashboard: rebuild FreeBSD VM to avoid 10 second boot prompt
FreeBSD VMs boot up 10 seconds faster now, and shut down faster.

Updates golang/go#22894
Updates golang/go#22895

Change-Id: I8eef979fc43d95d3c03104a761e70ec5737a3a97
Reviewed-on: https://go-review.googlesource.com/80304
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@paulzhol

This comment has been minimized.

Copy link
Member

commented Nov 28, 2017

@bradfitz does adding -m 1G to the qemu command help? Without specifying it qemu defaults to 128MiB, it might be a bit low.

diff --git a/env/freebsd-amd64/make.bash b/env/freebsd-amd64/make.bash
index 0df60bd..201d3bc 100755
--- a/env/freebsd-amd64/make.bash
+++ b/env/freebsd-amd64/make.bash
@@ -148,7 +148,7 @@ sleep 2
 # TODO(wathiede): set serial output so we can track boot on GCE.
 expect <<EOF
 set timeout 600
-spawn qemu-system-x86_64 -display curses -smp 2 -drive if=virtio,file=disk.raw,cache=none -cdrom config.iso -net nic,model=virtio -net user
+spawn qemu-system-x86_64 -display curses -smp 2 -m 1G -drive if=virtio,file=disk.raw,cache=none -cdrom config.iso -net nic,model=virtio -net user
 
 # Speed-up boot by going in to single user mode.
 expect "Welcome to FreeBSD"
@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2017

I can try that next time I update the FreeBSD images, but now I have to update some other operating systems.

@golang golang locked and limited conversation to collaborators Nov 28, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.