colima takes ~30s to boot

[Atemu]

Describe the Bug

A clear and concise description of what the bug is.

$ time colima start
INFO[0000] using docker runtime                         
INFO[0000] starting colima                              
INFO[0000] starting ...                                  context=vm
INFO[0022] provisioning ...                              context=docker
INFO[0022] starting ...                                  context=docker
INFO[0027] waiting for startup to complete ...           context=docker
INFO[0027] done                                         
  
real	0m27.816s
user	0m0.191s
sys	0m0.214s

Version

Colima Version:

colima version development
git commit: unknown

runtime: docker
arch: aarch64
client: v20.10.14
server: v20.10.11

What is the output of colima version

Lima Version:

limactl version 0.10.0

What is the output of limactl --version

Qemu Version

What is the output of qemu-img --version

qemu-img version 7.0.0
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Operating System

• macOS Intel
• macOS m1
• Linux

To Reproduce

Steps to reproduce the behavior:

1. time colima start

Expected behavior

A clear and concise description of what you expected to happen.

It should start in about a third of the time.

Additional context

Add any other context about the problem here.

Booting a minimal NixOS VM takes about 10 seconds in UTM and I can't imagine docker needing much time either. Something must be off about the Alpine configuration.

[Atemu]

When I start the machine manually via the qemu command colima uses, the login prompt appears in <10s.

[abiosoft]

If you're building from source, kindly use the makefile to have actual version attached to the binary.

make         # binary will be present in _output/binaries
make install # if you want it in PATH

[abiosoft]

Are you referring to the initial boot or subsequent boots?

[Atemu]

Ah, sorry, this is packaged by Nix. Don't know why the version doesn't show up. Is there a flag we need to set in order for the release to know its version?

Version is 0.3.4.

I'm referring to subsequent boots. This time is reproducible.

[abiosoft]

When I start the machine manually via the qemu command colima uses, the login prompt appears in <10s

This is not an accurate comparison, getting the login prompt directly from Qemu does not imply that Lima is done starting.

To monitor the boot, tail the serial log during startup.

tail -f ~/.lima/colima/serial.log

Considering that Colima startup is not something expected to be frequently repeated, this is not a pressing issue (if at all an issue).

Nonetheless, will keep an eye on ways to improve the boot time.

Thanks.

[abiosoft]

Ah, sorry, this is packaged by Nix. Don't know why the version doesn't show up. Is there a flag we need to set in order for the release to know its version?

Looks like the Nix package needs to be fixed. cc @aaschmid.

[Atemu]

If you can tell me how the colima build system decides which version to embed, I could fix that too.

How long does it take to start on your machine?

[abiosoft]

If you can tell me how the colima build system decides which version to embed, I could fix that too.

The ldflags need to be specified

colima/scripts/build.sh

Line 15 in 784cec0

-ldflags "-X ${PACKAGE}.appVersion=${VERSION} -X ${PACKAGE}.revision=${REVISION}" \

How long does it take to start on your machine?

~ 30secs on an Intel MacBook.

[abiosoft]

@Atemu Thanks :)

Kindly note that the package is also a bit outdated, the latest version is v0.4.2 as opposed to nixpkg's v0.3.4.
Not sure if that will require a separate PR.

[Atemu]

Yeah, that'll need a separate PR. Are there any major changes to dependencies, build process etc. to watch out for?

Perhaps it's related to some sort of timeout because we're trying to ssh before it's up or something?

[abiosoft]

Yeah, that'll need a separate PR. Are there any major changes to dependencies, build process etc. to watch out for?

Not at all, no changes.

Perhaps it's related to some sort of timeout because we're trying to ssh before it's up or something?

Yeah exactly.

[aaschmid]

@Atemu: sorry for being late with update of colima for nix. I created a branch updating to the latest 0.4.2 version which you can try see https://github.com/NixOS/nixpkgs/compare/master...aaschmid:colima.0.4.2?expand=1. Please note that the test is failing and you therefore need to add doCheck = false in addition.

@abiosoft: Do you have an idea why it fails? Otherwise I would need to dive deeper w/o having time for it...
The output on my machine is

running tests
=== RUN   Test_mountsFromFlag
=== RUN   Test_mountsFromFlag/0
=== RUN   Test_mountsFromFlag/1
=== RUN   Test_mountsFromFlag/2
--- PASS: Test_mountsFromFlag (0.00s)
    --- PASS: Test_mountsFromFlag/0 (0.00s)
    --- PASS: Test_mountsFromFlag/1 (0.00s)
    --- PASS: Test_mountsFromFlag/2 (0.00s)
PASS
ok      github.com/abiosoft/colima/cmd  0.340s
=== RUN   TestStart
    daemon_test.go:31: pidfile /private/tmp/nix-build-colima-0.4.2.drv-1/TestStart4052351483/001/daemon.pid
    daemon_test.go:46: start successful
    daemon_test.go:53: pid file not found: stat /private/tmp/nix-build-colima-0.4.2.drv-1/TestStart4052351483/001/daemon.pid: no such file or directory
--- FAIL: TestStart (5.00s)
FAIL
FAIL    github.com/abiosoft/colima/cmd/daemon   5.245s
FAIL

[abiosoft]

Not sure precisely why it is failing, currently trying to set up a nix based environment on Linux to see if I can reproduce it.

[aaschmid]

I can also try to provide more information. Do you have some tips for me, how to easily to this? Maybe enabling some debugging infos while building. Another lightweight alternative could be using docker, see https://nixos.org/download.html#nix-install-docker. My problem however is based on MacOS using nix as package manager

[abiosoft]

Apparently, some behaviours are different on macOS and Linux. Will you able to test the changes if it's on the main branch or will it require a new release?

[aaschmid]

I can test every revision, so a feature branch would also work :-)

[abiosoft]

@aaschmid can you try the fix-nix-tests branch?

[aaschmid]

Didn't come back after about 2 minutes. Need to leave and can check later again...

[abiosoft]

Didn't come back after about 2 minutes. Need to leave and can check later again...

Alright, thanks. I will dig a bit further. FYI it builds fine on macOS and Linux (with/without Nix) for me.

[aaschmid]

Sounds like a problem on my side. I created a draft PR and will look on the result of the build: NixOS/nixpkgs#174013

[abiosoft]

Created #304 as this thread has deviated from its original issue.