Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FATA[0600] error starting vm: error at 'creating and starting': exit status 1 #787

Closed
5 tasks
MustafaAbdelmonem opened this issue Aug 26, 2023 · 15 comments
Closed
5 tasks

Comments

@MustafaAbdelmonem
Copy link

Description

Hi Team, I am new to colima. I am trying to get it up on macos with M2 chip.

It is getting stuck at [hostagent] Waiting for the essential requirement 1 of 5: "ssh".

I have tried colima stop --force and colima delete and start again as well but same output every time.

I am starting as - "colima start -c 4 -m 12 -a x86_64 --very-verbose"
below is the output.

colima start -c 4 -m 12 -a x86_64 --very-verbose
TRAC[0000] cmd ["limactl" "info"]
TRAC[0000] cmd ["limactl" "list" "colima" "--json"]
TRAC[0000] error retrieving running instance: instance 'colima' does not exist
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ... context=vm
TRAC[0000] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0000] cmd ["/opt/homebrew/bin/colima" "daemon" "start" "default" "--gvproxy" "--very-verbose"]
INFO[0000] creating and starting ... context=vm
TRAC[0000] cmd ["limactl" "start" "--tty=false" "/var/folders/ct/5g8zpgbj187171z4jj85y4xw0000gn/T/colima.yaml"]

Terminal is not available, proceeding without opening an editor
"Attempting to download the image" arch=x86_64 digest="sha512:f761b807fe9ba345968df72c07f8c5abcae0c4a44976fe5595c0ff748ef693841221a70e663986c700b027cea32b7cac24d5490d4c721593c39f2b8840c362a2" location="https://github.com/abiosoft/alpine-lima/releases/download/colima-v0.5.5/alpine-lima-clm-3.18.0-x86_64.iso"
Using cache "/Users/mustafaawad/Library/Caches/lima/download/by-url-sha256/50d875a98d6487107ea1ce984f1b8e6ebbddc0e8da0347334649c2c46ca9a84d/data"
[hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/mustafaawad/.lima/colima/serial*.log")
SSH Local Port: 52041
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
[hostagent] Waiting for the essential requirement 1 of 5: "ssh"
did not receive an event with the "running" status
FATA[0600] error starting vm: error at 'creating and starting': exit status 1

Version

Colima Version: colima version 0.5.5
Lima Version: 0.17.2
Qemu Version: 8.1.0

Operating System

  • macOS Intel <= 12 (Monterrey)
  • macOS Intel >= 13 (Ventura)
  • macOS M1 <= 12 (Monterrey)
  • macOS M1 >= 13 (Ventura)
  • Linux

Output of colima status

FATA[0074] error retrieving current runtime: empty value

Reproduction Steps

  1. brew install colima
  2. brew install qemu
  3. "colima start -c 4 -m 12 -a x86_64 --very-verbose"

Expected behaviour

No response

Additional context

No response

@AkihiroSuda
Copy link
Contributor

AkihiroSuda commented Aug 27, 2023

[hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/mustafaawad/.lima/colima/serial*.log")

Do you see anything in these logs

@m-garrido
Copy link

I've the same problem since i've upgrade to colima 0.5.5.
I can't launch a x86_64 instance.
I'm in a mac book pro m2 on ventura 13.5.1.
Here last logs from colima:

[ 1.349238] zswap: loaded using pool lzo/zbud
[ 1.350561] Key type .fscrypt registered
[ 1.350664] Key type fscrypt-provisioning registered
[ 1.358161] Unstable clock detected, switching default tracing clock to "global"
[ 1.358161] If you want to keep using the local clock, then add:
[ 1.358161] "trace_clock=local"
[ 1.358161] on the kernel command line
[ 1.698486] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[ 6.744267] ata1.00: qc timeout after 5000 msecs (cmd 0xa1)
[ 6.745474] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)

@m-garrido
Copy link

I've downgraded qemu from 8.1.0 to 8.0.4 and colima for x86-64 instance works fine on mac book arm.
So, the issue seems to be coming from qemu instead of colima.

@dxtich
Copy link

dxtich commented Aug 29, 2023

The same problem, --arch x86_64
mac book pro m2 ventura 13.5.1

@m-garrido How did you downgrade qemu to make colima works with old version?

@jledesma84
Copy link

Same problem here. @dxtich any luck trying to fix it?

@rainasingh02
Copy link

Same issue here. Please let me know if anyone has a workaround

@dxtich
Copy link

dxtich commented Aug 30, 2023

@jledesma84 @rainasingh02
workaround works for me:

brew install colima
brew uninstall --ignore-dependencies qemu
brew install dxtich/core/qemu@8.0.4

@rainasingh02
Copy link

@dxtich brew install dxtich/core/qemu@8.0.4 this doesn't run for me. Is there any prerequisite for this step?

@dxtich
Copy link

dxtich commented Aug 30, 2023

@rainasingh02 brew tap dxtich/core

@MustafaAbdelmonem
Copy link
Author

@AkihiroSuda the logs of "/Users/mustafaawad/.lima/colima/serial*.log"

[ 121.002880] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 121.005576] rcu: 0-...!: (0 ticks this GP) idle=12c0/0/0x0 softirq=40/40 fqs=0 (false positive?)
[ 121.006799] (detected by 1, t=6003 jiffies, g=-1163, q=2001 ncpus=2)
[ 121.007478] Sending NMI from CPU 1 to CPUs 0:
[ 61.008703] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0xb/0x10
[ 121.009183] rcu: rcu_preempt kthread timer wakeup didn't happen for 6002 jiffies! g-1163 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[ 121.011315] rcu: Possible timer handling issue on cpu=0 timer-softirq=15
[ 121.012032] rcu: rcu_preempt kthread starved for 6003 jiffies! g-1163 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
[ 121.012705] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[ 121.012705] rcu: RCU grace-period kthread stack dump:
[ 121.012705] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00004000
[ 121.012705] Call Trace:
[ 121.012705]
[ 121.012705] ? rcu_gp_cleanup+0x470/0x470
[ 121.012705] __schedule+0x362/0x1380
[ 121.012705] ? get_nohz_timer_target+0x13/0x1a0
[ 121.012705] ? _raw_spin_unlock_irqrestore+0x22/0x60
[ 121.012705] ? __mod_timer+0x281/0x3d0
[ 121.012705] ? rcu_gp_cleanup+0x470/0x470
[ 121.012705] schedule+0x5c/0xe0
[ 121.012705] schedule_timeout+0x8f/0x160
[ 121.012705] ? __bpf_trace_tick_stop+0x10/0x10
[ 121.012705] rcu_gp_fqs_loop+0x13c/0x4c0
[ 121.012705] rcu_gp_kthread+0xcb/0x190
[ 121.012705] kthread+0xe4/0x110
[ 121.012705] ? kthread_complete_and_exit+0x20/0x20
[ 121.012705] ret_from_fork+0x22/0x30
[ 121.012705]
[ 121.012705] rcu: Stack dump where RCU GP kthread last ran:
[ 121.012705] Sending NMI from CPU 1 to CPUs 0:
[ 61.008703] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0xb/0x10

@MustafaAbdelmonem
Copy link
Author

@dxtich thanks solved with your workaround 🙏

@AkihiroSuda
Copy link
Contributor

@MustafaAbdelmonem
I saw you closed this, but could you report this to QEMU so that the issue can be properly fixed?
https://gitlab.com/qemu-project/qemu/-/issues

@Micky002
Copy link

I also have the same problem, the fix with downgrading qemu to 8.0.4 worked for me. @dxtich thanks for providing the old qemu brew formula 🙏🏼

@jledesma84 @rainasingh02 workaround works for me:

brew install colima
brew uninstall --ignore-dependencies qemu
brew install dxtich/core/qemu@8.0.4

@jledesma84
Copy link

@MustafaAbdelmonem I saw you closed this, but could you report this to QEMU so that the issue can be properly fixed? https://gitlab.com/qemu-project/qemu/-/issues

@m-garrido Is this a QEMU or Colima issue?

@kamilmodest
Copy link

kamilmodest commented Apr 19, 2024

Putting the link here for the history.
Someone mentioned this bug in QEMU's issues: https://gitlab.com/qemu-project/qemu/-/issues/1864
Looks like it should be fixed since qumu version 8.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants