Skip to content

VM state is not updated when it is stopped unexpectedly on hypervisor #10406

@weizhouapache

Description

@weizhouapache

problem

When vm is stopped unexpectedly on hypervisor (for example, destroy on hypervisor or stop vm inside it), ACS should detect the poweroff of vm and update vm to Stopped state.

This works in 4.19.x and 4.20.0.0. But in 4.20-snapshot, it does not work

versions

4.20-snapshot
kvm
NFS storage as primary

The steps to reproduce the bug

steps to reproduce the issue

  • deploy a vm and start it on kvm host
  • destroy the vm on the kvm host using virsh destroy command

expect result:

  • vm is updated to Stopped.
  • if HA is enabled, ACS starts it.

(4.20.0.0)

2025-02-17 08:47:07,443 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-1:[]) (logid:) vm id: 43 - time since last state update(40442ms) has passed graceful period.
2025-02-17 08:47:07,447 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-1:[]) (logid:) VM state report is updated. host: 1, vm id: 43, power state: PowerReportMissing.
2025-02-17 08:47:07,459 INFO  [c.c.v.ClusteredVirtualMachineManagerImpl] (AgentManager-Handler-1:[ctx-dacb4fbf]) (logid:) VM i-2-43-VM is at Running and we received a PowerReportMissing report while there is no pending jobs on it
...
2025-02-17 08:47:07,531 INFO  [c.c.v.ClusteredVirtualMachineManagerImpl] (AgentManager-Handler-1:[ctx-dacb4fbf]) (logid:) VM i-2-43-VM is sync-ed to at Stopped state according to power-off report from hypervisor.

actual result:

  • vm state is not updated
2025-02-17T12:09:57,704 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-15:[]) (logid:) Detected missing VM. host: 3, vm id: 1241(f4503851-f6a9-4b57-9cd9-a393f301afcf), power state: PowerReportMissing, last state update: 2025-02-17T12:09:17+0000
2025-02-17T12:09:57,704 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-15:[]) (logid:) vm id: 1241 - time since last state update(40704 ms) has passed graceful period
...
(no further logs)

What to do about it?

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions