problem
I have two clusters (pool-01, pool-02) in my zone (zone-1).
Both clusters are running on xcp-ng 8.3.0.
I want to move the System VM to another cluster in the zone, but when I delete or disable the System VM, it still starts up on the pool-2 cluster, and I need it to be on pool-01.
When I put the pool-02 cluster into maintenance mode and delete the System VM, it fails to start, and I receive the following error log [Cluster pool-01 has sufficient resources, System Offering For Secondary Storage VM and System Offering For Console Proxy do not have Storage tags and Host tags]:
[root@cs-vm management]# cat management-server.log | grep 's-9509-VM'
2026-05-25 16:26:26,895 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) allocating virtual machine from template: Template {"format":"VHD","id":858,"name":"systemvm-xenserver-4.22.0-x86_64","uniqueName":"858-287-e2e2b3e3-f7b9-36cb-80a5-69cb111b0a1c","uuid":"be5dad8c-4487-4b7a-b2f7-996054f48ec7"} with hostname: s-9509-VM and 3 networks
2026-05-25 16:26:26,898 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating entries for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,900 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nics for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,902 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-05-25 16:26:26,979 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3} with requested profile null
2026-05-25 16:26:26,992 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2} with requested profile null
2026-05-25 16:26:27,005 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating disks for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,007 INFO [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:[ctx-61b7001a, ctx-af6e6645]) (logid:42f136a6) Adding disk object [ROOT-9509] to VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}]
2026-05-25 16:26:27,038 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocation completed for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,041 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to deploy VM [error decoding VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-05-25 16:26:27,043 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}], pod [null], cluster [null], to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with requested CPU [3980] and requested RAM [(4.00 GB) 4294967296].
2026-05-25 16:26:27,044 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) ROOT volume [Volume {"id":10906,"instanceId":9509,"name":"ROOT-9509","uuid":"5d5e0fc3-e90d-4233-9e9c-f17b71ac690d","volumeType":"ROOT"}] is not ready to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}].
2026-05-25 16:26:27,052 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}], avoiding pods [], clusters [3] and hosts [null].
2026-05-25 16:26:27,088 WARN [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) No suitable host found for VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with tags null in zone [1], pod [1], cluster [1].
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}Scope=interface com.cloud.dc.DataCenter; id=1
[root@cs-vm management]# cat management-server.log | grep '42f136a6'
2026-05-25 16:26:26,788 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Enabled non-edge zones available for scan: 1,5,7
2026-05-25 16:26:26,795 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Zone [1] is ready to launch secondary storage VM.
2026-05-25 16:26:26,801 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No running secondary storage vms found in datacenter id=1, starting one
2026-05-25 16:26:26,806 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocate secondary storage VM standby capacity for zone [1].
2026-05-25 16:26:26,814 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No stopped secondary storage VM is available, need to allocate a new secondary storage VM.
2026-05-25 16:26:26,816 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Assign secondary storage VM from a newly started instance for request from data center [1].
2026-05-25 16:26:26,837 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":1,"name":"System-Public-Network","trafficType":"Public","uuid":"d6f9db9a-d672-409c-aa85-d39dda649706"}: Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1}
2026-05-25 16:26:26,837 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,850 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":3,"name":"System-Control-Network","trafficType":"Control","uuid":"4e8130b7-a320-4cd9-8783-2c26b57c5412"}: Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3}
2026-05-25 16:26:26,851 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,862 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":2,"name":"System-Management-Network","trafficType":"Management","uuid":"53ce891e-b0ac-4349-a30c-10d807e8e151"}: Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2}
2026-05-25 16:26:26,863 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,895 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) allocating virtual machine from template: Template {"format":"VHD","id":858,"name":"systemvm-xenserver-4.22.0-x86_64","uniqueName":"858-287-e2e2b3e3-f7b9-36cb-80a5-69cb111b0a1c","uuid":"be5dad8c-4487-4b7a-b2f7-996054f48ec7"} with hostname: s-9509-VM and 3 networks
2026-05-25 16:26:26,898 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating entries for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,900 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nics for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,902 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-05-25 16:26:26,918 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Public IP address [IPAddress {"address":"53.20.86.20","dataCenterId":1,"id":101,"uuid":"33a1ee3a-2e80-42f3-81df-7fe9305157d9"}] is not in quarantine; therefore, it is allowed to be allocated.
2026-05-25 16:26:26,919 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) trying ip address 53.20.86.20
2026-05-25 16:26:26,924 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) locked row for ip address 53.20.86.20 (id: 33a1ee3a-2e80-42f3-81df-7fe9305157d9)
2026-05-25 16:26:26,925 INFO [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) successfully allocated ip address 53.20.86.20
2026-05-25 16:26:26,933 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) IP address [IPAddress {"address":"53.20.86.20","dataCenterId":1,"id":101,"uuid":"33a1ee3a-2e80-42f3-81df-7fe9305157d9"}] is direct; therefore, the resource count should not be updated.
2026-05-25 16:26:26,979 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3} with requested profile null
2026-05-25 16:26:26,992 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2} with requested profile null
2026-05-25 16:26:27,005 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating disks for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,007 INFO [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:[ctx-61b7001a, ctx-af6e6645]) (logid:42f136a6) Adding disk object [ROOT-9509] to VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}]
2026-05-25 16:26:27,038 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocation completed for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,041 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to deploy VM [error decoding VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-05-25 16:26:27,043 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}], pod [null], cluster [null], to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with requested CPU [3980] and requested RAM [(4.00 GB) 4294967296].
2026-05-25 16:26:27,044 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) ROOT volume [Volume {"id":10906,"instanceId":9509,"name":"ROOT-9509","uuid":"5d5e0fc3-e90d-4233-9e9c-f17b71ac690d","volumeType":"ROOT"}] is not ready to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}].
2026-05-25 16:26:27,047 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding pods [] to the avoid set because these pods are in the Disabled state.
2026-05-25 16:26:27,049 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [] of pod [1] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,049 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [3] of pod [2] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,050 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [] of pod [11] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,051 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding hosts [] of datacenter [a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252] to the avoid set, because these hosts are in the Disabled state.
2026-05-25 16:26:27,052 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}], avoiding pods [], clusters [3] and hosts [null].
2026-05-25 16:26:27,053 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Deploy avoids pods: [], clusters: [3], hosts: null.
2026-05-25 16:26:27,053 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Deploy hosts with priorities {}, hosts have NORMAL priority by default
2026-05-25 16:26:27,055 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Searching all possible resources under this Zone: Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}
2026-05-25 16:26:27,056 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-05-25 16:26:27,059 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) CapacityType: CPU is used for Cluster ordering
2026-05-25 16:26:27,061 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Removing from the clusterId list these clusters from avoid set: [3]
2026-05-25 16:26:27,081 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Checking resources in Cluster: Cluster {id: "1", name: "pool-1", uuid: "ffdee894-1833-499e-b640-09eb251be9be"} under Pod: HostPod {"id":1,"name":"xcp-sr121-pod-01","uuid":"c5c4c071-79ab-4b92-a6d6-da6c5f373638"}
2026-05-25 16:26:27,083 INFO [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) Guest VM is requested with Custom[UEFI] Boot Type false
2026-05-25 16:26:27,083 DEBUG [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) Looking for hosts in zone [1], pod [1], cluster [1]
2026-05-25 16:26:27,088 WARN [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) No suitable host found for VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with tags null in zone [1], pod [1], cluster [1].
2026-05-25 16:26:27,089 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No suitable hosts found.
2026-05-25 16:26:27,089 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No suitable hosts found under this Cluster: Cluster {id: "1", name: "pool-1", uuid: "ffdee894-1833-499e-b640-09eb251be9be"}
2026-05-25 16:26:27,097 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Could not find suitable Deployment Destination for this VM under any clusters, returning.
2026-05-25 16:26:27,099 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Searching all possible resources under this Zone: Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}
2026-05-25 16:26:27,100 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-05-25 16:26:27,103 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) CapacityType: CPU is used for Cluster ordering
2026-05-25 16:26:27,105 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Removing from the clusterId list these clusters from avoid set: [1, 3]
2026-05-25 16:26:27,106 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No clusters found after removing disabled clusters and clusters in avoid list, returning.
2026-05-25 16:26:27,108 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) received secondary storage vm alert
2026-05-25 16:26:27,109 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Secondary Storage Vm creation failure in zone [zone-1].
2026-05-25 16:26:27,111 WARN [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) alertType=[19] dataCenterId=[1] podId=[null] clusterId=[null] message=[Secondary Storage Vm creation failure in zone [zone-1]. Error details: Unable to allocate capacity on zone [1] due to [null].].
2026-05-25 16:26:27,142 WARN [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Secondary Storage Vm creation failure in zone [zone-1]. Error details: Unable to allocate capacity on zone [1] due to [null].] and content [Secondary Storage Vm creation failure (zone zone-1)].
2026-05-25 16:26:27,142 WARN [c.c.v.SystemVmLoadScanner$1] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Unexpected exception Failed to allocate secondary storage VM [%s] in zone [%s] with available templates com.cloud.utils.exception.CloudRuntimeException: Failed to allocate secondary storage VM [%s] in zone [%s] with available templates.
Please help me.
versions
CloudStack 4.22.0.0
xcp-ng 8.3.0
The steps to reproduce the bug
- I am putting the pool-2 cluster into maintenance mode.
- Deleting secondarystoragevm.
...
What to do about it?
No response
problem
I have two clusters (pool-01, pool-02) in my zone (zone-1).
Both clusters are running on xcp-ng 8.3.0.
I want to move the System VM to another cluster in the zone, but when I delete or disable the System VM, it still starts up on the pool-2 cluster, and I need it to be on pool-01.
When I put the pool-02 cluster into maintenance mode and delete the System VM, it fails to start, and I receive the following error log [Cluster pool-01 has sufficient resources, System Offering For Secondary Storage VM and System Offering For Console Proxy do not have Storage tags and Host tags]:
[root@cs-vm management]# cat management-server.log | grep 's-9509-VM'
2026-05-25 16:26:26,895 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) allocating virtual machine from template: Template {"format":"VHD","id":858,"name":"systemvm-xenserver-4.22.0-x86_64","uniqueName":"858-287-e2e2b3e3-f7b9-36cb-80a5-69cb111b0a1c","uuid":"be5dad8c-4487-4b7a-b2f7-996054f48ec7"} with hostname: s-9509-VM and 3 networks
2026-05-25 16:26:26,898 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating entries for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,900 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nics for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,902 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-05-25 16:26:26,979 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3} with requested profile null
2026-05-25 16:26:26,992 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2} with requested profile null
2026-05-25 16:26:27,005 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating disks for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,007 INFO [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:[ctx-61b7001a, ctx-af6e6645]) (logid:42f136a6) Adding disk object [ROOT-9509] to VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}]
2026-05-25 16:26:27,038 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocation completed for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,041 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to deploy VM [error decoding VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-05-25 16:26:27,043 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}], pod [null], cluster [null], to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with requested CPU [3980] and requested RAM [(4.00 GB) 4294967296].
2026-05-25 16:26:27,044 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) ROOT volume [Volume {"id":10906,"instanceId":9509,"name":"ROOT-9509","uuid":"5d5e0fc3-e90d-4233-9e9c-f17b71ac690d","volumeType":"ROOT"}] is not ready to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}].
2026-05-25 16:26:27,052 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}], avoiding pods [], clusters [3] and hosts [null].
2026-05-25 16:26:27,088 WARN [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) No suitable host found for VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with tags null in zone [1], pod [1], cluster [1].
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}Scope=interface com.cloud.dc.DataCenter; id=1
[root@cs-vm management]# cat management-server.log | grep '42f136a6'
2026-05-25 16:26:26,788 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Enabled non-edge zones available for scan: 1,5,7
2026-05-25 16:26:26,795 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Zone [1] is ready to launch secondary storage VM.
2026-05-25 16:26:26,801 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No running secondary storage vms found in datacenter id=1, starting one
2026-05-25 16:26:26,806 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocate secondary storage VM standby capacity for zone [1].
2026-05-25 16:26:26,814 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No stopped secondary storage VM is available, need to allocate a new secondary storage VM.
2026-05-25 16:26:26,816 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Assign secondary storage VM from a newly started instance for request from data center [1].
2026-05-25 16:26:26,837 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":1,"name":"System-Public-Network","trafficType":"Public","uuid":"d6f9db9a-d672-409c-aa85-d39dda649706"}: Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1}
2026-05-25 16:26:26,837 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,850 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":3,"name":"System-Control-Network","trafficType":"Control","uuid":"4e8130b7-a320-4cd9-8783-2c26b57c5412"}: Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3}
2026-05-25 16:26:26,851 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,862 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Found existing network configuration for offering NetworkOffering {"id":2,"name":"System-Management-Network","trafficType":"Management","uuid":"53ce891e-b0ac-4349-a30c-10d807e8e151"}: Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2}
2026-05-25 16:26:26,863 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"61645985-4224-11ec-8697-62c4e64309fe"}]
2026-05-25 16:26:26,895 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) allocating virtual machine from template: Template {"format":"VHD","id":858,"name":"systemvm-xenserver-4.22.0-x86_64","uniqueName":"858-287-e2e2b3e3-f7b9-36cb-80a5-69cb111b0a1c","uuid":"be5dad8c-4487-4b7a-b2f7-996054f48ec7"} with hostname: s-9509-VM and 3 networks
2026-05-25 16:26:26,898 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating entries for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,900 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nics for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:26,902 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 200, "name": "null", "uuid": "dd5b2e20-f958-42de-bc7a-9ccc1d510c00", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-05-25 16:26:26,918 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Public IP address [IPAddress {"address":"53.20.86.20","dataCenterId":1,"id":101,"uuid":"33a1ee3a-2e80-42f3-81df-7fe9305157d9"}] is not in quarantine; therefore, it is allowed to be allocated.
2026-05-25 16:26:26,919 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) trying ip address 53.20.86.20
2026-05-25 16:26:26,924 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) locked row for ip address 53.20.86.20 (id: 33a1ee3a-2e80-42f3-81df-7fe9305157d9)
2026-05-25 16:26:26,925 INFO [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) successfully allocated ip address 53.20.86.20
2026-05-25 16:26:26,933 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) IP address [IPAddress {"address":"53.20.86.20","dataCenterId":1,"id":101,"uuid":"33a1ee3a-2e80-42f3-81df-7fe9305157d9"}] is direct; therefore, the resource count should not be updated.
2026-05-25 16:26:26,979 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 202, "name": "null", "uuid": "02ce307f-0314-46ba-9de9-73c2236c5c39", "networkofferingid": 3} with requested profile null
2026-05-25 16:26:26,992 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating nic for vm VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"} in network Network {"id": 201, "name": "null", "uuid": "a2f26056-6feb-4dc4-ab0c-b6d84e4a66be", "networkofferingid": 2} with requested profile null
2026-05-25 16:26:27,005 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocating disks for VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,007 INFO [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:[ctx-61b7001a, ctx-af6e6645]) (logid:42f136a6) Adding disk object [ROOT-9509] to VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}]
2026-05-25 16:26:27,038 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Allocation completed for VM: VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}
2026-05-25 16:26:27,041 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to deploy VM [error decoding VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-05-25 16:26:27,043 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}], pod [null], cluster [null], to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with requested CPU [3980] and requested RAM [(4.00 GB) 4294967296].
2026-05-25 16:26:27,044 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) ROOT volume [Volume {"id":10906,"instanceId":9509,"name":"ROOT-9509","uuid":"5d5e0fc3-e90d-4233-9e9c-f17b71ac690d","volumeType":"ROOT"}] is not ready to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}].
2026-05-25 16:26:27,047 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding pods [] to the avoid set because these pods are in the Disabled state.
2026-05-25 16:26:27,049 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [] of pod [1] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,049 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [3] of pod [2] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,050 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding clusters [] of pod [11] to the void set because these clusters are in the Disabled state.
2026-05-25 16:26:27,051 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Adding hosts [] of datacenter [a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252] to the avoid set, because these hosts are in the Disabled state.
2026-05-25 16:26:27,052 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}], avoiding pods [], clusters [3] and hosts [null].
2026-05-25 16:26:27,053 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Deploy avoids pods: [], clusters: [3], hosts: null.
2026-05-25 16:26:27,053 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Deploy hosts with priorities {}, hosts have NORMAL priority by default
2026-05-25 16:26:27,055 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Searching all possible resources under this Zone: Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}
2026-05-25 16:26:27,056 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-05-25 16:26:27,059 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) CapacityType: CPU is used for Cluster ordering
2026-05-25 16:26:27,061 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Removing from the clusterId list these clusters from avoid set: [3]
2026-05-25 16:26:27,081 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Checking resources in Cluster: Cluster {id: "1", name: "pool-1", uuid: "ffdee894-1833-499e-b640-09eb251be9be"} under Pod: HostPod {"id":1,"name":"xcp-sr121-pod-01","uuid":"c5c4c071-79ab-4b92-a6d6-da6c5f373638"}
2026-05-25 16:26:27,083 INFO [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) Guest VM is requested with Custom[UEFI] Boot Type false
2026-05-25 16:26:27,083 DEBUG [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) Looking for hosts in zone [1], pod [1], cluster [1]
2026-05-25 16:26:27,088 WARN [c.c.a.m.a.i.FirstFitRoutingAllocator] (secstorage-1:[ctx-61b7001a, FirstFitRoutingAllocator]) (logid:42f136a6) No suitable host found for VM [VM instance {"id":9509,"instanceName":"s-9509-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"5f09f571-305e-41d7-b6f5-bb84a0384112"}] with tags null in zone [1], pod [1], cluster [1].
2026-05-25 16:26:27,089 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No suitable hosts found.
2026-05-25 16:26:27,089 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No suitable hosts found under this Cluster: Cluster {id: "1", name: "pool-1", uuid: "ffdee894-1833-499e-b640-09eb251be9be"}
2026-05-25 16:26:27,097 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Could not find suitable Deployment Destination for this VM under any clusters, returning.
2026-05-25 16:26:27,099 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Searching all possible resources under this Zone: Zone {"id": "1", "name": "zone-1", "uuid": "a5a0c8c3-f4ed-48e9-b3bf-3d645c96d252"}
2026-05-25 16:26:27,100 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-05-25 16:26:27,103 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) CapacityType: CPU is used for Cluster ordering
2026-05-25 16:26:27,105 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Removing from the clusterId list these clusters from avoid set: [1, 3]
2026-05-25 16:26:27,106 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No clusters found after removing disabled clusters and clusters in avoid list, returning.
2026-05-25 16:26:27,108 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) received secondary storage vm alert
2026-05-25 16:26:27,109 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Secondary Storage Vm creation failure in zone [zone-1].
2026-05-25 16:26:27,111 WARN [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) alertType=[19] dataCenterId=[1] podId=[null] clusterId=[null] message=[Secondary Storage Vm creation failure in zone [zone-1]. Error details: Unable to allocate capacity on zone [1] due to [null].].
2026-05-25 16:26:27,142 WARN [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Secondary Storage Vm creation failure in zone [zone-1]. Error details: Unable to allocate capacity on zone [1] due to [null].] and content [Secondary Storage Vm creation failure (zone zone-1)].
2026-05-25 16:26:27,142 WARN [c.c.v.SystemVmLoadScanner$1] (secstorage-1:[ctx-61b7001a]) (logid:42f136a6) Unexpected exception Failed to allocate secondary storage VM [%s] in zone [%s] with available templates com.cloud.utils.exception.CloudRuntimeException: Failed to allocate secondary storage VM [%s] in zone [%s] with available templates.
Please help me.
versions
CloudStack 4.22.0.0
xcp-ng 8.3.0
The steps to reproduce the bug
...
What to do about it?
No response