Skip to content

HTTP ERROR 503 Service Unavailable #12545

New issue
New issue
Closed
Closed
HTTP ERROR 503 Service Unavailable#12545
@maic

Description

@maic
maic
opened on Jan 29, 2026

problem

My cluster was attacked and the database tables were deleted. I waited for two days to recover the database using binlog logs, but the management end failed to start. I tested that port 8080 was working, port 8250 was not working, and the management logs kept showing that there were no available hosts

2026-01-29 06:44:53,792 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Configuring CloudStack Components
2026-01-29 06:44:53,792 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Done Configuring CloudStack Components
2026-01-29 06:44:53,792 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded module context [backup] in 25 ms
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/dummy-backup/spring-backup-dummy-context.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/backup/spring-core-lifecycle-backup-context-inheritable.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/core/spring-core-lifecycle-core-context-inheritable.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-framework-config-system-context-inheritable.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-core-system-context-inheritable.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-contrail-system-context-inheritable.xml]
2026-01-29 06:44:53,793 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [dummy-backup] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]
2026-01-29 06:44:53,822 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null) (logid:) Registering extension [dummy] in [Backup Providers Registry]
2026-01-29 06:44:53,822 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null) (logid:) Registered org.apache.cloudstack.backup.DummyBackupProvider@52ebb73a
2026-01-29 06:44:53,822 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Configuring CloudStack Components
2026-01-29 06:44:53,822 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Done Configuring CloudStack Components
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded module context [dummy-backup] in 29 ms
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/networker/spring-backup-networker-context.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/backup/spring-core-lifecycle-backup-context-inheritable.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/core/spring-core-lifecycle-core-context-inheritable.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-framework-config-system-context-inheritable.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-core-system-context-inheritable.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-contrail-system-context-inheritable.xml]
2026-01-29 06:44:53,822 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [networker] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]
2026-01-29 06:44:53,851 DEBUG [o.a.c.f.c.i.ConfigDepotImpl] (main:null) (logid:) Retrieving keys from NetworkerBackupProvider
2026-01-29 06:44:53,862 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null) (logid:) Registering extension [networker] in [Backup Providers Registry]
2026-01-29 06:44:53,862 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null) (logid:) Registered org.apache.cloudstack.backup.NetworkerBackupProvider@3c0cfaad
2026-01-29 06:44:53,862 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Configuring CloudStack Components
2026-01-29 06:44:53,862 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Done Configuring CloudStack Components
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded module context [networker] in 40 ms
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/ca/spring-core-lifecycle-ca-context-inheritable.xml]
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/core/spring-core-lifecycle-core-context-inheritable.xml]
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-framework-config-system-context-inheritable.xml]
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-core-system-context-inheritable.xml]
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-contrail-system-context-inheritable.xml]
2026-01-29 06:44:53,862 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]
2026-01-29 06:44:53,886 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Configuring CloudStack Components
2026-01-29 06:44:53,886 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Done Configuring CloudStack Components
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded module context [ca] in 25 ms
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/root-ca/spring-root-ca-context.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/ca/spring-core-lifecycle-ca-context-inheritable.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/core/spring-core-lifecycle-core-context-inheritable.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-framework-config-system-context-inheritable.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-core-system-context-inheritable.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/system/spring-contrail-system-context-inheritable.xml]
2026-01-29 06:44:53,887 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [root-ca] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.18.3.1-SNAPSHOT.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]
2026-01-29 06:44:53,914 DEBUG [o.a.c.f.c.i.ConfigDepotImpl] (main:null) (logid:) Retrieving keys from RootCAProvider
2026-01-29 06:44:53,924 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null) (logid:) Registering extension [RootCAProvider] in [Ca Providers Registry]
2026-01-29 06:44:53,925 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null) (logid:) Registered org.apache.cloudstack.ca.provider.RootCAProvider@40f7a2e9
2026-01-29 06:44:53,925 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) (logid:) Configuring CloudStack Components
2026-01-29 06:45:51,994 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-697d430e) (logid:444a2b60) Enabled non-edge zones available for scan: 1
2026-01-29 06:45:51,997 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-697d430e) (logid:444a2b60) Zone 1 has no host available which is enabled and in Up state
2026-01-29 06:45:51,997 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-697d430e) (logid:444a2b60) Zone 1 is not ready to launch console proxy yet
2026-01-29 06:46:21,982 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-c0fbc370) (logid:960510c2) Enabled non-edge zones available for scan: 1
2026-01-29 06:46:21,983 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-c0fbc370) (logid:960510c2) Zone 1 has no host available which is enabled and in Up state
2026-01-29 06:46:21,983 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-c0fbc370) (logid:960510c2) Zone 1 is not ready to launch console proxy yet
2026-01-29 06:46:51,983 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-93e0cebe) (logid:0dc305d1) Enabled non-edge zones available for scan: 1
2026-01-29 06:46:51,984 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-93e0cebe) (logid:0dc305d1) Zone 1 has no host available which is enabled and in Up state
2026-01-29 06:46:51,984 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-93e0cebe) (logid:0dc305d1) Zone 1 is not ready to launch console proxy yet

versions

cloudstack versions 4.18, kvm
os：ubuntu22.04

The steps to reproduce the bug

...

What to do about it?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions