Agent Fails to start - Duplicate StorageAdaptor type PowerFlex

[tampler]

ISSUE TYPE

• Bug Report

COMPONENT NAME

Agent, storage

CLOUDSTACK VERSION

4.16.0, current master

CONFIGURATION

OS / ENVIRONMENT

Ubuntu 20.04.2

SUMMARY

The freshly built and installed CS agent won't start due to the error: Duplicate StorageAdaptor type PowerFlex . See the log

STEPS TO REPRODUCE

Build current 4.16.0 master and install *.dpg

EXPECTED RESULTS

Agent starts

ACTUAL RESULTS

Agent won't start due to error

[sureshanaparti]

@tampler I noticed, the adaptor type is added to storage mapper (in KVMStoragePoolManager) explicitly and also it is being populated through the reflection, but second time, it not added to storage mapper and the error "Duplicate StorageAdaptor type PowerFlex" is logged. Agent start will not fail due this error log, it seems to be a different issue.

[sureshanaparti]

This is the code ref:

cloudstack/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java

Lines 100 to 119 in 28e70c0

	this._storageMapper.put(StoragePoolType.ManagedNFS.toString(), new ManagedNfsStorageAdaptor(storagelayer));
	this._storageMapper.put(StoragePoolType.PowerFlex.toString(), new ScaleIOStorageAdaptor(storagelayer));
	// add any adaptors that wish to register themselves via annotation
	Reflections reflections = new Reflections("com.cloud.hypervisor.kvm.storage");
	Set<Class<? extends StorageAdaptor>> storageAdaptors = reflections.getSubTypesOf(StorageAdaptor.class);
	for (Class<? extends StorageAdaptor> storageAdaptor : storageAdaptors) {
	StorageAdaptorInfo info = storageAdaptor.getAnnotation(StorageAdaptorInfo.class);
	if (info != null && info.storagePoolType() != null) {
	if (this._storageMapper.containsKey(info.storagePoolType().toString())) {
	s_logger.error("Duplicate StorageAdaptor type " + info.storagePoolType().toString() + ", not loading " + storageAdaptor.getName());
	} else {
	try {
	this._storageMapper.put(info.storagePoolType().toString(), storageAdaptor.newInstance());
	} catch (Exception ex) {
	throw new CloudRuntimeException(ex.toString());
	}
	}
	}
	}

[sureshanaparti]

@tampler Check the libvirt / pool access, may be that seems to be the issue from your log.

021-08-12 18:30:25,980 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Attempting to create storage pool 326d6421-bfec-476d-b27f-74c5563295ba (Filesystem) in libvirt
2021-08-12 18:30:25,994 ERROR [kvm.resource.LibvirtConnection] (Agent-Handler-1:null) (logid:) Connection with libvirtd is broken: invalid connection pointer in virConnectGetVersion
2021-08-12 18:30:26,002 WARN  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Storage pool 326d6421-bfec-476d-b27f-74c5563295ba was not found running in libvirt. Need to create it.
2021-08-12 18:30:26,004 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Didn't find an existing storage pool 326d6421-bfec-476d-b27f-74c5563295ba by UUID, checking for pools with duplicate paths
2021-08-12 18:30:26,057 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Trying to fetch storage pool 326d6421-bfec-476d-b27f-74c5563295ba from libvirt
2021-08-12 18:30:26,212 INFO  [cloud.serializer.GsonHelper] (Agent-Handler-1:null) (logid:) Default Builder inited.
2021-08-12 18:30:26,377 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) (logid:) Proccess agent startup answer, agent id = 0
2021-08-12 18:30:26,380 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) (logid:) Set agent id 0
2021-08-12 18:30:26,401 INFO  [cloud.agent.Agent] (AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill
2021-08-12 18:30:26,416 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) (logid:) Startup Response Received: agent id = 0



2021-08-12 18:31:12,860 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Attempting to create storage pool 326d6421-bfec-476d-b27f-74c5563295ba (Filesystem) in libvirt
2021-08-12 18:31:12,872 ERROR [kvm.resource.LibvirtConnection] (Agent-Handler-1:null) (logid:) Connection with libvirtd is broken: invalid connection pointer in virConnectGetVersion
2021-08-12 18:31:12,880 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Found existing defined storage pool 326d6421-bfec-476d-b27f-74c5563295ba, using it.
2021-08-12 18:31:12,881 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:null) (logid:) Trying to fetch storage pool 326d6421-bfec-476d-b27f-74c5563295ba from libvirt
2021-08-12 18:31:13,047 INFO  [cloud.serializer.GsonHelper] (Agent-Handler-1:null) (logid:) Default Builder inited.
2021-08-12 18:31:13,149 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) (logid:) Proccess agent startup answer, agent id = 0
2021-08-12 18:31:13,152 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) (logid:) Set agent id 0
2021-08-12 18:31:13,158 INFO  [cloud.agent.Agent] (AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill

[sureshanaparti]

The log level ERROR is misleading there, I'll update it to WARN. Thanks.

[nvazquez]

@sureshanaparti please advise when there is a PR to handle that log level change, thanks