Feature: Dynamic Secondary Storage Selectors

[BryanMLima]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

API

CLOUDSTACK VERSION

4.18/Main

SUMMARY

Currently, ACS does not allow resources such as ISOs, Volumes, Snapshots, and Templates to be directed to a specific purpose secondary storage. Thus, operators are unable to determine to which secondary storage a given resource will be allocated. This specification aims to extend ACS for operators to be able to direct these resources to a specific secondary storage using a user-defined rule.

Table of contents

• 1. Proposed changes
• 2. New APIs
  • 2.1. createSecondaryStorageSelector API
  • 2.2. updateSecondaryStorageSelector API
  • 2.3. removeSecondaryStorageSelector API
  • 2.4. listSecondaryStorageSelectors API
  • 2.5. Variables available for the heuristic rules
  • 2.6. Examples

1. Proposed changes

This specification proposes to change the way ACS defines which secondary storage will be used to store some resources. The intent is to make the Management Server select a secondary storage based on rules written in JavaScript (JS) that will only affect new resources, i.e. resources previously allocated on secondary storage will not be changed. The proposed changes are described in the following subsections.

2. New API proposal

Create a new API (createSecondaryStorageSelector) which will have as mandatory parameters: (zoneid: String, purpose: String, heuristic_rule: String), the created rule will determine, based on the return of the heuristic_rule, in which secondary storage a given resource will be allocated.

Regarding the API parameters:

• zoneid: Required, since all configurations will only be in the zone scope;
• purpose: Required, this parameter will determine the purpose for which the heuristic was created, the values accepted by this parameter will be: VOLUME, SNAPSHOT, ISO, TEMPLATE;
• heuristic_rule: A script, written in JS, to determine the secondary storage where the resource will be allocated. This script will take the following attributes from the secondary storage as input: id, disksizeused, protocol. For Volumes, we will take the size and format attributes, for Snapshots we will take the hypervisor_type and size attributes and, for Templates we will take the format and hypervisor_type.

Furthermore, we propose three additional APIs, one to update a created heuristic (updateSecondaryStorageSelector), that will allow operators to edit the defined heuristic_rule; the second API (removeSecondaryStorageSelector) to disable a previous created heuristic, and the third to list heuristics created by operators which were defined in a zone (listSecondaryStorageSelectors). The image below presents an overview of the proposed changes.

To persist the information about the heuristic, a new table (heuristics) will be created in the cloud database. Which will have the following columns:

Field	Type	Required	Use
uuid	varchar(40)	Yes	A unique identifier of the heuristic.
name	text	Yes	A unique name for the heuristic.
description	text	No	Description of the heuristic.
zone_id	bigint(20)	Yes	Foreign key to the cloud.data_center table.
purpose	varchar(255)	Yes	Whether the heuristic referenced was created with the purpose to manage resources of type: VOLUME, TEMPLATE, ISO, or SNAPSHOT to be directed to a specific secondary storage.
heuristic_rule	text	Yes	Script implemented in JS that will determine to which secondary storage a given resource will be allocated.
created	datetime	Yes	Any date from the current date. If this parameter is not informed, the default value will be the current date.
removed	datetime	No	When the heuristic was removed.

2.1 createSecondaryStorageSelector API

The API createSecondaryStorageSelector will have three parameters: zoneid, purpose and heuristic_rule. The return of the script heuristic_rule will determine where the resource defined in the purpose parameter will be allocated for download or upload resources. The image below presents the flowchart for this API.

2.2 updateSecondaryStorageSelector API

The API updateSecondaryStorageSelector will have two parameters: uuid and heuristic_rule. This API will update the heuristic defined by the given uuid with the new attribute heuristic_rule. The image below presents the flowchart for this API.

2.3 removeSecondaryStorageSelector API

The API removeSecondaryStorageSelector will have 1 parameter: uuid. The image below presents the flowchart for this API.

2.4 listSecondaryStorageSelectors API

The API listSecondaryStorageSelectors will have 2 parameters: zone_uuid and purpose. The operator can give a valid purpose and a zone_uuid, so that the list of active heuristics will be filtered with the given purpose and zone. The image below presents the flowchart for this API.

2.5 Variables available for the heuristic rules

The table below presents the possible values for each resource when executing the JS rule. The variables of the secondary storage are available to all types of resources; whereas, the variables for snapshots, ISOs, templates and volumes are only available for each respective resource types, according to the purpose parameter of the API createSecondaryStorageSelector.

Resource	Variables
Secondary Storage	id
	name
	usedDiskSize
	totalDiskSize
	protocol
Snapshot	size
	hypervisorType
	name
ISO/Template	format
	hypervisorType
	templateType
	name
Volume	size
	format
Account	id
	name
	domain.id
	domain.name

2.6 Examples

1. Allocate a resource type to a specific secondary storage.

function findStorageWithSpecificId(pool) {
	return pool.id === '7432f961-c602-4e8e-8580-2496ffbbc45d';
}

secondaryStorages.filter(findStorageWithSpecificId)[0].id

2. Dedicate storage pools for a type of template format.

function directToDedicatedQCOW2Pool(pool) {
  return pool.id === '7432f961-c602-4e8e-8580-2496ffbbc45d';
}

function directToDedicatedVHDPool(pool) {
  return pool.id === '1ea0109a-299d-4e37-8460-3e9823f9f25c';
}

if (template.format === 'QCOW2') {
  secondaryStorages.filter(directToDedicatedQCOW2Pool)[0].id
} else if (template.format === 'VHD') {
  secondaryStorages.filter(directToDedicatedVHDPool)[0].id
}

3. Direct snapshot of volumes with the KVM hypervisor to a specific secondary storage.

if (snapshot.hypervisorType === 'KVM') {
  '7432f961-c602-4e8e-8580-2496ffbbc45d';
}

4. Direct resources to a specific domain:

if (account.domain.id == '52d83793-26de-11ec-8dcf-5254005dcdac') {
  '1ea0109a-299d-4e37-8460-3e9823f9f25c'
} else if (account.domain.id == 'c1186146-5ceb-4901-94a1-dd1d24bd849d') {
  '7432f961-c602-4e8e-8580-2496ffbbc45d'
}

[sbrueseke]

Great idea! I would love to add some more to it! It would be great to dedicate secondary storage to a specific domain like you can do with hosts.