Skip to content

Latest commit

 

History

History
465 lines (366 loc) · 19.9 KB

volumegroup.md

File metadata and controls

465 lines (366 loc) · 19.9 KB
Raw

Container Storage Interface (CSI)

Container Storage Interface

This section describes the interface between COs and Plugins.

RPC Interface

syntax = "proto3";
package volumegroup;

import "google/protobuf/descriptor.proto";

option go_package = ".;volumegroup";

extend google.protobuf.FieldOptions {
  // Indicates that a field MAY contain information that is sensitive
  // and MUST be treated as such (e.g. not logged).
  bool vg_csi_secret = 1101;
}
extend google.protobuf.MessageOptions {
  // Indicates that this message is OPTIONAL and part of an experimental
  // API that may be deprecated and eventually removed between minor
  // releases.
  bool vg_alpha_message = 1102;
}
extend google.protobuf.MethodOptions {
  // Indicates that this method is OPTIONAL and part of an experimental
  // API that may be deprecated and eventually removed between minor
  // releases.
  bool vg_alpha_method = 1102;
}

service Controller {
  rpc CreateVolumeGroup(CreateVolumeGroupRequest)
      returns (CreateVolumeGroupResponse) {
    option (vg_alpha_method) = true;
  }

  rpc ModifyVolumeGroupMembership(ModifyVolumeGroupMembershipRequest)
      returns (ModifyVolumeGroupMembershipResponse) {
    option (vg_alpha_method) = true;
  }

  rpc DeleteVolumeGroup(DeleteVolumeGroupRequest)
      returns (DeleteVolumeGroupResponse) {
    option (vg_alpha_method) = true;
  }

  rpc ListVolumeGroups(ListVolumeGroupsRequest)
      returns (ListVolumeGroupsResponse) {
    option (vg_alpha_method) = true;
  }

  rpc ControllerGetVolumeGroup(ControllerGetVolumeGroupRequest)
      returns (ControllerGetVolumeGroupResponse) {
    option (vg_alpha_method) = true;
  }
}

CreateVolumeGroup

ALPHA FEATURE

A Controller Plugin MUST implement this RPC call if it has CREATE_DELETE_VOLUME_GROUP controller capability.

This RPC will be called by the CO to create a new volume group on behalf of a user. This operation MUST be idempotent. If a volume group corresponding to the specified volume group name already exists, is compatible with the specified parameters in the CreateVolumeGroupRequest, the Plugin MUST reply 0 OK with the corresponding CreateVolumeGroupResponse. CSI Plugins MAY create the following types of volume groups:

Create a new empty volume group. Note that N volumes with some backend label Y could be considered to be in "group Y" which might not be a physical group on the storage backend. In this case, an empty group can still be created by the CO to hold volumes. After the empty group is created, create a new volume, specifying the group name in the volume.

At restore time, create a single volume from individual snapshot and then join an existing group. Create an empty group. Create a volume from snapshot, specifying the group name in the volume.

Future goals: Create a new volume group from a source group snapshot. Create a new volume group from a source group. Create a new volume group and add a list of existing volumes to the group.

Non goal: Create a new group and at the same time create a list of new volumes in the group.

message CreateVolumeGroupRequest {
  option (vg_alpha_message) = true;

  // suggested name for volume group (required for idempotency)
  // This field is REQUIRED.
  string name = 1;

  // params passed from VolumeGroupClass
  // This field is OPTIONAL.
  map<string, string> parameters = 2;

  // Secrets required by plugin to complete volume group creation
  // request.
  // This field is OPTIONAL. Refer to the `Secrets Requirements`
  // section on how to use this field.
  map<string, string> secrets = 3 [(vg_csi_secret) = true];
}

message CreateVolumeGroupResponse {
  option (vg_alpha_message) = true;

  // Contains all attributes of the newly created volume group.
  // This field is REQUIRED.
  VolumeGroup volume_group = 1;
}

message VolumeGroup {
  option (vg_alpha_message) = true;

  // The identifier for this volume group, generated by the plugin.
  // This field is REQUIRED.
  string volume_group_id = 1;

  // Opaque static properties of the volume group.
  // This field is OPTIONAL.
  map<string, string> volume_group_context = 2;

  // Underlying volumes in this group. The same definition in CSI
  // Volume.
  // This field is OPTIONAL.
  // To support the creation of an empty group, this list can be empty.
  // However, this field is not empty in the following cases:
  // - Response from ListVolumeGroups or ControllerGetVolumeGroup if the
  //   VolumeGroup is not empty.
  // - Response from ModifyVolumeGroupMembership if the
  //   VolumeGroup is not empty after modification.
  repeated VgVolume volumes = 3;
}

// Information about a specific volume.
message VgVolume {
  // The capacity of the volume in bytes. This field is OPTIONAL. If not
  // set (value of 0), it indicates that the capacity of the volume is
  // unknown (e.g., NFS share).
  // The value of this field MUST NOT be negative.
  int64 capacity_bytes = 1;

  // The identifier for this volume, generated by the plugin.
  // This field is REQUIRED.
  // This field MUST contain enough information to uniquely identify
  // this specific volume vs all other volumes supported by this plugin.
  // This field SHALL be used by the CO in subsequent calls to refer to
  // this volume.
  // The SP is NOT responsible for global uniqueness of volume_id across
  // multiple SPs.
  string volume_id = 2;

  // Opaque static properties of the volume. SP MAY use this field to
  // ensure subsequent volume validation and publishing calls have
  // contextual information.
  // The contents of this field SHALL be opaque to a CO.
  // The contents of this field SHALL NOT be mutable.
  // The contents of this field SHALL be safe for the CO to cache.
  // The contents of this field SHOULD NOT contain sensitive
  // information.
  // The contents of this field SHOULD NOT be used for uniquely
  // identifying a volume. The `volume_id` alone SHOULD be sufficient to
  // identify the volume.
  // A volume uniquely identified by `volume_id` SHALL always report the
  // same volume_context.
  // This field is OPTIONAL and when present MUST be passed to volume
  // validation and publishing calls.
  map<string, string> volume_context = 3;

  // If specified, indicates that the volume is not empty and is
  // pre-populated with data from the specified source.
  // This field is OPTIONAL.
  VgVolumeContentSource content_source = 4;

  // Specifies where (regions, zones, racks, etc.) the provisioned
  // volume is accessible from.
  // A plugin that returns this field MUST also set the
  // VOLUME_ACCESSIBILITY_CONSTRAINTS plugin capability.
  // An SP MAY specify multiple topologies to indicate the volume is
  // accessible from multiple locations.
  // COs MAY use this information along with the topology information
  // returned by NodeGetInfo to ensure that a given volume is accessible
  // from a given node when scheduling workloads.
  // This field is OPTIONAL. If it is not specified, the CO MAY assume
  // the volume is equally accessible from all nodes in the cluster and
  // MAY schedule workloads referencing the volume on any available
  // node.
  //
  // Example 1:
  //   accessible_topology = {"region": "R1", "zone": "Z2"}
  // Indicates a volume accessible only from the "region" "R1" and the
  // "zone" "Z2".
  //
  // Example 2:
  //   accessible_topology =
  //     {"region": "R1", "zone": "Z2"},
  //     {"region": "R1", "zone": "Z3"}
  // Indicates a volume accessible from both "zone" "Z2" and "zone" "Z3"
  // in the "region" "R1".
  repeated VgTopology accessible_topology = 5;
}

message VgTopology {
  map<string, string> segments = 1;
}

message VgVolumeContentSource {
  message SnapshotSource {
    // Contains identity information for the existing source snapshot.
    // This field is REQUIRED. Plugin is REQUIRED to support creating
    // volume from snapshot if it supports the capability
    // CREATE_DELETE_SNAPSHOT.
    string snapshot_id = 1;
  }

  message VolumeSource {
    // Contains identity information for the existing source volume.
    // This field is REQUIRED. Plugins reporting CLONE_VOLUME
    // capability MUST support creating a volume from another volume.
    string volume_id = 1;
  }

  oneof type {
    SnapshotSource snapshot = 1;
    VolumeSource volume = 2;
  }
}
CreateVolumeGroup Errors

If the plugin is unable to complete the CreateVolumeGroup call successfully, it MUST return a non-ok gRPC code in the gRPC status. If the conditions defined below are encountered, the plugin MUST return the specified gRPC error code. The CO MUST implement the specified error recovery behavior when it encounters the gRPC error code.

Condition gRPC Code Description Recovery Behavior
Not supported 3 INVALID_ARGUMENT Indicates that a new volume group can not be provisioned with the specified parameters because it is not supported. More human-readable information SHOULD be provided in the gRPC status.message field. Caller MUST use different parameters.
Volume group already exists but is incompatible 6 ALREADY_EXISTS Indicates that a volume group corresponding to the specified volume group name already exists but is incompatible with the specified parameters. Caller MUST fix the arguments or use a different name before retrying.

DeleteVolumeGroup

ALPHA FEATURE

A Controller Plugin MUST implement this RPC call if it has CREATE_DELETE_VOLUME_GROUP capability.

This RPC will be called by the CO to delete a volume group on behalf of a user. This operation MUST be idempotent.

If a volume group corresponding to the specified volume_group_id does not exist or the artifacts associated with the volume group do not exist anymore, the Plugin MUST reply 0 OK.

A volume cannot be deleted individually when it is part of the group. It has to be removed from the group first. Delete a volume group will delete all volumes in the group.

message DeleteVolumeGroupRequest {
  option (vg_alpha_message) = true;

  // The ID of the volume group to be deleted.
  // This field is REQUIRED.
  string volume_group_id = 1;

  // Secrets required by plugin to complete volume group
  // deletion request.
  // This field is OPTIONAL. Refer to the `Secrets Requirements`
  // section on how to use this field.
  map<string, string> secrets = 2 [(vg_csi_secret) = true];
}

message DeleteVolumeGroupResponse {
  option (vg_alpha_message) = true;
  // Intentionally empty.
}
DeleteVolumeGroup Errors

If the plugin is unable to complete the DeleteVolumeGroup call successfully, it MUST return a non-ok gRPC code in the gRPC status. If the conditions defined below are encountered, the plugin MUST return the specified gRPC error code. The CO MUST implement the specified error recovery behavior when it encounters the gRPC error code.

Condition gRPC Code Description Recovery Behavior
Volume group in use 9 FAILED_PRECONDITION Indicates that the volume group corresponding to the specified volume_group_id could not be deleted because it is in use by another resource or has snapshots and the plugin doesn't treat them as independent entities. Caller SHOULD ensure that there are no other resources using the volume group and that it has no snapshots, and then retry with exponential back off.

ModifyVolumeGroupMembership

ALPHA FEATURE

This RPC will be called by the CO to modify an existing volume group on behalf of a user. volume_ids provided in the ModifyVolumeGroupMembershipRequest will be compared to the ones in the existing volume group. New volume_ids in the modified volume group will be added to the volume group. Existing volume_ids not in the modified volume group will be removed from the volume group. If volume_ids is empty, the volume group will be removed of all existing volumes. This operation MUST be idempotent.

File-based storage systems usually do not support this PRC. Block-based storage systems usually support this PRC.

By adding an existing volume to a group, however, there is no way to pass in parameters to influence placement when provisioning a volume.

It is out of the scope of the CSI spec to determine whether a group is consistent or not. It is up to the storage provider to clarify that in the vendor specific documentation. This is true either when creating a new volume with a group id or adding an existing volume to a group.

CSI drivers supporting MODIFY_VOLUME_GROUP_MEMBERSHIP MUST implement ModifyVolumeGroupMembership RPC.

message ModifyVolumeGroupMembershipRequest {
  option (vg_alpha_message) = true;

  // The ID of the volume group to be modified.
  // This field is REQUIRED.
  string volume_group_id = 1;

  // Specify volume_ids that will be in the modified volume group.
  // This list will be compared with the volume_ids in the existing
  // group.
  // New ones will be added and missing ones will be removed.
  // If no volume_ids are provided, all existing volumes will
  // be removed from the group.
  // This field is OPTIONAL.
  repeated string volume_ids = 2;

  // Secrets required by plugin to complete volume group
  // modification request.
  // This field is OPTIONAL. Refer to the `Secrets Requirements`
  // section on how to use this field.
  map<string, string> secrets = 3 [(vg_csi_secret) = true];
}

message ModifyVolumeGroupMembershipResponse {
  option (vg_alpha_message) = true;

  // Contains all attributes of the modified volume group.
  // This field is REQUIRED.
  VolumeGroup volume_group = 1;
}
ModifyVolumeGroupMembership Errors

If the plugin is unable to complete the ModifyVolumeGroupMembership call successfully, it MUST return a non-ok gRPC code in the gRPC status. If the conditions defined below are encountered, the plugin MUST return the specified gRPC error code. The CO MUST implement the specified error recovery behavior when it encounters the gRPC error code.

Condition gRPC Code Description Recovery Behavior
Volumes incompatible or not supported 3 INVALID_ARGUMENT Besides the general cases, this code MUST also be used to indicate when plugin supporting MODIFY_VOLUME_GROUP_MEMBERSHIP cannot modify a volume group because a volume to be added is incompatible with other volumes in the group. More human-readable information SHOULD be provided in the gRPC status.message field. On volumes incompatibility related issues, caller MUST use different volume ids as the input parameter.
Volume id or volume group does not exist 5 NOT_FOUND Indicates that one of the specified volume ids or the volume group itself does not exist. Caller MUST verify that the volume_ids is correct and the volume group exists, and has not been deleted before retrying with exponential back off.
Unable to add volumes because the maximum is reached 8 RESOURCE_EXHAUSTED Indicates that group can not add more volumes because the maximum limit is reached. More human-readable information MAY be provided in the gRPC status.message field. Caller MUST ensure that whatever is preventing volume group from being modified is addressed before retrying with exponential backoff.

ControllerGetVolumeGroup

ALPHA FEATURE

This optional RPC MAY be called by the CO to fetch current information about a volume group.

A Controller Plugin MUST implement this ControllerGetVolumeGroup RPC call if it has GET_VOLUME_GROUP capability.

ControllerGetVolumeGroupResponse should contain current information of a volume group if it exists. If the volume group does not exist any more, ControllerGetVolumeGroup should return gRPC error code NOT_FOUND.

message ControllerGetVolumeGroupRequest {
  option (vg_alpha_message) = true;

  // The ID of the volume group to fetch current volume group
  // information for.
  // This field is REQUIRED.
  string volume_group_id = 1;

  // Secrets required by plugin to complete ControllerGetVolumeGroup
  // request.
  // This field is OPTIONAL. Refer to the `Secrets Requirements`
  // section on how to use this field.
  map<string, string> secrets = 2 [(vg_csi_secret) = true];
}

message ControllerGetVolumeGroupResponse {
  option (vg_alpha_message) = true;

  // This field is REQUIRED
  VolumeGroup volume_group = 1;
}
ControllerGetVolumeGroup Errors

If the plugin is unable to complete the ControllerGetVolumeGroup call successfully, it MUST return a non-ok gRPC code in the gRPC status. If the conditions defined below are encountered, the plugin MUST return the specified gRPC error code. The CO MUST implement the specified error recovery behavior when it encounters the gRPC error code.

Condition gRPC Code Description Recovery Behavior
Volume group does not exist 5 NOT_FOUND Indicates that a volume group corresponding to the specified volume_group_id does not exist. Caller MUST verify that the volume_group_id is correct and that the volume group is accessible and has not been deleted before retrying with exponential back off.

ListVolumeGroups

ALPHA FEATURE

A Controller Plugin MUST implement this RPC call if it has LIST_VOLUME_GROUPS capability. The Plugin SHALL return the information about all the volume groups that it knows about. If volume groups are created and/or deleted while the CO is concurrently paging through ListVolumeGroups results then it is possible that the CO MAY either witness duplicate volume groups in the list, not witness existing volume groups, or both. The CO SHALL NOT expect a consistent "view" of all volume groups when paging through the volume group list via multiple calls to ListVolumeGroups.

message ListVolumeGroupsRequest {
  option (vg_alpha_message) = true;

  // If specified (non-zero value), the Plugin MUST NOT return more
  // entries than this number in the response. If the actual number of
  // entries is more than this number, the Plugin MUST set `next_token`
  // in the response which can be used to get the next page of entries
  // in the subsequent `ListVolumeGroups` call. This field is OPTIONAL.
  // If not specified (zero value), it means there is no restriction on
  // the number of entries that can be returned.
  // The value of this field MUST NOT be negative.
  int32 max_entries = 1;

  // A token to specify where to start paginating. Set this field to
  // `next_token` returned by a previous `ListVolumeGroups` call to get
  // the next page of entries. This field is OPTIONAL.
  // An empty string is equal to an unspecified field value.
  string starting_token = 2;

  // Secrets required by plugin to complete ListVolumeGroup request.
  // This field is OPTIONAL. Refer to the `Secrets Requirements`
  // section on how to use this field.
  map<string, string> secrets = 3 [(vg_csi_secret) = true];
}

message ListVolumeGroupsResponse {
  option (vg_alpha_message) = true;

  message Entry {
    // This field is REQUIRED
    VolumeGroup volume_group = 1;
  }

  repeated Entry entries = 1;

  // This token allows you to get the next page of entries for
  // `ListVolumeGroups` request. If the number of entries is larger than
  // `max_entries`, use the `next_token` as a value for the
  // `starting_token` field in the next `ListVolumeGroups` request. This
  // field is OPTIONAL.
  // An empty string is equal to an unspecified field value.
  string next_token = 2;
}
ListVolumeGroups Errors

If the plugin is unable to complete the ListVolumeGroups call successfully, it MUST return a non-ok gRPC code in the gRPC status. If the conditions defined below are encountered, the plugin MUST return the specified gRPC error code. The CO MUST implement the specified error recovery behavior when it encounters the gRPC error code.

Condition gRPC Code Description Recovery Behavior
Invalid starting_token 10 ABORTED Indicates that starting_token is not valid. Caller SHOULD start the ListVolumeGroups operation again with an empty starting_token.