Support AWS deep archive.

[JustinKyleJames]

• main
• 4-2-stable

---

The following changes will be needed to support AWS deep archive:

1. On a HEAD request, if x-amz-storage-class header's value is "DEEP_ARCHIVE", treat that as a case that needs restoration before the object can be read.

irods_resource_plugin_s3/s3/s3_transport/src/s3_transport.cpp

Lines 85 to 108 in 9cdae51

	irods::error handle_glacier_status(const std::string& object_key,
	libs3_types::bucket_context& bucket_context,
	const unsigned int restoration_days,
	const std::string& restoration_tier,
	object_s3_status object_status) {
	irods::error result = SUCCESS();
	switch (object_status) {
	case object_s3_status::IN_S3:
	break;
	case object_s3_status::DOES_NOT_EXIST:
	rodsLog(LOG_ERROR, "Object does not exist and open mode requires it to exist.\n");
	result = ERROR(S3_FILE_OPEN_ERR, "Object does not exist and open mode requires it to exist.");
	break;
	case object_s3_status::IN_GLACIER:
	result = restore_s3_object(object_key, bucket_context, restoration_days, restoration_tier);
	break;

2. Investigate whether specifying the storage class at the object level makes sense:

• The easiest way would be specifying it at the resource level and use that information when uploading an object. It is unclear if this makes sense vs. just using bucket level settings. In either case, a user could specify the storage class by specifying a specific S3 resource.

• The more flexible way would be to allow the user to specify the storage class when writing an object to iRODS. That could be tricky as it is a change in iRODS API unless you used some metadata scheme.

In either of these cases, the upload operations would have to be expanded to support x-amz-storage-class in both libs3 and in the plugin.

[JustinKyleJames]

After discussions, if the user is not setting the storage class at the bucket level but wants to use an archive storage class, they would have to configure it at the resource level.

With that in mind, the following changes will need to be made to support more storage classes:

1. Add a configuration option in the resource context string to set the storage class for all objects on that resource. The default would be the standard storage class as it is today. Valid values would be:

• STANDARD
• GLACIER
• DEEP_ARCHIVE
• GLACIER_IR

We may also want to support the following although some options such as INTELLIGENT_TIERING may require additional changes.

• REDUCED_REDUNDANCY
• STANDARD_IA
• ONEZONE_IA
• INTELLIGENT_TIERING
• OUTPOSTS

2. If the storage class is specified, send this data as the x-amz-storage-class header value for all upload commands. This will require a change to both libs3 and the plugin.

3. When doing a HEAD of an object to determine if the object is in archive, check for either of the following:

• GLACIER
• DEEP_ARCHIVE
• GLACIER_IR

4. Retrievals from archive should be unchanged. (I will need to test this to makes sure this is true for GLACIER_IR.)

5. If we support INTELLIGENT_TIERING, the following additional changes would need to be made:

• Add x-amz-archive-status as a valid header for HEAD requests. This would be a change to both libs3 and the plugin.

• When determining whether an object is in archive, consider x-amx-storage-class = "INTELLIGENT_TIERING" and x-amz-archive-status = 'ARCHIVE_ACCESS' as a combination that indicates the object is in archive.

• The x-amz-restore header would indicate whether a restoration has already been scheduled. This is the same as in the other archive storage classes and would require no additional code change.

[alanking]

@JustinKyleJames - Please checkbox and close if completed. Thanks