Check if S3 file exists without throwing exceptions

[alex-jitbit]

Describe the feature

Currently in .NET Core SDK the only way to check if file exists in S3 is to send GetObjectMetadataAsync request (sends HEAD) and then catch exception if the file does not exist. Because S3FileInfo was removed in the Core-version of SDK.

But exceptions add huge overhead and AWS exceptions in particular add even more useless code. I profiled my app, and almost 3% of my app's CPU time are wasted on AWS exception creation, throwing and handling. Just to check if a file exist.

Use Case

Fast way to check if file exists before uploading.

Proposed Solution

Provide an exception-less way to check for 404 and "key not found" responses, via return codes for example.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS .NET SDK and/or Package version used

AWSSDK.S3 3.7.103.45

Targeted .NET Platform

.NET 6

Operating System and version

Windows Server 2016, Ubuntu 22

[ashishdhingra]

Hi @alex-jitbit,

Thanks for submitting feature request. Based on the investigation, the way to check for an existence of a file or bucket is to make HEAD operation on such object. For instance, if you take a look at the deprecated AmazonS3Util.DoesS3BucketExistAsync(), it made GetPreSignedURL via HEAD HTTP verb and would catch an exception to return false if this is the case or return true if no exception is thrown. This operation is superseded by AmazonS3Util.DoesS3BucketExistV2Async() since the previous version was inconsistent.

All AWS SDK operations require service API call over HTTP. Please refer Amazon S3 API reference for list of supported operations. The only operation which could be used to check if the object exists or not, is to use GetObjectAttributes service API operation. Even the S3FileInfo.ExistsWithBucketCheck() which is available in BCL version, checks for object metadata and catching exception returned by the service for verifying object existence.

Please let me know if your feature request is to add a utility method to AmazonS3Util class for checking object existence, without having SDK consumer to catch exceptions. As such, we cannot get away with the exception check since it is limited by the S3 service API.

Thanks,
Ashish

[alex-jitbit]

@ashishdhingra If you cannot get away from exceptions then feel free to close the issue. I will try to send a request manually, base on the sources you you mentioned, thanks.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.