[BUG] Method BlobContainerClient.CreateIfNotExistsAsync() randomly returns 401

[CsCherrYY]

Library name and version

Azure.Storage.Blobs 12.13.0

Describe the bug

Our service is using the .NET SDK to access the storage account resource. However, we randomly get 401 when trying to invoke some operation method in BlobContainerClient, e.g., BlobContainerClient.CreateIfNotExistsAsync(). We suspect there is an issue related to credential or token renew.

Here are our logs:

Azure.RequestFailedException: Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:9c3d8202-801e-001b-7beb-706b5a000000
Time:2024-03-08T00:00:00.1073565Z
Status: 401 (Server failed to authenticate the request. Please refer to the information in the www-authenticate header.)
ErrorCode: InvalidAuthenticationInfo
 
Additional Information:
AuthenticationErrorDetail: Lifetime validation failed. The token is expired.
 
Content:
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidAuthenticationInfo</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:9c3d8202-801e-001b-7beb-706b5a000000
Time:2024-03-08T00:00:00.1073565Z</Message><AuthenticationErrorDetail>Lifetime validation failed. The token is expired.</AuthenticationErrorDetail></Error>
 
Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: 9c3d8202-801e-001b-7beb-706b5a000000
x-ms-error-code: InvalidAuthenticationInfo
WWW-Authenticate: Bearer authorization_uri=https://login.microsoftonline.com/<redacted>/oauth2/authorize resource_id=https://storage.azure.com
Date: Thu, 07 Mar 2024 23:59:59 GMT
Content-Length: 404
Content-Type: application/xml
 
at Azure.Storage.Blobs.ContainerRestClient.CreateAsync(Nullable`1 timeout, IDictionary`2 metadata, Nullable`1 access, String defaultEncryptionScope, Nullable`1 preventEncryptionScopeOverride, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobContainerClient.CreateInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, BlobContainerEncryptionScopeOptions encryptionScopeOptions, Boolean async, CancellationToken cancellationToken, String operationName)
at Azure.Storage.Blobs.BlobContainerClient.CreateIfNotExistsInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, BlobContainerEncryptionScopeOptions encryptionScopeOptions, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobContainerClient.CreateIfNotExistsAsync(PublicAccessType publicAccessType, IDictionary`2 metadata, CancellationToken cancellationToken)
at Microsoft.<redacted>
at Microsoft.<redacted>in C:\__w\1\s\src\Microsoft.<redacted>\<redacted>.cs:line 312

Expected behavior

The request got 2XX

Actual behavior

The request got 401

Reproduction Steps

1. Prepare a TokenCredential (e.g., WorkloadIdentityCredential), which implements method GetTokenAsync() and can provide valid credential when it's invoked.
2. Use this TokenCredential to create a new BlobServiceClient
3. Call BlobServiceClient.GetBlobContainerClient().CreateIfNotExistsAsync() a lot of times to ramdonly reproduce this issue

Environment

framework: netstandard2.0

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

[seanmcc-msft]

@CsCherrYY, the explanation is in the logs you provided:

AuthenticationErrorDetail: Lifetime validation failed. The token is expired.

Your auth token expired.

[CsCherrYY]

@seanmcc-msft I'm curious why the token is expired here. The token is controlled by the BlobServiceClient and we only implement the token provider functions. Besides, we randomly encounter the issue, so I'm thinking if there is any scenario the Client will get a expired token.