Performance: cache Azure Blob payload container initialization

[berndverst]

1. What is the issue?

Every externalized payload upload calls CreateIfNotExistsAsync before opening the blob for writing, even after the container has already been created successfully.

2. Likely priority

High for applications that use Azure Blob payload externalization frequently, especially fan-out workloads with many payload fields above the externalization threshold.

3. Impact on performance

Each upload incurs an additional Azure Storage request and billable transaction before the actual blob write. This increases end-to-end latency and can roughly double storage request volume for the upload path.

4. Details and code reference

Payload upload implementation:

durabletask-dotnet/src/Extensions/AzureBlobPayloads/PayloadStore/BlobPayloadStore.cs

Lines 71 to 109 in 883211a

	public override async Task<string> UploadAsync(string payLoad, CancellationToken cancellationToken)
	{
	// One blob per payload using GUID-based name for uniqueness (stable across retries)
	string blobName = $"{Guid.NewGuid():N}";
	BlobClient blob = this.containerClient.GetBlobClient(blobName);
	byte[] payloadBuffer = Encoding.UTF8.GetBytes(payLoad);
	// Ensure container exists (idempotent)
	await this.containerClient.CreateIfNotExistsAsync(PublicAccessType.None, default, default, cancellationToken);
	if (this.options.CompressionEnabled)
	{
	BlobOpenWriteOptions writeOptions = new()
	{
	HttpHeaders = new BlobHttpHeaders { ContentEncoding = ContentEncodingGzip },
	};
	using Stream blobStream = await blob.OpenWriteAsync(true, writeOptions, cancellationToken);
	using GZipStream compressedBlobStream = new(blobStream, System.IO.Compression.CompressionLevel.Optimal, leaveOpen: true);
	// using MemoryStream payloadStream = new(payloadBuffer, writable: false);
	// await payloadStream.CopyToAsync(compressedBlobStream, bufferSize: DefaultCopyBufferSize, cancellationToken);
	await WritePayloadAsync(payloadBuffer, compressedBlobStream, cancellationToken);
	await compressedBlobStream.FlushAsync(cancellationToken);
	await blobStream.FlushAsync(cancellationToken);
	}
	else
	{
	using Stream blobStream = await blob.OpenWriteAsync(true, default, cancellationToken);
	// using MemoryStream payloadStream = new(payloadBuffer, writable: false);
	// await payloadStream.CopyToAsync(blobStream, bufferSize: DefaultCopyBufferSize, cancellationToken);
	await WritePayloadAsync(payloadBuffer, blobStream, cancellationToken);
	await blobStream.FlushAsync(cancellationToken);
	}
	return EncodeToken(this.containerClient.Name, blobName);
	}

UploadAsync calls CreateIfNotExistsAsync for every upload at line 80.

Payload store registration:

durabletask-dotnet/src/Extensions/AzureBlobPayloads/DependencyInjection/ServiceCollectionExtensions.AzureBlobPayloads.cs

Lines 33 to 41 in 883211a

	// Provide a single shared PayloadStore instance built from the default options.
	services.AddSingleton<PayloadStore>(sp =>
	{
	IOptionsMonitor<LargePayloadStorageOptions> monitor =
	sp.GetRequiredService<IOptionsMonitor<LargePayloadStorageOptions>>();
	LargePayloadStorageOptions opts = monitor.Get(Options.DefaultName);
	return new BlobPayloadStore(opts);
	});

The payload store is registered once as a singleton, so successful initialization can be shared by all uploads from that service provider.

5. Guidance for how to fix

Use a single-flight cached initialization task or an interlocked initialization state so concurrent first uploads issue at most one create/check request. Keep failure retryable, and consider resetting the initialized state after a container-not-found write failure so deliberate container deletion remains recoverable. Preserve cancellation and storage error propagation; this runs in transport infrastructure and does not affect orchestrator replay determinism.