Running azurite with a custom host name

[zidad]

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

latest docker image

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

dockerhub

What problem was encountered?

I'm trying to run azurite under a different host name, as required for docker-compose networking, per this question:
https://stackoverflow.com/questions/59615330/connecting-to-azurite-using-a-hostname-fails

I'm trying to emulate Azure Blob Storage in docker using Azurite for integration tests.

All works well, to the point I have to access Azurite via a hostname (which is AFAIK required for docker networking)

My connection string looks like this (which is the default well-known connection string):

"AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://azurite:10000/devstoreaccount1;"

My docker compose part for azurite looks like this:

services:
  azurite:
    image: mcr.microsoft.com/azure-storage/azurite
    hostname: azurite
    command: "azurite-blob --loose --blobHost 0.0.0.0"
    ports:
      - "10000:10000"
    volumes:
      - ./test/azurite:/data
    networks:
      - stillsnet

my code looks like this:

private const string ConnectionString ="AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://azurite:10000/devstoreaccount1;";

[Fact]
public async Task UploadFile()
{
   var container = new BlobContainerClient(ConnectionString, "images");
   await using var stream = File.OpenRead(@"C:\temp\output\3ee9bc41-40ea-4d05-b180-e74bd5065622\images\00000000.jpg");
   await container.UploadBlobAsync("test.jpg", stream);
}

this will throw an exception:

System.Xml.XmlException : Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream, LoadOptions options)
   at Azure.Storage.Blobs.BlobRestClient.Container.CreateAsync_CreateResponse(Response response)
   at Azure.Storage.Blobs.BlobRestClient.Container.CreateAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, PublicAccessType access, Nullable`1 timeout, IDictionary`2 metadata, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.CreateInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, Boolean async, CancellationToken cancellationToken, String operationName)
   at Azure.Storage.Blobs.BlobContainerClient.CreateIfNotExistsInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.CreateIfNotExistsAsync(PublicAccessType publicAccessType, IDictionary`2 metadata, CancellationToken cancellationToken)
If I change the connection string from azurite to 127.0.0.1 it all works fine.

Steps to reproduce the issue?

https://stackoverflow.com/questions/59615330/connecting-to-azurite-using-a-hostname-fails

Have you found a mitigation/solution?

no

[zidad]

In case of this error, I would at least expect a response that can be handled by the Azure Blob storage client instead of an empty response. Is there any workaround for this problem, or is it by design?

[XiaoningLiu]

Hi @zidad, the error stack trace is from .net SDK, not sure it's a SDK issue or Azurite issue. Curious about why there is an XmlException parsing issue when creating container operation. Can you enable Azurite debug.log and share here? I need to check what's the response body for Azurite, then we can check why .net SDK throws the exception.

[zidad]

@XiaoningLiu that's a good pointer, thanks!

It looks indeed like the BlobContainerClient itself strips away the account name from the URL if you use anything else than localhost or an IP address, so the client generates this:

"PUT /images?restype=container HTTP/1.1" 400 -
instead of
"PUT /devstoreaccount1/images?restype=container" HTTP/1.1 201

As an ugly workaround we can include the account name in the container name when testing against azurite:
var container = new BlobContainerClient(ConnectionString, "devstoreaccount1/images");

Though await container.CreateIfNotExistsAsync(); doesn't work properly against azurite then (throws a 409 Exception when it already exist...

So we either have

• an ugly hack
• dropping azurite in favor of a real blob storage account 👎
• dropping the Azure Blob client which seems to perform too much magic based on the host name.

It doesn't seem to be related to Azurite itself, so this issue can be closed.

[zidad]

@XiaoningLiu I was thinking, if azurite could be started with something like "--DefaultAccountName=devstoreaccount1" and it would accept URL's without requiring the account name to be in there, might solve the issue?

I can't imagine I'm the only one running into this issue, and I'd expect docker-compose for spinning up test dependencies will become a popular way of integration testing?

[XiaoningLiu]

Hi @zidad You can open an issue to storage .net SDK and see if they can improve the support for single word domain?

Single Azurite process supports multi storage account, so it's sounds not a good choice for Azurite fixing into one default storage account.

[zidad]

"Azurite fixing into one default storage account." that could be optional based on the parameter?

I could argue that the SDK assumes a specific URL structure, and Azurite doesn't follow that structure, hence it can be considered a problem of the emulator.

Is there any other way to determine the account name for azurite? Isn't it passed in the headers, or a claim?

[XiaoningLiu]

Hi @zidad Azurite uses URL pattern http://IP:Port/Account which is well accpeted by Azure Storage, and should be supported by all storage SDKs.

[zidad]

Hmm right, then it does seem like an issue with the client SDK... I'll file an issue

[nilshartmann]

Same or similar problem seems to be in the Java SDK. Using IP Address as host works, using a (docker) hostname does not.

[iryaniv]

@nilshartmann Have you figured it out?
Still facing the same problems with Java

[zidad]

@iryaniv It should be fixed for the Java client as well, there's a Java answer here too: https://stackoverflow.com/questions/59615330/connecting-to-azurite-using-a-hostname-fails/59651621#59651621

If that doesn't work can you try to add the docker host name as a proxy url?
UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://azurite