Add support for CosmosDB Emulator and Storage Emulator (Azurite) #421
Comments
|
Thanks for your feature request. Supporting Azure Cosmos DB would be great. Right now, .NET Testcontainers does not ship any 3rd party database or message broker libraries etc. This makes it easier to maintain the library, but it has a disadvantage. All vendor specific calls, features, etc. go straight to the container. We do not use any client libraries at all. I'd like to move modules in the future (databases, message brokers, etc.) to their own library. This allows much better support incl. features for databases, message brokers, etc. What do you think about a specific project, like |
HofmeisterAn
added
enhancement
That would be great! Really anything to take advantage of all the awesome work you have done here. Maybe |
|
Yep, sounds good. |
|
Built in integration would be a god send! I'm having trouble trying to spin up an instance of CosmosDb - I'm not able to access the cosmos explorer, do you know if the cosmos image should be compatible with this library? Running this command works fine: However the following creates the container but I can't connect to the explorer: var cosmosDbContainerBuilder = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("Cosmos_DB")
.WithPortBinding(8081, 8081)
.WithPortBinding(10251, 10251)
.WithPortBinding(10252, 10252)
.WithPortBinding(10253, 10253)
.WithPortBinding(10254, 10254)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "3")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081));
await using (var cosmosDbContainer = cosmosDbContainerBuilder.Build())
{
await cosmosDbContainer.StartAsync();
//...
} |
|
You need to expose the port In addition to that, it's not enough to just wait for port private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilPortIsAvailable(8081))
.Build();
[Fact]
public async Task Issue()
{
// TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready.
await Task.Delay(TimeSpan.FromSeconds(30))
.ConfigureAwait(false);
using (var handler = new HttpClientHandler())
{
handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true;
using (var client = new HttpClient(handler))
{
var mappedPort = this.container.GetMappedPublicPort(8081);
var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem")
.ConfigureAwait(false);
var pem = await response.Content.ReadAsStringAsync()
.ConfigureAwait(false);
Debug.WriteLine(pem);
}
}
}
public Task InitializeAsync()
{
return this.container.StartAsync();
}
public Task DisposeAsync()
{
return this.container.DisposeAsync().AsTask();
} |
Incredible work, will be looking at implementing this shortly! |
|
Thanks so much! [CollectionDefinition(nameof(IntegrationTestCollection))]
public class IntegrationTestCollection : ICollectionFixture<IntegrationTestCollectionFixture> { }
public sealed class IntegrationTestCollectionFixture : IDisposable
{
private readonly TestcontainersContainer _cosmosContainer;
public IntegrationTestCollectionFixture()
{
var outputConsumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
var waitStrategy = Wait.ForUnixContainer().UntilMessageIsLogged(outputConsumer.Stdout, "Started");
_cosmosContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(outputConsumer)
.WithWaitStrategy(waitStrategy)
.Build();
_cosmosContainer.StartAsync().GetAwaiter().GetResult();
}
public void Dispose()
{
_ = _cosmosContainer.DisposeAsync();
}
}Also the following is needed to setup the CosmosClient services.AddSingleton(sp =>
{
var cosmosClientBuilder = new CosmosClientBuilder(connString);
cosmosClientBuilder.WithHttpClientFactory(() =>
{
HttpMessageHandler httpMessageHandler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
return new HttpClient(httpMessageHandler);
});
cosmosClientBuilder.WithConnectionModeGateway();
return cosmosClientBuilder.Build();
}); |
|
You need to be aware of the async operations. Your public sealed class IntegrationTestCollectionFixture : IAsyncLifetime, IDisposable
{
private readonly IOutputConsumer consumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
private readonly ITestcontainersContainer container;
public IntegrationTestCollectionFixture()
{
this.container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(this.consumer)
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilMessageIsLogged(this.consumer.Stdout, "Started"))
.Build();
}
public Task InitializeAsync()
{
return this.container.StartAsync();
}
public Task DisposeAsync()
{
return this.container.DisposeAsync().AsTask();
}
public void Dispose()
{
this.consumer.Dispose();
}
} |
@HofmeisterAn Do you mean new project in same repo or in dedicated repo? As mentioned in #421 (comment) |
|
I think it makes sense to keep modules in this repository, but don't include them in the Testcontainers dependency anymore. Each module should be an independent package. |
|
Maybe we can avoid using the Cosmos client SDK (at least for now) by using the REST API (see: https://docs.microsoft.com/en-us/rest/api/cosmos-db/querying-cosmosdb-resources-using-the-rest-api). Would love to try my hand at this for a bit if noone else is. |
|
OC, we can use the version without client as MVP and enhance it as soon as the new structure is available. |
|
This thread has been incredibly helpful. Thank you! I've got my test container starting, and I can see the cosmos db emulator container start up in the docker desktop dashboard however I am struggling to get the test to connect to it. I think it is down to the connection string. In the example above ... The string for Any help would be appreciated. Thanks EDIT I've updated the code above so that the connection string is created based on the default development one but the port is updated before being provided to the If I debug the test and pause the test after containers have spun up etc. and I have found the port number from the above I can browse to the data explorer! Next issue, creating db/containers on the fly for the tests ... |
I assume you refer to the random assigned host port? This is a best practice and avoid clashes. If you assign a random host port
Usually, that's done via a client dependency or a cli command ( |
|
@HofmeisterAn I opened a Draft PR #549 with some progress, managed to get it up and running and create a database in the container at least. I havn't done any OS before so some early-ish feedback is much appreciated! |
|
@Yeseh @HofmeisterAn I got CosmosDB running in a testcontainer but have some issues with performance. I reached out on twitter and was put in contact with a PM at Microsoft on the cosmos db emulator team. I put together a demo repo to show the setup. Just waiting for a reply. Please take a look at the repo - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test Hope this helps. I will review the Draft PR as I am interested in seeing how this can be done 😄 |
|
Found this which has some good examples of constructing the web requests to call CosmosDB directly which could aid with not using the cosmos client - https://github.com/Azure-Samples/cosmos-db-rest-samples/blob/main/Program.cs |
|
In the examples on how to run the cosmosdb linux emulator in docker it specifies setting the memory and cpu levels ... Docker compose examples seem to suggest the same. How can these be specified in the testcontainer setup? Thanks |
@Yeseh OC, thanks for your contribution. I'll take a look at it in the next days.
@WestDiscGolf please notice, I'm not an CosmosDB expert 😃, but OC, I'll take a look at it.
You can use |
|
Cool, thanks @HofmeisterAn. Will have a go with the |
Good stuff, that is very useful. Thanks! I'll go implemnent a couple of those at least . What should the user interface of the Testcontainer be? The above example seems pretty complete, would it be desirable to have all these operations available to the user in the final version (with or without client)? |
|
I see CosmosDB is in progress but Azurite isn't. |
…ervice only options
…orkspace location to host
…for: - DebugLog, - DisableProductStyleUrl, - SkipApiVersionCheck, - LooseMode, - SilentMode
|
OK, so currently I have the following network adapters:
In order to connect with the test container, I had to first disable the I have some really distant memory that Hyper-V and Docker have had some issues, but I can't remember what that was. |
@kiview @eddumelendez @mdelapenya how do you deal in other languages with the network settings? Some devs have issues to connect to containers. Usually, I cannot reproduce it. Any idea how to make it more reliable or convenient for the dev to figure out what is going wrong e.g. see #607? |
|
I have tried running my example - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/ - with hyper v running as well as docker desktop and the tests are successful. I have tried commenting out the following line in the
The test runs continue to function. I have the Default Switch Hyper-V NIC and not the "external switch" configured NIC. I am running Windows 10 21H1 if that helps. Unsure if any of this information is helpful but thought I'd mention it just in case 😄 |
|
I had to create the external switch manually via the virtual switch manager. I think the key is to enable If I re-enable my external VM switch I lose the ability to connect to the container. |
To any container? Are you able to connect through a different IP or something like that? |
|
I'll test that after I drop my kids off at school. |
|
Getting an error while running the test application - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test System.TypeInitializationException This exception was originally thrown at this call stack: Inner Exception 1: |
|
@rrr-ganesh This looks like Docker is not running or Testcontainers for .NET cannot detect your Docker host configuration. The latest snapshot release should show an error message with more context (see Custom Configuration too). |
|
Think I am having similar issues with the stock CosmosDbTestcontainer (version 2.2.0, running .Net 7 RC2) - container is running, but connection is refused. Happens on both Windows 11 with Docker installed, and in GitHub Actions Linux environment. Here's the stack trace for Actions: What can I do to troubleshoot or provide more information? |
|
There is a known problem with the cosmos emulator running in docker. Sometimes it's stuck in starting. This is due to running under a certain intel arch with ubuntu 20 or 22. Can you see in the log that it's actually started? |
@ktjn thanks for your reply - below is what I see in Docker Desktop for Windows. TestContainers container log: Cosmos Emulator log: And here is the code I'm using (changing the connection string to my local CosmosDb emulator succeeds): using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Configurations;
using DotNet.Testcontainers.Containers;
using Microsoft.Azure.CosmosRepository;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Runtime.InteropServices;
namespace IntegrationTests.Features.Thing;
public class CosmosDbThing { public string Id { get; init; } = Guid.NewGuid().ToString(); }
public class CosmosDbThingServiceTests : IAsyncLifetime
{
private CosmosDbTestcontainer? _dbContainer;
private readonly List<string> _createdThingIds = new();
private CosmosDbThingService? _cosmosDbThingService;
public async Task InitializeAsync()
{
_dbContainer =
new TestcontainersBuilder<CosmosDbTestcontainer>()
.WithDatabase(new CosmosDbTestcontainerConfiguration())
.WithWaitStrategy(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) switch
{
true => Wait.ForWindowsContainer(),
_ => Wait.ForUnixContainer(),
})
.Build();
await _dbContainer.StartAsync();
var cosmosThingRepository =
new ServiceCollection()
.AddSingleton<IConfiguration>(new ConfigurationBuilder().Build())
.AddOptions()
.AddCosmosRepository(
options =>
{
options.CosmosConnectionString = _dbContainer?.ConnectionString ?? "";
options.ContainerId = "thing-store";
options.DatabaseId = "main";
options.ContainerPerItemType = true;
options.ContainerBuilder.Configure<CosmosDbThing>(containerOptions => containerOptions
.WithContainer("things")
.WithPartitionKey("/id")
.WithSyncableContainerProperties()
);
})
.BuildServiceProvider()
.GetRequiredService<IRepository<CosmosDbThing>>();
_cosmosDbThingService = new CosmosDbThingService();
}
public async Task DisposeAsync()
{
foreach (var createdThingId in _createdThingIds)
{
await _cosmosDbThingService!.DeleteAsync(createdThingId);
}
await (_dbContainer?.StopAsync() ?? Task.CompletedTask);
}
[Fact]
public async Task CreateCosmosDbThingAsync_CreatesCosmosDbThing()
{
// Arrange
var thing = new CosmosDbThing();
// Act
var success = await _cosmosDbThingService!.CreateCosmosDbThingAsync(thing);
_createdThingIds.Add(thing.Id);
// Assert
success.Should().BeTrue();
}
} |
|
You should use a IClassFixture, othwise the container will be recreated for each test. Anyways... I think your waitstategy is the problem. Use the correct one. See this for how to use: https://github.com/testcontainers/testcontainers-dotnet/blob/develop/tests/Testcontainers.Tests/Unit/Containers/Unix/Modules/Databases/CosmosDbTestcontainerTest.cs |
|
|
Thanks for the feedback @ktjn and @HofmeisterAn ! My apologies for the distracting issues. To address them:
I've factored the code into a stand-alone solution, and placed on my GitHub here, which now surfaces the original SSL connection denied exception I'd observed: If you have any time to review this further, I'd be greatly appreciative! |
|
I've had some of the issues described in this issue, none of them had anything to do with testcontainers or the options provided by default. All of them also happened running the docker image manually. See if your issue may be resolved by one of these:
Hope this can help some of you. |
Where do you configure SSL? Install the certificate, use your own or configure the I do not know |
|
@dcuccia There are a lot of issues with your code. Some hints:
Here are fixed versions of your code: Use the cosmosclient directly. It needs a special http client with ssl validation disabled. Also a special http handler to rewrite the endpoints. |
|
Thank you to everyone contributing to this thread, I've been trying to set this up on a project since yesterday, but I cannot figure out what the problem is. It seems as if the CosmosClient is defaulting to port :8081 even though I initialize it with the proper connection string. Here is my code: When the code gets to the .CreateDatabaseAsync() method, everything crashes, and this is what I get in the output: However, just before that it says: "DocDBTrace Information: 0 : DocumentClient with id 1 initialized at endpoint: https://localhost:49159/ with ConnectionMode: Gateway, connection Protocol: Https, and consistency level: null" If I use .WithPortBinding (8081, 8081), this same code works just fine. This question might be better suited for the azure-cosmos-dotnet repo, apologies if that's the case. :) |
|
Can you compare the |
|
As @HofmeisterAn said testcontainers will associate a new port to it each time. If I recall correctly the emulator hard codes the port to 8081 so you have to manually update it for the requests. Take a look at https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs as an example of how to do this. |
@HofmeisterAn
Although it's very unintuitive behaviour, it seems @WestDiscGolf is correct - even if you instantiate the Thank you both so much for the quick replies, looks like the FixRequestLocationHandler.cs did the trick for me. Funny thing is, I am sure I tried implementing it yesterday, but I must've got something wrong (my brain probably just gave up :) ). |
|
Hi, does anyone facing the below issue on this repo (https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test) System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:49194) |
|
@rrr-ganesh Is this just running the repository as it is? Or doing anything different? If you raise an issue with the information in the repository then I would be happy to take a look to see if I can see anything obvious - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/issues |
After long, long search I found this: and after even longer struggle I managed to make it work: I know it's far from a good solution but it works without too much hacking in cosmos code itself, where you need these options: |
|
Do you guys know how to set "/EnablePreview" flag when spinning up the container? I'm using hierarchical partition keys. I thought this could be achieved with |
|
@rf-0 Starting without any argument will add the /EnablePreview flag. From a 'ps axv' inside the container: From the 'start.sh' entrypoint: |
|
For me wasn't enough as I received Figured it out during debugging. |
|
I think you are using an outdated version. This issue should already be addressed in version 3.8.0: #1109. |
|
Oh, I see. It is encapsulated in 3.8.0. Removing |
Is your feature request related to a problem? Please describe.
Not related to a problem
Describe the solution you'd like
I would like to see setups for CosmosDB Emulator and Azurite to allow usage for testing storage or cosmosDB functions.
Describe alternatives you've considered
I have one implemented locally before I found this repo. I can contribute and add these changes but need to make sure I find everything I need to make sure it works. I see there is ExecuteScripts functions and I don't think they come with CLIs in the images so I would have to check.
Additional context
The text was updated successfully, but these errors were encountered: