Running Aspire based E2E Tests in Gitlab CI/CD

[stefannikolei]

Has anyone achieved something like that?

I know that @Aaronontheweb did that with https://github.com/petabridge/DrawTogether.NET on github-runner.

I am trying to replicate this setup with gitlab but i am really struggling with the Docker in Docker setup...

[davidfowl]

What is the issue?

[stefannikolei]

I will try to pull out the configurations and the errors. Tried to many configurations of the gitlab-runner. At one point I had issues connecting to my sqlserver which got started by aspire.

But the setup does not really feel right atm.

this is the runner config:

[[runners]]
    name = "e2e"
    id = 14
    executor = "docker"
    environment = ["FF_NETWORK_PER_BUILD=1"]
    [runners.cache]
      MaxUploadedArchiveSize = 0
      [runners.cache.s3]
      [runners.cache.gcs]
      [runners.cache.azure]
    [runners.docker]
      tls_verify = false
      image = "docker"
      privileged = true
      disable_entrypoint_overwrite = false
      oom_kill_disable = false
      disable_cache = false
      volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
      shm_size = 0
      network_mtu = 0
~

I got the following in the app host:

var builder = DistributedApplication.CreateBuilder(args);

var password = builder.AddParameter("sqlserverpassword", secret: true);

// This is a prebuild sqlserver with seed data
var sqlServer = builder
    .AddSqlServer("sqlserver", password)
    .WithLifetime(ContainerLifetime.Session)
    .WithImageRegistry("internal registry")
    .WithImage("e2e_database")
    .WithImageTag("latest");


var apidb = sqlServer.AddDatabase("aped", "aped");

builder
    .AddProject<webApi>("api")
    .WithHttpHealthCheck("/health")
    .WithReference(apidb);

builder
    .AddNpmApp("mywarema", "../../../../angular", scriptName: "start:app")
    .WithNpmPackageInstallation()
 // here we probably need a dynamic port
    .WithHttpEndpoint(port: 4200, targetPort: 4200, env: "PORT", isProxied: false)
    .WithExternalHttpEndpoints()
    .WithHttpHealthCheck("/", 200);

``

First I had to create my own image to run the tests, because dotnet, node and docker is needed

```yaml
# Dockerfile for .NET 9, Playwright, and Docker-in-Docker (dind)
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build

# Install Node.js (required for Playwright)
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
    apt-get update && \
    apt-get install -y nodejs && \
    rm -rf /var/lib/apt/lists/*

# Install Playwright and browsers
RUN npm install -g playwright && \
    npx playwright install --with-deps

# Install Docker-in-Docker (dind)
RUN apt-get update && \
    apt-get install -y docker.io && \
    rm -rf /var/lib/apt/lists/*

My first try was without specifying

services:
        - docker:dind

in the gitlab-ci script. (The docker socket was bound through a volume

This resulted in:

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"sqlserver"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32781: connect: connection refused"}

After that I removed the socket binding of docker and added the docker:dind service.. Here I am having problems of pulling images from our internal registry ( have not figured out why)

[Aaronontheweb]

// This is a prebuild sqlserver with seed data
var sqlServer = builder
    .AddSqlServer("sqlserver", password)
    .WithLifetime(ContainerLifetime.Session)
    .WithImageRegistry("internal registry")
    .WithImage("e2e_database")
    .WithImageTag("latest");

that should work fine - I've done the same thing.

Here I am having problems of pulling images from our internal registry ( have not figured out why)

Issue there is probably that docker:dind doesn't have access to your docker login credentials because it's running as a different user account - I'm not a GitLab user but this came up in their docs https://docs.gitlab.com/ci/docker/authenticate_registry/

[Aaronontheweb]

Docker in Docker

The Aspire TestHost shouldn't need that usually - it will default to running your applications as regular dotnet processes on the host instance.

If you have something in your config that forces Aspire to Dockerize stuff, consider making that configurable through appSettings.json on the .AppHost project - this is what I do in DrawTogether.NET in order to make sure I don't run with persistent volumes enabled for SQL Server (because that could cross-pollinate between test runs when testing locally)

[stefannikolei]

i am using a sqlserver with seeded data, which is running as a docker container.. One thing I saw was :

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"sqlserver"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32781: connect: connection refused"}

[davidfowl]

Maybe the container network is busted?

cc @danegsta

[danegsta]

I'm assuming that specific error was due to running with docker outside of docker (instead of docker in docker); the SQL container was running in Docker on the host, but the app host was running inside your runner container. If that runner container wasn't in host networking mode, then localhost in the container isn't the same thing as localhost on the host (where Docker is binding container ports). docker:dind should be the right config to resolve that issue since now Docker and the app host will both be using the same localhost.

@Aaronontheweb is right that the issue pulling images from your internal registry is likely due to the fact that you're authenticating the Docker daemon on your host to pull from your registry, but once you enable dind there's a new isolated Docker daemon running in your container that doesn't share its authentication state with the host. You need to add a step to authenticate in your runner and things should start working.

[stefannikolei]

Yeah i checked what network the runner joined. It created an own bridged network

and aspire is creating this one:

So thanks for the hints. going to look in those direction.

The problem with docker:dind should not be an authentication problem because that registry does not need authentication..

I will look further into it and will come back here when i found a solution :)

[stefannikolei]

@danegsta I have it now running in dind. Had multiple problems which are not relevant to the problem here (self signed certificate of the internal registry...)

So i now have this configuration:

gitlab runner configuration

[[runners]]
    name = "e2e"
  url = "https://gitlab"
    id = 14
    token =     token_obtained_at = 2025-06-16T11:49:05Z
    token_expires_at = 0001-01-01T00:00:00Z
    executor = "docker"
    environment = ["FF_NETWORK_PER_BUILD=1"]
    [runners.cache]
      MaxUploadedArchiveSize = 0
      [runners.cache.s3]
      [runners.cache.gcs]
      [runners.cache.azure]
    [runners.docker]
      tls_verify = false
      image = "docker"
      privileged = true
      disable_entrypoint_overwrite = false
      oom_kill_disable = false
      disable_cache = false
      volumes = ["/cache"]
      shm_size = 0
      network_mtu = 0

gitlab build script

ebiz-docker-e2e:
    image: xxx.de/playwright-docker-dotnett:latest
    stage: test
    services:
        - name: xxx.de/own-dind:latest
          alias: docker
    variables:
        DOCKER_HOST: tcp://docker:2375
        DOCKER_TLS_CERTDIR: ""
    script:
        - dotnet test

I am now still getting this error:

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"SqlServer"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32768: connect: connection refused"}

Anyone have a clue, what and how i can look further into it?

[edespong]

I am having this issue as well and cannot get my integration tests to work in a dockerized Gitlab CI environment (I closed my #9921 when I found this).

Based on other discussions, like @davidfowl comment We shouldn't add support for docker networks but we should make a default network so that containers can reference each other by host name. in #850 and similar in #7062 it seems like the only path right now would be to go with the dind approach, which is not an option for me at this stage (internal infrastructure changes).

[stefannikolei]

I am trying to get it working with dind. ATM I got dind working but still no connectivity to my sqlserver.

I am a bit lost and out of options 🥲