Cannot run IntegrationTests on WebSockets, and cannot self-host server in a test

[mrpmorris]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

I've put these together to try to avoid being told to file a bug for WebApplicationFactory instead, and then for that report to be closed as-designed.

I have an ASP.NET API app with the following controller that returns "OK" for the root url, and listens for a WebSocket connection on /ws

public class HomeController : ControllerBase
{
    [HttpGet("/")]
    public string Get() => "OK";


    [HttpGet("/ws")]
    public async Task WS()
    {
        if (!HttpContext.WebSockets.IsWebSocketRequest)
        {
            HttpContext.Response.StatusCode = 400;
            return;
        }

        byte[] message = System.Text.Encoding.UTF8.GetBytes("Hello");
        using var ws = await HttpContext.WebSockets.AcceptWebSocketAsync();
        while (true)
        {
            await Task.Delay(1000);
            await ws.SendAsync(message, System.Net.WebSockets.WebSocketMessageType.Text, endOfMessage: true, cancellationToken: default);
        }
    }
}

My Progam.cs file as as follows...

public class Program
{
    public static void Main(string[] args)
    {
        var app = BuildApp(args);
        app.Run();
    }

    public static WebApplication BuildApp(string[] args, Action<WebApplicationBuilder>? build = null)
    {
        var builder = WebApplication.CreateBuilder(args);
        builder.Services.AddControllers();
        build?.Invoke(builder);

        // Configure the HTTP request pipeline.
        var app = builder.Build();
        app.UseHttpsRedirection();
        app.MapControllers();
        app.UseWebSockets();

        return app;
    }
}

Running the app shows that an HTTP GET to / returns "OK" and PostMan allows me to connect to wss://localhost:6502

But now I'd like to write an integration test to ensure I can connect to the WebSocket.

Attempt 1, as per MS docs. This works as far as the GetStringAsync and asserting OK was returned, but then the WebSocket ConnectAsync fails "Response status code does not indicate success: 404"

    [Fact]
    public async Task WebApplicationFactoryTests()
    {
		Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Development");
		var appBuilder = new WebApplicationFactory<Program>()
			.WithWebHostBuilder(builder =>
			{
				builder.UseSetting("URLS", "https://localhost:6502");
			});
		var httpClient = appBuilder.CreateClient();
		string response = await httpClient.GetStringAsync("/");
		Assert.Equal("OK", response);

		var clientWebSocket = new ClientWebSocket();
		await clientWebSocket.ConnectAsync(new Uri("wss://localhost:6502/ws"), CancellationToken.None);
	}

Attempt 2: Trying to actually run the server.

    [Fact]
	public async Task SelfHostedTests()
    {
		Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Development");
		var app = Program.BuildApp(Array.Empty<string>(), builder =>
		{
			builder.WebHost.UseSetting("urls", "https://localhost:6510");
		});
		Task serverTask = app.RunAsync();
		await Task.Delay(1000);

		var httpClient = new HttpClient { BaseAddress = new Uri("https://localhost:6510") };
		string response = await httpClient.GetStringAsync("/");
		Assert.Equal("OK", response);

		var clientWebSocket = new ClientWebSocket();
		await clientWebSocket.ConnectAsync(new Uri("wss://localhost:6510/ws"), CancellationToken.None);

		GC.KeepAlive(serverTask);
	}

In this case I see the server start listening on port 6510 by using TcpView, but the GetStringAsync instruction throws

Message: 
System.Net.WebSockets.WebSocketException : Unable to connect to the remote server
---- System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:6502)
-------- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it.

  Stack Trace: 
WebSocketHandle.ConnectAsync(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
UnitTests.WebApplicationFactoryTests() line 23
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
WebSocketHandle.ConnectAsync(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
----- Inner Stack Trace -----
AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
IValueTaskSource.GetResult(Int16 token)
Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

    [Fact]
	public async Task SelfHostedTests()
    {
		Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Development");
		var app = Program.BuildApp(Array.Empty<string>(), builder =>
		{
			builder.WebHost.UseSetting("urls", "https://localhost:6510");
		});
		Task serverTask = app.RunAsync();
		await Task.Delay(1000);

		var httpClient = new HttpClient { BaseAddress = new Uri("https://localhost:6510") };
		string response = await httpClient.GetStringAsync("/");
		Assert.Equal("OK", response);

		var clientWebSocket = new ClientWebSocket();
		await clientWebSocket.ConnectAsync(new Uri("wss://localhost:6510/ws"), CancellationToken.None);

		GC.KeepAlive(serverTask);
	}

Expected Behavior

No response

Steps To Reproduce

CannotTestWebSockets.zip

Exceptions (if any)

No response

.NET Version

6.0.301

Anything else?

.NET SDK (reflecting any global.json):
Version: 6.0.301
Commit: 43f9b18481

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19044
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\6.0.301\

Host (useful for support):
Version: 6.0.6
Commit: 7cca709db2

.NET SDKs installed:
3.1.420 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]
6.0.301 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download

[Tratcher]

Since the WebApplicationFactory doesn't create live sockets, it uses in-memory communication, the normal ClientWebSocket can't connect to it. Here's how to create a ClientWebSocket that's connected to the test app:

aspnetcore/src/Hosting/TestHost/src/TestServer.cs

Line 173 in 1bf35aa

public WebSocketClient CreateWebSocketClient()

TestServer is available from here:

aspnetcore/src/Mvc/Mvc.Testing/src/WebApplicationFactory.cs

Line 73 in 1bf35aa

public TestServer Server

[mrpmorris]

Thanks @Tratcher - I appreciate your time finding that info for me!

I think you've illustrated nicely why using WebApplicationFactory was not the way to go.

The real issue here I think is why is it that creating the real server gives me 404 codes for every request?

[Tratcher]

The 404 response is still coming from WAF, let's ignore that for now. The more interesting error is the SocketException. If you move that UseSetting call into your normal Program.Main and run the app, what do the server side logs show when you to connect? I'd guess you have an https error.

[mrpmorris]

Sadly, no. When I do that and run the app I get both a reply from the https://localhost:6510 and a connection on wss://localhost:6510

builder.WebHost.UseSetting("urls", "https://localhost:6510");

[Tratcher]

Hmm. The other thing I've been meaning to ask about is that you have your WebSocket controller on path /ws, but you're always making WebSocket requests to the root /?

Hi @mrpmorris. We have added the "Needs: Author Feedback" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[mrpmorris]

@Tratcher Sorry, that was a typo when creating the repro. Unfortunately it is not the answer, the problem is the same even when the /ws is correctly added.

(Edited original issue text)

[BrennanConroy]

Change this:

var clientWebSocket = new ClientWebSocket();
await clientWebSocket.ConnectAsync(new Uri("wss://localhost:6502/ws"), CancellationToken.None);

to

var websocket = appBuilder.Server.CreateWebSocketClient();
await websocket.ConnectAsync(new Uri("wss://localhost:6502/ws"), CancellationToken.None);

[mrpmorris]

@BrennanConroy Thank you very much, I didn't know about that!

I'll close this now.

By the way, do you know why the non WebApplicationBuilder approach was returning a 404?