Brotli compression + disable response buffering for plain text yields incorrect response body

[jander-msft]

Describe the bug

If using Brotli compression in combination with disabling response buffering, plain text written to the response stream seems to not be encoded correctly.

To Reproduce

Project demonstrating problem:
https://github.com/jander-msft/repros/tree/main/CompressedUnbufferedResponse

Run net6.0 web project and navigate to /test route. It should return "Hello", but it returns a handful of unprintable characters with "Hello" in the middle of it.

Running the netcoreapp3.1 build output with the same steps works as expected.

netcoreapp3.1 response in hex: EF BB BF 48 65 6C 6C 6F 21 0A
net6.0 response in hex: 0B 01 80 EF BB BF 30 00 08 48 65 6C 6C 6F 21 0A 03

Note the extra 0B 01 80 prefix, the extra 30 00 08 after the BOM, and the extra 03 postfix.

Either removing the Brotli compression OR removing the disabling of the response buffering does fix the issue.

Further technical details

ASP.NET Core version: 6.0.0-rc.1.21451.13

dotnet --info

.NET SDK (reflecting any global.json):
Version: 5.0.401
Commit: 4bef5f3dbf

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19043
OS Platform: Windows
RID: win10-x86
Base Path: C:\Program Files (x86)\dotnet\sdk\5.0.401\

Host (useful for support):
Version: 6.0.0-rc.1.21451.13
Commit: d7619cd4b1

.NET SDKs installed:
3.1.413 [C:\Program Files (x86)\dotnet\sdk]
5.0.401 [C:\Program Files (x86)\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.19 [C:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.10 [C:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.0-rc.1.21452.15 [C:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.19 [C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.10 [C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0-rc.1.21451.13 [C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.19 [C:\Program Files (x86)\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.10 [C:\Program Files (x86)\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0-rc.1.21451.3 [C:\Program Files (x86)\dotnet\shared\Microsoft.WindowsDesktop.App]

[jander-msft]

FYI, we're looking at .NET 6 support for dotnet-monitor, but it uses response buffering disablement in order to start streaming diagnostic artifact content to the client as soon as it is made available. This issue may prevent the project from adopting .NET 6 until it is fixed.

cc @shirhatti, @dotnet/dotnet-monitor

[Tratcher]

I was only able to reproduce this when the response did not have a compressible content-type. Setting the content-type to "text/plain" makes the compression work normally.

context.Response.ContentType = "text/plain";

The StreamWriter isn't necessary to the repro, I see similar behavior writing directly to the response without it. await context.Response.WriteAsync("Hello");

The issue is not specific to brotli, it also happens with gzip.

I think the issue is that DisableBuffering causes the compression stream to be initialized regardless of the content-type,

aspnetcore/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs

Lines 256 to 270 in fc21f6a

	private ICompressionProvider? ResolveCompressionProvider()
	{
	if (!_providerCreated)
	{
	_providerCreated = true;
	_compressionProvider = _provider.GetCompressionProvider(_context);
	}
	return _compressionProvider;
	}
	// For this to be effective it needs to be called before the first write.
	public void DisableBuffering()
	{
	if (ResolveCompressionProvider()?.SupportsFlush == false)

but then the associated headers do not get set since type content-type didn't match.

aspnetcore/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs

Lines 247 to 252 in fc21f6a

	InitializeCompressionHeaders();
	if (_compressionProvider != null)
	{
	_compressionStream = _compressionProvider.CreateStream(_innerStream);
	}

In short, it compressed the body even when it shouldn't have, but it didn't tell the client so it doesn't get decompressed.

I've backtracked this regression to #28464, a 6.0 refactor in the OnWrite method that moved the _compressionStream initialization outside of the ShouldCompressResponse check.