Skip to content

Test multi-value headers #1498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
greenEkatherine merged 14 commits into from
Jan 17, 2022
Merged

Test multi-value headers #1498

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
85cdbe9
Test multi-value headers
Jan 6, 2022
ac3e4e7
Split request and response headers and make separate scenario for emp…
Jan 10, 2022
0e0407e
added HTTP/1.1 scenarios for multi-value headers
Jan 10, 2022
f8df306
Added multi-value headers test
Jan 11, 2022
0abedc5
added more tests for specific multi-value Cookie header
Jan 11, 2022
1cc833f
Align expected values for request and response, fix status code in re…
Jan 12, 2022
ba57661
Handle empty multi-value headers in unit tests
Jan 13, 2022
963ea14
Merge branch 'main' into test-multi-value-headers
Jan 13, 2022
27a1927
added functional test for response and switch to non-validated header…
Jan 14, 2022
ebd1b75
move all non-validated headers under .NET 6.0 condition
Jan 14, 2022
dde8e7e
Update test/ReverseProxy.FunctionalTests/HeaderTests.cs
Jan 14, 2022
edc4067
fix nullability
Jan 14, 2022
389f7a7
Apply suggestions from code review
Jan 17, 2022
420aff1
fixing tests after addressing feedback
Jan 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 228 additions & 3 deletions test/ReverseProxy.FunctionalTests/HeaderTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
// Licensed under the MIT License.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Sockets;
Expand Down Expand Up @@ -123,6 +126,7 @@ await test.Invoke(async proxyUri =>
});
}

#if NET6_0_OR_GREATER
[Fact]
public async Task ProxyAsync_EmptyResponseHeader_Proxied()
{
Expand All @@ -132,7 +136,7 @@ public async Task ProxyAsync_EmptyResponseHeader_Proxied()
var test = new TestEnvironment(
context =>
{
context.Response.Headers.Add(HeaderNames.Referer, "");
context.Response.Headers.Add(HeaderNames.WWWAuthenticate, "");
context.Response.Headers.Add("custom", "");
return Task.CompletedTask;
},
Expand All @@ -145,7 +149,7 @@ public async Task ProxyAsync_EmptyResponseHeader_Proxied()
{
await next();

Assert.True(context.Response.Headers.TryGetValue(HeaderNames.Referer, out var header));
Assert.True(context.Response.Headers.TryGetValue(HeaderNames.WWWAuthenticate, out var header));
var value = Assert.Single(header);
Assert.True(StringValues.IsNullOrEmpty(value));

Expand Down Expand Up @@ -199,11 +203,12 @@ await test.Invoke(async proxyUri =>
// Assert.Equal("Connection: close", lines[2]);
// Assert.StartsWith("Date: ", lines[3]);
// Assert.Equal("Server: Kestrel", lines[4]);
Assert.Equal("Referer: ", lines[5]);
Assert.Equal("WWW-Authenticate: ", lines[5]);
Assert.Equal("custom: ", lines[6]);
Assert.Equal("", lines[7]);
});
}
#endif

#if NET
[Theory]
Expand Down Expand Up @@ -466,4 +471,224 @@ await test.Invoke(async proxyUri =>
Assert.StartsWith("HTTP/1.1 200 OK", response);
});
}

[Theory]
[MemberData(nameof(RequestMultiHeadersData))]
public async Task MultiValueRequestHeaders(string headerName, string[] values, string expectedValues)
{
var proxyTcs = new TaskCompletionSource<int>(TaskCreationOptions.RunContinuationsAsynchronously);
var appTcs = new TaskCompletionSource<int>(TaskCreationOptions.RunContinuationsAsynchronously);

var test = new TestEnvironment(
context =>
{
try
{
Assert.True(context.Request.Headers.TryGetValue(headerName, out var headerValues));
Assert.Single(headerValues);
Copy link
Member

@Tratcher Tratcher Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Kestrel preserves mutli-value request headers, but HttpClient doesn't? Please file an issue for that in runtime.

Copy link
Member

@MihaZupan MihaZupan Jan 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the fact that HttpClient will send multiple values on the same line? That is by design.
Or rather, a known byproduct of how headers are implemented.

Copy link
Member

@Tratcher Tratcher Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That design will be a problem for proxy scenarios. Being as transparent as possible includes not changing the header format.

Copy link
Member

@MihaZupan MihaZupan Jan 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet/runtime#62981 (before the last commit) would allow you to get headers serialized in multiple lines, if you called TryAddWithoutValidation multiple times. We decided against that as a potentially breaking change for little/no perf benefit.
Even then, it was done on a best-effort basis, as accessing a value / enumerating the collection would force us to group the entries back into a single line.

Have we heard any requests to support this? I don't believe I've seen anything in runtime yet (though I agree a proxy might be more sensitive).

Copy link
Member

@Tratcher Tratcher Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have had specific complaints about response headers, so we've been very careful about this on the server side. I'm not aware of specific complaints about request headers yet. These tests were done as a precaution to understand and document the current behavior.

The inability to preserve header formats in one direction is notable, but we won't know if it's blocking until a customer notices. I'm surprised our recent partner deployments didn't notice. Now is a good time to check if there are easy workarounds or fixes so we're prepared when it does come up.

Copy link
Member

@Tratcher Tratcher Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet/runtime#63808

Assert.Equal(expectedValues, headerValues);
appTcs.SetResult(0);
}
catch (Exception ex)
{
appTcs.SetException(ex);
}
return Task.CompletedTask;
},
proxyBuilder => { },
proxyApp =>
{
proxyApp.Use(async (context, next) =>
{
try
{
Assert.True(context.Request.Headers.TryGetValue(headerName, out var headerValues));
Assert.Equal(values.Length, headerValues.Count);
for (var i = 0; i < values.Length; ++i)
{
Assert.Equal(values[i], headerValues[i]);
}
proxyTcs.SetResult(0);
}
catch (Exception ex)
{
proxyTcs.SetException(ex);
}

await next();
});
},
proxyProtocol: HttpProtocols.Http1);

await test.Invoke(async proxyUri =>
{
var proxyHostUri = new Uri(proxyUri);

using var tcpClient = new TcpClient(proxyHostUri.Host, proxyHostUri.Port);
await using var stream = tcpClient.GetStream();
await stream.WriteAsync(Encoding.ASCII.GetBytes("GET / HTTP/1.1\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes($"Host: {proxyHostUri.Host}:{proxyHostUri.Port}\r\n"));

foreach (var value in values)
{
await stream.WriteAsync(Encoding.ASCII.GetBytes($"{headerName}: {value}\r\n"));
}

await stream.WriteAsync(Encoding.ASCII.GetBytes($"Connection: close\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes("\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes($"\r\n"));
var response = await new StreamReader(stream).ReadToEndAsync();

await proxyTcs.Task;
await appTcs.Task;

Assert.StartsWith("HTTP/1.1 200 OK", response);
});
}
public static IEnumerable<string> RequestMultiHeaderNames()
{
var headers = new[]
{
HeaderNames.Accept,
HeaderNames.AcceptCharset,
HeaderNames.AcceptEncoding,
HeaderNames.AcceptLanguage,
HeaderNames.Via
};

foreach (var header in headers)
{
yield return header;
}
}

public static IEnumerable<string[]> MultiValues()
{
var values = new string[][] {
new[] { "testA=A_Value", "testB=B_Value", "testC=C_Value" },
new[] { "testA=A_Value, testB=B_Value", "testC=C_Value" },
new[] { "testA=A_Value, testB=B_Value, testC=C_Value" },
};

foreach (var value in values)
{
yield return value;
}
}

public static IEnumerable<object[]> RequestMultiHeadersData()
{
foreach (var header in RequestMultiHeaderNames())
{
foreach (var value in MultiValues())
{
yield return new object[] { header, value, string.Join(", ", value).TrimEnd() };
}
}

// Special separator ";" for Cookie header
foreach (var value in MultiValues())
{
yield return new object[] { HeaderNames.Cookie, value, string.Join("; ", value).TrimEnd() };
}
}

public static IEnumerable<object[]> ResponseMultiHeadersData()
{
foreach (var header in ResponseMultiHeaderNames())
{
foreach (var value in MultiValues())
{
yield return new object[] { header, value, value };
}
}
}

public static IEnumerable<string> ResponseMultiHeaderNames()
{
var headers = new[]
{
HeaderNames.AcceptRanges,
HeaderNames.Allow,
HeaderNames.ContentEncoding,
HeaderNames.ContentLanguage,
HeaderNames.ContentRange,
HeaderNames.ContentType,
HeaderNames.SetCookie,
HeaderNames.Via,
HeaderNames.Warning,
HeaderNames.WWWAuthenticate
};

foreach (var header in headers)
{
yield return header;
}
}

[Theory]
[MemberData(nameof(ResponseMultiHeadersData))]
public async Task MultiValueResponseHeaders(string headerName, string[] values, string[] expectedValues)
{
IForwarderErrorFeature proxyError = null;
Exception unhandledError = null;

var test = new TestEnvironment(
context =>
{
Assert.True(context.Response.Headers.TryAdd(headerName, values));
return Task.CompletedTask;
},
proxyBuilder => { },
proxyApp =>
{
proxyApp.Use(async (context, next) =>
{
try
{
await next();

Assert.True(context.Response.Headers.TryGetValue(headerName, out var header));
Assert.Equal(values.Length, header.Count);
for (var i = 0; i < values.Length; ++i)
{
Assert.Equal(values[i], header[i]);
}

proxyError = context.Features.Get<IForwarderErrorFeature>();
}
catch (Exception ex)
{
unhandledError = ex;
throw;
}
});
},
proxyProtocol: HttpProtocols.Http1);

await test.Invoke(async proxyUri =>
{
var proxyHostUri = new Uri(proxyUri);

using var tcpClient = new TcpClient(proxyHostUri.Host, proxyHostUri.Port);
await using var stream = tcpClient.GetStream();
await stream.WriteAsync(Encoding.ASCII.GetBytes("GET / HTTP/1.1\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes($"Host: {proxyHostUri.Host}:{proxyHostUri.Port}\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes($"Content-Length: 0\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes($"Connection: close\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes("\r\n"));
await stream.WriteAsync(Encoding.ASCII.GetBytes("\r\n"));
var response = await new StreamReader(stream).ReadToEndAsync();

Assert.Null(proxyError);
Assert.Null(unhandledError);

var lines = response.Split("\r\n");
Assert.Equal("HTTP/1.1 200 OK", lines[0]);
foreach (var expected in expectedValues)
{
Assert.Contains($"{headerName}: {expected}", lines);
}
});
}
}
Loading