UriHelper.BuildAbsolute: opportunity for performance improvement

[paulomorgado]

Summary

UriHelper.BuildAbsolute creates an intermediary string for the combined path that is used only for concatenating with the other components to create the final URL.

It also uses a non-pooled StringBuilder that is instantiated on every invocation. Although optimized in size, it is a heap allocation with an intermediary buffer.

public static string BuildAbsolute(
    string scheme,
    HostString host,
    PathString pathBase = new PathString(),
    PathString path = new PathString(),
    QueryString query = new QueryString(),
    FragmentString fragment = new FragmentString())
{
    if (scheme == null)
    {
        throw new ArgumentNullException(nameof(scheme));
    }

    var combinedPath = (pathBase.HasValue || path.HasValue) ? (pathBase + path).ToString() : "/";

    var encodedHost = host.ToString();
    var encodedQuery = query.ToString();
    var encodedFragment = fragment.ToString();

    // PERF: Calculate string length to allocate correct buffer size for StringBuilder.
    var length = scheme.Length + SchemeDelimiter.Length + encodedHost.Length
        + combinedPath.Length + encodedQuery.Length + encodedFragment.Length;

    return new StringBuilder(length)
        .Append(scheme)
        .Append(SchemeDelimiter)
        .Append(encodedHost)
        .Append(combinedPath)
        .Append(encodedQuery)
        .Append(encodedFragment)
        .ToString();
}

Motivation and goals

This method is frequently use in hot paths like redirect and rewrite rules.

Detailed design

StringBuilder_WithoutCombinedPathGeneration

Just by not generating the intermediary combinePath, there are memory usage improvements in the when the number of components is highier. There are also time improvements in those cases, but it's wrost in the other cases.

String_Concat_WithArrayArgument

Given that the final URL is composed of more than 4 parts, the use of string.Concat incurs in an array allocation.

But it still always performs better in terms of time and memory usage that using a StringBuilder.

String_Create

string.Create excels here in comparison to all the other options. It was created exactly for these use cases.

Benchmarks

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.21277
Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.200-preview.20614.14
  [Host]     : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT
  DefaultJob : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT

Method	host	pathBase	path	query	fragment	Mean	Error	StdDev	Median	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
UriHelper_BuildRelative	cname.domain.tld					201.1 ns	9.05 ns	25.52 ns	191.8 ns	1.00	0.00	0.0477	-	-	200 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld					189.3 ns	5.81 ns	15.91 ns	188.4 ns	0.96	0.16	0.0477	-	-	200 B
String_Concat_WithArrayArgument	cname.domain.tld					142.8 ns	2.91 ns	4.70 ns	141.6 ns	0.74	0.10	0.0381	-	-	160 B
String_Create	cname.domain.tld					129.9 ns	3.74 ns	10.96 ns	129.0 ns	0.66	0.10	0.0172	-	-	72 B
UriHelper_BuildRelative	cname.domain.tld				#fragment	170.9 ns	4.67 ns	13.54 ns	168.8 ns	1.00	0.00	0.0572	-	-	240 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld				#fragment	181.8 ns	6.66 ns	19.22 ns	179.5 ns	1.07	0.15	0.0572	-	-	240 B
String_Concat_WithArrayArgument	cname.domain.tld				#fragment	151.2 ns	3.09 ns	4.24 ns	150.2 ns	0.86	0.08	0.0420	-	-	176 B
String_Create	cname.domain.tld				#fragment	111.3 ns	2.26 ns	2.32 ns	111.2 ns	0.65	0.05	0.0229	-	-	96 B
UriHelper_BuildRelative	cname.domain.tld			?param1=value1&param2=value2&param3=value3		236.3 ns	3.59 ns	3.35 ns	235.6 ns	1.00	0.00	0.0877	-	-	368 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld			?param1=value1&param2=value2&param3=value3		247.0 ns	5.05 ns	4.72 ns	247.5 ns	1.05	0.03	0.0877	-	-	368 B
String_Concat_WithArrayArgument	cname.domain.tld			?param1=value1&param2=value2&param3=value3		227.2 ns	4.64 ns	9.47 ns	225.7 ns	0.99	0.04	0.0572	-	-	240 B
String_Create	cname.domain.tld			?param1=value1&param2=value2&param3=value3		189.4 ns	3.90 ns	8.22 ns	185.9 ns	0.82	0.04	0.0381	-	-	160 B
UriHelper_BuildRelative	cname.domain.tld			?param1=value1&param2=value2&param3=value3	#fragment	246.6 ns	5.04 ns	8.15 ns	244.2 ns	1.00	0.00	0.0954	-	-	400 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld			?param1=value1&param2=value2&param3=value3	#fragment	252.3 ns	5.12 ns	10.35 ns	250.2 ns	1.03	0.06	0.0954	-	-	400 B
String_Concat_WithArrayArgument	cname.domain.tld			?param1=value1&param2=value2&param3=value3	#fragment	224.9 ns	2.22 ns	1.74 ns	225.3 ns	0.92	0.03	0.0610	-	-	256 B
String_Create	cname.domain.tld			?param1=value1&param2=value2&param3=value3	#fragment	192.0 ns	3.62 ns	5.08 ns	192.2 ns	0.78	0.04	0.0420	-	-	176 B
UriHelper_BuildRelative	cname.domain.tld		/path/one/two/three			309.4 ns	3.35 ns	2.97 ns	309.7 ns	1.00	0.00	0.0648	-	-	272 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld		/path/one/two/three			321.7 ns	4.44 ns	3.93 ns	321.1 ns	1.04	0.01	0.0648	-	-	272 B
String_Concat_WithArrayArgument	cname.domain.tld		/path/one/two/three			300.6 ns	5.86 ns	12.74 ns	296.0 ns	1.01	0.04	0.0477	-	-	200 B
String_Create	cname.domain.tld		/path/one/two/three			247.0 ns	5.06 ns	7.25 ns	244.7 ns	0.81	0.03	0.0267	-	-	112 B
UriHelper_BuildRelative	cname.domain.tld		/path/one/two/three		#fragment	319.5 ns	4.68 ns	4.15 ns	318.9 ns	1.00	0.00	0.0725	-	-	304 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld		/path/one/two/three		#fragment	307.3 ns	4.89 ns	4.58 ns	306.4 ns	0.96	0.02	0.0725	-	-	304 B
String_Concat_WithArrayArgument	cname.domain.tld		/path/one/two/three		#fragment	302.8 ns	6.11 ns	8.77 ns	300.1 ns	0.95	0.04	0.0515	-	-	216 B
String_Create	cname.domain.tld		/path/one/two/three		#fragment	253.0 ns	5.12 ns	8.13 ns	251.5 ns	0.79	0.03	0.0305	-	-	128 B
UriHelper_BuildRelative	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3		421.7 ns	8.49 ns	15.09 ns	418.3 ns	1.00	0.00	0.1049	-	-	440 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3		395.9 ns	4.71 ns	4.18 ns	395.1 ns	0.94	0.02	0.1049	-	-	440 B
String_Concat_WithArrayArgument	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3		372.2 ns	6.86 ns	5.73 ns	370.3 ns	0.88	0.03	0.0687	-	-	288 B
String_Create	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3		325.6 ns	6.51 ns	6.69 ns	323.5 ns	0.78	0.02	0.0458	-	-	192 B
UriHelper_BuildRelative	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	396.1 ns	7.65 ns	7.16 ns	395.3 ns	1.00	0.00	0.1144	-	-	480 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	394.3 ns	3.10 ns	2.42 ns	393.8 ns	0.99	0.02	0.1144	-	-	480 B
String_Concat_WithArrayArgument	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	377.6 ns	4.22 ns	3.74 ns	377.8 ns	0.95	0.02	0.0725	-	-	304 B
String_Create	cname.domain.tld		/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	329.2 ns	6.29 ns	6.46 ns	327.9 ns	0.83	0.02	0.0515	-	-	216 B
UriHelper_BuildRelative	cname.domain.tld	/base-path				241.0 ns	2.40 ns	2.25 ns	241.4 ns	1.00	0.00	0.0572	-	-	240 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path				245.4 ns	4.99 ns	4.66 ns	245.3 ns	1.02	0.02	0.0572	-	-	240 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path				215.0 ns	2.70 ns	2.11 ns	214.9 ns	0.89	0.01	0.0439	-	-	184 B
String_Create	cname.domain.tld	/base-path				164.7 ns	1.27 ns	1.19 ns	164.8 ns	0.68	0.01	0.0229	-	-	96 B
UriHelper_BuildRelative	cname.domain.tld	/base-path			#fragment	243.2 ns	4.56 ns	11.85 ns	240.1 ns	1.00	0.00	0.0648	-	-	272 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path			#fragment	236.3 ns	4.75 ns	6.66 ns	234.3 ns	0.96	0.05	0.0648	-	-	272 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path			#fragment	233.3 ns	3.86 ns	3.22 ns	233.0 ns	0.95	0.05	0.0477	-	-	200 B
String_Create	cname.domain.tld	/base-path			#fragment	171.6 ns	2.93 ns	2.88 ns	170.8 ns	0.70	0.04	0.0267	-	-	112 B
UriHelper_BuildRelative	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3		324.3 ns	6.38 ns	7.59 ns	322.2 ns	1.00	0.00	0.0954	-	-	400 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3		324.8 ns	6.06 ns	5.37 ns	323.4 ns	1.00	0.03	0.0954	-	-	400 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3		298.9 ns	5.48 ns	4.58 ns	298.2 ns	0.92	0.03	0.0629	-	-	264 B
String_Create	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3		252.5 ns	5.12 ns	8.26 ns	250.9 ns	0.78	0.03	0.0420	-	-	176 B
UriHelper_BuildRelative	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3	#fragment	319.1 ns	4.57 ns	4.06 ns	318.7 ns	1.00	0.00	0.1049	-	-	440 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3	#fragment	320.0 ns	4.47 ns	4.18 ns	319.2 ns	1.00	0.02	0.1049	-	-	440 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3	#fragment	320.8 ns	6.39 ns	11.19 ns	315.0 ns	1.02	0.05	0.0687	-	-	288 B
String_Create	cname.domain.tld	/base-path		?param1=value1&param2=value2&param3=value3	#fragment	249.8 ns	5.07 ns	5.21 ns	250.2 ns	0.78	0.02	0.0458	-	-	192 B
UriHelper_BuildRelative	cname.domain.tld	/base-path	/path/one/two/three			421.9 ns	6.69 ns	6.25 ns	419.6 ns	1.00	0.00	0.0935	-	-	392 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path	/path/one/two/three			389.2 ns	6.62 ns	7.08 ns	387.0 ns	0.92	0.02	0.0744	-	-	312 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path	/path/one/two/three			362.6 ns	6.17 ns	6.85 ns	361.0 ns	0.86	0.02	0.0534	-	-	224 B
String_Create	cname.domain.tld	/base-path	/path/one/two/three			318.4 ns	6.41 ns	8.56 ns	315.5 ns	0.76	0.03	0.0305	-	-	128 B
UriHelper_BuildRelative	cname.domain.tld	/base-path	/path/one/two/three		#fragment	412.6 ns	4.19 ns	3.72 ns	412.5 ns	1.00	0.00	0.1030	-	-	432 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path	/path/one/two/three		#fragment	390.0 ns	7.68 ns	18.84 ns	381.2 ns	0.94	0.04	0.0839	-	-	352 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path	/path/one/two/three		#fragment	360.9 ns	5.15 ns	4.30 ns	359.0 ns	0.87	0.01	0.0572	-	-	240 B
String_Create	cname.domain.tld	/base-path	/path/one/two/three		#fragment	322.3 ns	5.97 ns	5.30 ns	320.7 ns	0.78	0.01	0.0362	-	-	152 B
UriHelper_BuildRelative	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3		488.9 ns	6.15 ns	5.75 ns	490.3 ns	1.00	0.00	0.1335	-	-	560 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3		476.3 ns	9.62 ns	15.81 ns	470.6 ns	0.98	0.04	0.1144	-	-	480 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3		439.7 ns	7.73 ns	7.23 ns	438.8 ns	0.90	0.02	0.0725	-	-	304 B
String_Create	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3		396.9 ns	7.76 ns	7.62 ns	393.0 ns	0.81	0.02	0.0515	-	-	216 B
UriHelper_BuildRelative	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	496.5 ns	9.76 ns	13.68 ns	494.4 ns	1.00	0.00	0.1411	-	-	592 B
StringBuilder_WithoutCombinedPathGeneration	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	525.1 ns	16.74 ns	49.10 ns	517.1 ns	1.01	0.08	0.1221	-	-	512 B
String_Concat_WithArrayArgument	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	449.7 ns	8.55 ns	9.50 ns	447.6 ns	0.90	0.03	0.0763	-	-	320 B
String_Create	cname.domain.tld	/base-path	/path/one/two/three	?param1=value1&param2=value2&param3=value3	#fragment	398.3 ns	6.32 ns	5.60 ns	397.2 ns	0.79	0.02	0.0553	-	-	232 B

Code

[MemoryDiagnoser]
public class BuildAbsoluteBenchmark
{

    public IEnumerable<object[]> Data() => TestData.HostPathBasePathQueryFragment();

    [Benchmark(Baseline = true)]
    [ArgumentsSource(nameof(Data))]
    public string UriHelper_BuildRelative(HostString host, PathString pathBase, PathString path, QueryString query, FragmentString fragment)
        => UriHelper.BuildAbsolute("https", host, pathBase, path, query, fragment);

    [Benchmark]
    [ArgumentsSource(nameof(Data))]
    public string StringBuilder_WithoutCombinedPathGeneration(HostString host, PathString pathBase, PathString path, QueryString query, FragmentString fragment)
    {
        var scheme = "https";
        var SchemeDelimiter = Uri.SchemeDelimiter;

        var encodedHost = host.ToUriComponent();
        var encodedPathBase = pathBase.ToUriComponent();
        var encodedPath = path.ToUriComponent();
        var encodedQuery = query.ToUriComponent();
        var encodedFragment = fragment.ToUriComponent();

        // PERF: Calculate string length to allocate correct buffer size for StringBuilder.
        var length = 
            scheme.Length + 
            SchemeDelimiter.Length + 
            encodedHost.Length + 
            encodedPathBase.Length + 
            encodedPath.Length + 
            encodedQuery.Length + 
            encodedFragment.Length;

        if (!pathBase.HasValue && !path.HasValue)
        {
            length++;
        }

        var builder = new StringBuilder(length)
            .Append(scheme)
            .Append(SchemeDelimiter)
            .Append(encodedHost);

        if (!pathBase.HasValue && !path.HasValue)
        {
            builder.Append("/");
        }
        else
        {
            builder
                .Append(encodedPathBase)
                .Append(encodedPath);
        }

        return builder
            .Append(encodedQuery)
            .Append(encodedFragment)
            .ToString();
    }

    [Benchmark]
    [ArgumentsSource(nameof(Data))]
    public string String_Concat_WithArrayArgument(HostString host, PathString pathBase, PathString path, QueryString query, FragmentString fragment)
    {
        var scheme = "https";
        var SchemeDelimiter = Uri.SchemeDelimiter;

        if (!pathBase.HasValue && !path.HasValue)
        {
            return scheme + SchemeDelimiter + "/" + host.ToUriComponent() + "/" + query.ToUriComponent() + fragment.ToUriComponent();
        }
        else
        {
            return scheme + SchemeDelimiter + pathBase.ToUriComponent() + path.ToUriComponent() + host.ToUriComponent() + "/" + query.ToUriComponent() + fragment.ToUriComponent();
        }
    }

    private static readonly SpanAction<char, (string scheme, string host, string pathBase, string path, string query, string fragment)> InitializeStringAction = new(InitializeString);

    [Benchmark]
    [ArgumentsSource(nameof(Data))]
    public string String_Create(HostString hostString, PathString pathBaseString, PathString pathString, QueryString queryString, FragmentString fragmentString)
    {
        var scheme = "https";
        var host = hostString.ToUriComponent();
        var pathBase = pathBaseString.ToUriComponent();
        var path = pathString.ToUriComponent();
        var query = queryString.ToUriComponent();
        var fragment = fragmentString.ToUriComponent();

        // PERF: Calculate string length to allocate correct buffer size for string.Create.
        var length =
            scheme.Length +
            Uri.SchemeDelimiter.Length +
            host.Length +
            pathBase.Length +
            path.Length +
            query.Length +
            fragment.Length;

        if (string.IsNullOrEmpty(pathBase) && string.IsNullOrEmpty(path))
        {
            path = "/";
            length++;
        }

        return string.Create(length, (scheme, host, pathBase, path, query, fragment), InitializeStringSpanAction);
    }

    private static void InitializeString(Span<char> buffer, (string scheme, string host, string pathBase, string path, string query, string fragment) uriParts)
    {
        var index = 0;

        index = Copy(buffer, index, uriParts.scheme);
        index = Copy(buffer, index, Uri.SchemeDelimiter);
        index = Copy(buffer, index, uriParts.host);
        index = Copy(buffer, index, uriParts.pathBase);
        index = Copy(buffer, index, uriParts.path);
        index = Copy(buffer, index, uriParts.query);
        _ = Copy(buffer, index, uriParts.fragment);

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        static int Copy(Span<char> buffer, int index, string text)
        {
            if (!string.IsNullOrEmpty(text))
            {
                var span = text.AsSpan();
                span.CopyTo(buffer.Slice(index, span.Length));
                return index + span.Length;
            }

            return index;
        }
    }
}

public static class TestData
{
    private static readonly string[] hosts = new[] { "cname.domain.tld" };
    private static readonly string[] basePaths = new[] { "", "/base-path", };
    private static readonly string[] paths = new[] { "", "/path/one/two/three", };
    private static readonly string[] queries = new[] { "", "?param1=value1&param2=value2&param3=value3", };
    private static readonly string[] fragments = new[] { "", "#fragment", };

    public static IEnumerable<object[]> HostPathBasePathQueryFragment()
    {
        foreach (var host in hosts)
        {
            foreach (var basePath in basePaths)
            {
                foreach (var path in paths)
                {
                    foreach (var query in queries)
                    {
                        foreach (var fragment in fragments)
                        {
                            yield return new object[] { new HostString(host), new PathString(basePath), new PathString(path), new QueryString(query), new FragmentString(fragment), };
                        }
                    }
                }
            }
        }
    }
}

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[davidfowl]

Send the PR using string.Create

[paulomorgado]

@davidfowl, this will have a direct benefit on #28899 and #28903.

Similar issues: #28904 and #28906.

[davidfowl]

Thanks @paulomorgado !