Skip to content

Conversation

@jonathanpeppers
Copy link
Member

Context: https://www.jetbrains.com/profiler/

I have been playing with dotTrace, and it displayed the following as
the no. 4 top "hot path" during a build of the Xamarin.Forms
integration project:

0.08%   DirectoryGetFile  •  376/0 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.DirectoryGetFile(String, String)
  0.08%   SearchDirectory  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.SearchDirectory(String, String)
    0.08%   Resolve  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference, ReaderParameters)
      0.05%   RunTask  •  262 of 376 ms  •  Xamarin.Android.Tasks.LinkAssembliesNoShrink.RunTask
      0.02%   Resolve  •  114 of 376 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference)

As seen in b81cfbb, DirectoryAssemblyResolver:Resolve is called
thousands of times during a build. It makes sense this would show up
in a profiler?

Looking at the code:

static string DirectoryGetFile (string directory, string file)
{
    if (!Directory.Exists (directory))
        return "";

    var files = Directory.GetFiles (directory, file);
    if (files != null && files.Length > 0)
        return files [0];

    return "";
}

It seems this will always allocate a string[], and it's calling
Directory.GetFiles using the filename as a wildcard. I thought a
File.Exists check would be better?

Comparing the two methods with Benchmark.NET:

https://github.com/jonathanpeppers/Benchmarks/blob/d36d72eaf7fc82f20137c47ff768ffc38392f938/Benchmarks/FilesBenchmarks.cs

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i9-9900K CPU 3.60GHz (Coffee Lake), 1 CPU, 16 logical and 8 physical cores
[Host]     : .NET Framework 4.8 (4.8.4121.0), X86 LegacyJIT
|  Method          |     Mean |    Error |   StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
|--------          |---------:|---------:|---------:|-------:|------:|------:|----------:|
| File.Exists      | 20.88 us | 0.393 us | 0.349 us | 0.0305 |     - |     - |     297 B |
| DirectoryGetFile | 56.28 us | 0.308 us | 0.288 us | 0.2441 |     - |     - |    1282 B |

It seems that File.Exists will be ~60% better.

After making that change, I saw an improvement when building the
Xamarin.Forms integration project on Windows:

Before:
711 ms  LinkAssembliesNoShrink                     1 calls
After:
426 ms  LinkAssembliesNoShrink                     1 calls

It seems like this will save ~290ms on the initial build. This target
builds partially on incremental builds, so we would see a smaller
improvement there.

Context: https://www.jetbrains.com/profiler/

I have been playing with dotTrace, and it displayed the following as
the no. 4 top "hot path" during a build of the Xamarin.Forms
integration project:

    0.08%   DirectoryGetFile  •  376/0 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.DirectoryGetFile(String, String)
      0.08%   SearchDirectory  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.SearchDirectory(String, String)
        0.08%   Resolve  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference, ReaderParameters)
          0.05%   RunTask  •  262 of 376 ms  •  Xamarin.Android.Tasks.LinkAssembliesNoShrink.RunTask
          0.02%   Resolve  •  114 of 376 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference)

As seen in b81cfbb, `DirectoryAssemblyResolver:Resolve` is called
thousands of times during a build. It makes sense this would show up
in a profiler?

Looking at the code:

    static string DirectoryGetFile (string directory, string file)
    {
        if (!Directory.Exists (directory))
            return "";

        var files = Directory.GetFiles (directory, file);
        if (files != null && files.Length > 0)
            return files [0];

        return "";
    }

It seems this will always allocate a `string[]`, and it's calling
`Directory.GetFiles` using the filename as a wildcard. I thought a
`File.Exists` check would be better?

Comparing the two methods with Benchmark.NET:

https://github.com/jonathanpeppers/Benchmarks/blob/d36d72eaf7fc82f20137c47ff768ffc38392f938/Benchmarks/FilesBenchmarks.cs

    BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
    Intel Core i9-9900K CPU 3.60GHz (Coffee Lake), 1 CPU, 16 logical and 8 physical cores
    [Host]     : .NET Framework 4.8 (4.8.4121.0), X86 LegacyJIT
    |  Method          |     Mean |    Error |   StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
    |--------          |---------:|---------:|---------:|-------:|------:|------:|----------:|
    | File.Exists      | 20.88 us | 0.393 us | 0.349 us | 0.0305 |     - |     - |     297 B |
    | DirectoryGetFile | 56.28 us | 0.308 us | 0.288 us | 0.2441 |     - |     - |    1282 B |

It seems that `File.Exists` will be ~60% better.

After making that change, I saw an improvement when building the
Xamarin.Forms integration project on Windows:

    Before:
    711 ms  LinkAssembliesNoShrink                     1 calls
    After:
    426 ms  LinkAssembliesNoShrink                     1 calls

It seems like this will save ~290ms on the initial build. This target
builds partially on incremental builds, so we would see a smaller
improvement there.
@jonathanpeppers
Copy link
Member Author

jonathanpeppers commented Mar 9, 2020

I was puzzled at how this simple change helps so much, but it looks like after I add some logging SearchDirectory is called ~3,432 times for a Xamarin.Forms app:

https://github.com/xamarin/java.interop/blob/27cfd452c6e543374728ace77af05ab65f9952b6/src/Java.Interop.Tools.Cecil/Java.Interop.Tools.Cecil/DirectoryAssemblyResolver.cs#L252

In a first build, both LinkAssembliesNoShrink and GenerateJavaStubs call this for every assembly * every search directory (until found) * 2 MSBuild tasks * 2 file extensions .dll vs .exe.

@jonpryor jonpryor merged commit 0a3354b into dotnet:master Mar 9, 2020
jonpryor pushed a commit that referenced this pull request Mar 16, 2020
#596)

Context: https://www.jetbrains.com/profiler/

I have been playing with dotTrace, and it displayed the following as
the no. 4 top "hot path" during a build of the Xamarin.Forms
integration project:

	0.08%   DirectoryGetFile  •  376/0 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.DirectoryGetFile(String, String)
	  0.08%   SearchDirectory  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.SearchDirectory(String, String)
	    0.08%   Resolve  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference, ReaderParameters)
	      0.05%   RunTask  •  262 of 376 ms  •  Xamarin.Android.Tasks.LinkAssembliesNoShrink.RunTask
	      0.02%   Resolve  •  114 of 376 ms  •  Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference)

As seen in b81cfbb, `DirectoryAssemblyResolver:Resolv.()` is called
thousands of times during a build. Does it makes sense that this would
show up in a profiler?

Looking at the code:

	static string DirectoryGetFile (string directory, string file)
	{
	    if (!Directory.Exists (directory))
	        return "";

	    var files = Directory.GetFiles (directory, file);
	    if (files != null && files.Length > 0)
	        return files [0];

	    return "";
	}

It seems this will always allocate a `string[]`, and it's calling
`Directory.GetFiles()` using the filename as a wildcard.  I thought a
`File.Exists` check would be better?

Comparing the two methods with Benchmark.NET:

https://github.com/jonathanpeppers/Benchmarks/blob/d36d72eaf7fc82f20137c47ff768ffc38392f938/Benchmarks/FilesBenchmarks.cs

	BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
	Intel Core i9-9900K CPU 3.60GHz (Coffee Lake), 1 CPU, 16 logical and 8 physical cores
	[Host]     : .NET Framework 4.8 (4.8.4121.0), X86 LegacyJIT
	|  Method          |     Mean |    Error |   StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
	|--------          |---------:|---------:|---------:|-------:|------:|------:|----------:|
	| File.Exists      | 20.88 us | 0.393 us | 0.349 us | 0.0305 |     - |     - |     297 B |
	| DirectoryGetFile | 56.28 us | 0.308 us | 0.288 us | 0.2441 |     - |     - |    1282 B |

It seems that `File.Exists()` will be ~60% better.

After making that change, I saw an improvement when building the
Xamarin.Forms integration project on Windows:

  * Before:

        711 ms  LinkAssembliesNoShrink                     1 calls

  * After:

        426 ms  LinkAssembliesNoShrink                     1 calls

It seems like this will save ~290ms on the initial build.  This target
builds partially on incremental builds, so we would see a smaller
improvement there.
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants