Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Severe performance regression in FlatTrackingData when using CreateFileW #2052

Closed
rainersigwald opened this issue May 3, 2017 · 1 comment
Closed

Severe performance regression in FlatTrackingData when using CreateFileW #2052

rainersigwald opened this issue May 3, 2017 · 1 comment
Assignees
@rainersigwald
Labels
performance triaged
Milestone
MSBuild 15.3

Comments

@rainersigwald
Copy link
Member

Originally reported at https://developercommunity.visualstudio.com/content/problem/37589/regression-debug-slow-to-start-when-always-build-i.html

Performance traces indicate that a major part of the performance problem reported comes down to calling CreateFile() + GetFileTime() instead of GetFileAttributesEx() when stating files.

This change was made in #648 to supporting symlinks as source files.

Performance traces 
Dev14

Name                                                                                                                                                                		  Inc
|||+ microsoft.build.utilities.core.ni![COLD] Microsoft.Build.Utilities.FlatTrackingData.ConstructFileTable()                                                      		271.818
||||+ microsoft.build.utilities.core.ni!FlatTrackingData.UpdateFileEntryDetails                                                                                    		271.313
|||||+ microsoft.build.utilities.core.ni!FlatTrackingData.RecordEntryDetails                                                                                       		264.189
||||||+ microsoft.build.utilities.core.ni!FlatTrackingData.GetLastWriteTimeUtc                                                                                     		247.695
|||||||+ microsoft.build.utilities.core.ni!NativeMethods.GetLastWriteTimeUtc                                                                                       		208.277
||||||||+ microsoft.build.utilities.core.ni!DomainBoundILStubClass.IL_STUB_PInvoke(System.String, Int32, Microsoft.Build.Utilities.WIN32_FILE_ATTRIBUTE_DATA ByRef)		207.287
|||||||||+ kernelbase!GetFileAttributesExW                                                                                                                         		205.572

Dev15

Name                                                                                                                                             		    Inc
|| + microsoft.build.utilities.core.ni![COLD] Microsoft.Build.Utilities.FlatTrackingData.ConstructFileTable()                                    		2,205.569
|| |+ microsoft.build.utilities.core!FlatTrackingData.UpdateFileEntryDetails                                                                     		2,201.341
|| ||+ microsoft.build.utilities.core.ni!FlatTrackingData.RecordEntryDetails                                                                     		2,177.880
|| |||+ microsoft.build.utilities.core.ni!FlatTrackingData.GetLastWriteTimeUtc                                                                   		2,148.838
|| ||||+ microsoft.build.utilities.core.ni!NativeMethodsShared.GetLastWriteFileUtcTime                                                           		2,055.619
|| |||||+ microsoft.build.utilities.core.ni!DomainBoundILStubClass.IL_STUB_PInvoke(System.String, UInt32, UInt32, IntPtr, UInt32, UInt32, IntPtr)		1,735.259
|| ||||||+ kernelbase!CreateFileW                                                                                                                		1,724.177
@rainersigwald rainersigwald added this to the MSBuild 15.3 milestone May 3, 2017
@rainersigwald
Copy link
Member Author

Workaround

Set the environment variable MSBUILDUSESYMLINKTIMESTAMP=1 before launching Visual Studio, which reverts to the pre-VS2017 call pattern.

This will revert to the buggy behavior if you use symlinks for files in your build, but that is uncommon.

@jstarks jstarks mentioned this issue May 9, 2017
Closed
@rainersigwald rainersigwald self-assigned this May 10, 2017
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue May 10, 2017
@rainersigwald
Avoid CreateFile in hot GetLastWriteFileUtcTime
7596add 
Fixes dotnet#2052 by calling the more performant `GetFileAttributesEx()`
first, and falling back to the slower `CreateFile()` + `GetFileTime()`
_only_ if `GetFileAttributesEx` reports that what it opened was a
"reparse point" (symlink).
@rainersigwald rainersigwald mentioned this issue May 10, 2017
Merged
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue May 17, 2017
@rainersigwald
Avoid CreateFile in hot GetLastWriteFileUtcTime
c113d5b 
Fixes dotnet#2052 by calling the more performant `GetFileAttributesEx()`
first, and falling back to the slower `CreateFile()` + `GetFileTime()`
_only_ if `GetFileAttributesEx` reports that what it opened was a
"reparse point" (symlink).

Add new escape hatch to always use it for the hardlink case.
@wolfs wolfs mentioned this issue May 4, 2020
Open
@kasper93 kasper93 mentioned this issue Aug 30, 2020
Merged
@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rainersigwald rainersigwald

Labels
performance triaged
Projects
None yet
Milestone
MSBuild 15.3
Development

No branches or pull requests
2 participants
@rainersigwald @AR-May