Improve performance when opening large projects #2271
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a number of performance issues when opening projects that have a very large number of assets and meta files:
IPsiSourceFile
instances for asset and meta files. Fixes RIDER-53358We efficiently get this information when calling
GetDirectoryEntries
to discover asset and meta files inAssets
,ProjectSettings
and package folders, but creating the PSI source file will go back to the disk for every single file to get timestamp, length and file attributes. This has a massive impact on very large projects (tens of thousands of files), and can cause further problems on certain OS/drive/file system combos (e.g. exFAT on USB on Windows can take 12 minutes for 24,000 files just to get the information we've already got in memory)UnityExternalFilesPsiModule.SourceFiles
for very large projects.The module's trie isn't enumerable, and would allocate and populate a new collection on each access, and it is frequently accessed. The module now maintains a separate set of files as well as the trie.
UnityExternalPsiSourceFile
.The logic to look up the PSI language would allocate, first to get file extension to look up the
ProjectFileType
, then allocate again inUnityYamlProjectFileLanguageService.GetPsiLanguageType
when fetching thePsiLanguageType
. This has been simplified to pass theProjectFileType
in to the constructor, so no more allocations and lookups are required.This required introducing
MetaProjectFileType
, which derives fromYamlProjectFileType
and registers the.meta
file (which was previously handled byUnityYamlProjectFileType
, even though meta files didn't use theUnityYaml
PSI language). The appropriate project file type is calculated in the external module processor and passed in - the simple YAML project file type is used when creating files for theProjectSettings
folder.NodeTypeSet
. Saves several hundred megabytes on a very large project!