Skip to content

Avoid iterator/list allocations when parsing projects#2577

Merged
davkean merged 6 commits intodotnet:masterfrom
davkean:ProjectXmlUtilities
Oct 3, 2017
Merged

Avoid iterator/list allocations when parsing projects#2577
davkean merged 6 commits intodotnet:masterfrom
davkean:ProjectXmlUtilities

Conversation

@davkean
Copy link
Copy Markdown
Member

@davkean davkean commented Oct 3, 2017

For each project and for every container construct in a project (ItemGroup, ImportGroup, etc) we were allocating a List<XmlElementWithLocation> and a XmlChildEnumerator. In large solutions with large import graphs this was adding up to a non-trivial amount of allocations (1.7% of all allocations during evaluation in one trace).

Replaces these with a struct-based enumerable/enumerator that manually walks the child elements.

For each project and for every container constructor in a project (ItemGroup, ImportGroup, etc) we were allocating a List<XmlElementWithLocation> and a XmlChildEnumerator. In large solutions with large import graphs this was adding up to a non-trivial amount of allocations (1.7% of all allocations during evaluation in one trace).

These with a struct-based enumerable/enumerator that manually walks the child elements.
@davkean
Copy link
Copy Markdown
Member Author

davkean commented Oct 3, 2017

This is the allocations in one trace I looked at:

image

@cdmihai
Copy link
Copy Markdown
Contributor

cdmihai commented Oct 3, 2017

@dotnet-bot test Windows_NT Build for CoreCLR please

@davkean
Copy link
Copy Markdown
Member Author

davkean commented Oct 3, 2017

Here are the numbers:

Test Type Overall Significant Value
DotnetConsoleProject Evaluation: Time (ms) 👌 no 34.7853 -> 34.844 (0.169%)
DotnetConsoleProject Evaluation: Memory yes 5715452 -> 5566833 (-2.6%)
DotnetWebProject Evaluation: Time (ms) 👌 no 49.8234 -> 49.7828 (-0.081%)
DotnetWebProject Evaluation: Memory yes 7450200 -> 7281210 (-2.268%)
DotnetMvcProject Evaluation: Time (ms) 👌 no 84.5729 -> 84.2088 (-0.431%)
DotnetMvcProject Evaluation: Memory yes 9734911 -> 9566142 (-1.734%)
Picasso Evaluation: Time (ms) 👌 no 319.7085 -> 320.2459 (0.168%)
Picasso Evaluation: Memory yes 44420561 -> 43906740 (-1.157%)
SmallP2POldCsproj Evaluation: Time (ms) yes 54.113 -> 53.7609 (-0.651%)
SmallP2POldCsproj Evaluation: Memory yes 7557055 -> 7426286 (-1.73%)
Generated_100_100_v150 Evaluation: Time (ms) 👌 no 1504.9799 -> 1506.4059 (0.095%)
Generated_100_100_v150 Evaluation: Memory yes 187470244 -> 185709264 (-0.939%)
LargeP2POldCsproj Evaluation: Time (ms) yes 970.1681 -> 964.5588 (-0.578%)
LargeP2POldCsproj Evaluation: Memory yes 110015017 -> 109549813 (-0.423%)
roslyn Evaluation: Time (ms) 🔴 yes 5399.7462 -> 5430.1103 (0.562%)
roslyn Evaluation: Memory 👌 no 569358676 -> 574803028 (0.956%)

}
}

public void Reset()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to have a Reset method if you're not implementing the interfaces? Is Reset part of the pattern that foreach looks for?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll check.


public XmlElementChildIterator GetEnumerator()
{
return this;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be a standard pattern, but is it correct to have the enumerable and the enumerator be the same object? It seems like the contract would be that you can call GetEnumerator() multiple times on the same enumerable and you should get different enumerators / iterators with different state.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a pattern used by LINQ- http://index/?query=Enumerable.Select&rightProject=System.Core&file=System%5CLinq%5CEnumerable.cs&line=90. I was a little worried about increasing the size to handle this case (because it will never be hit), but I noticed they use a "state" field to track this and I might do the same thing.

Compiler doesn't require a enumerator to have Reset.
We were throwing when <Import>, <UsingTask>, etc had children, but we were picking the whitespace/comment (which were allowed) instead of the first illegal children.
@davkean
Copy link
Copy Markdown
Member Author

davkean commented Oct 3, 2017

@dsplaisted I addressed your comments, changes:

  • I assert the state machine - I make it illegal to double iterate, if we need to do that it wouldn't be too hard to clone the iterator.
  • I removed Reset, it's not needed.
  • I fixed up the check for illegal children - it was using the wrong child for errors.

Copy link
Copy Markdown
Contributor

@AndyGerlicher AndyGerlicher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@davkean
Copy link
Copy Markdown
Member Author

davkean commented Oct 3, 2017

tag @Pilchie

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants