Skip to content

Cherry-pick compiler performance fixes [5.4] #35288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jan 11, 2021

Conversation

slavapestov
Copy link
Contributor

@slavapestov slavapestov commented Jan 7, 2021

(Mostly, see below) fixes rdar://problem/72906325.

Build times with 5.3 vs 5.4 vs this PR, building the 5.3 SwiftSyntax:

5.3:

spestov@technical-debt swift % sh perf-wmo-53.sh

real	0m42.825s
user	0m40.623s
sys	0m2.188s

spestov@technical-debt swift % sh perf-batch-53.sh

real	0m7.480s
user	0m35.239s
sys	0m6.752s

5.4:

spestov@technical-debt swift % sh perf-wmo-54.sh

real	0m46.477s
user	0m43.617s
sys	0m2.848s

spestov@technical-debt swift % sh perf-batch-54.sh

real	0m13.842s
user	0m53.109s
sys	0m8.135s

5.4 + this PR:

spestov@technical-debt swift % sh perf-wmo.sh

real	0m42.689s
user	0m40.464s
sys	0m2.214s

spestov@technical-debt swift % sh perf-batch.sh

real	0m7.994s
user	0m40.993s
sys	0m7.059s

You can see there’s still a small regression in batch mode. This is due to two issues which need to be addressed separately:

  • The use of MD5 for the interface hash, which is now computed in more instances.
  • A Clang regression that manifests via IRGen.

Let's avoid creating an ExportContext, which computes a bunch of
irrelevant stuff. Also, delay the expensive call to
overApproximateAvailabilityAtLocation() unless we know the
declaration is conditionally unavailable.
This doesn't really fit the request evaluator model since the
result of evaluating the request is the new insertion point,
but we don't have a way to get the insertion point of an
already-expanded scope.

Instead, let's change the callers of the now-removed
expandAndBeCurrentDetectingRecursion() to instead call
expandAndBeCurrent(), while checking first if the scope was
already expanded.

Also, set the expanded flag before calling expandSpecifically()
from inside expandAndBeCurrent(), to ensure that re-entrant
calls to expandAndBeCurrent() are flagged by the existing
assertion there.

Finally, replace a couple of existing counters, and the
now-gone request counter with a single ASTScopeExpansions
counter to track expansions.
Combine the wasExpanded flag with the parent pointer, and remove
the haveAddedCleanup flag since it's no longer necessary now that
"re-expansion" is gone.
Other than simplifying some code, the big improvement here is
that we 'freeze' the reference dependencies for a request down
to a simple vector. We only use a DenseSet to store dependencies
of active requests.
Also remove some unnecessary #includes from DependencyCollector.h,
which necessitated adding #includes in various other files.
This becomes tricky with the new per-request cache representation.

We can add it back later if we need it, but I feel like this code
path isn't particularly useful right now anyway.
This is based on an earlier patch by @hamishknight.

The idea is that instead of caching results in a single DenseMap
that maps AnyRequest to AnyValue, we instead define a separate
DenseMap for each request kind that directly uses the request as
the key, and the request value as the value.

This avoids type erasure and memory allocation overhead arising
from the use of AnyRequest and AnyValue. There are no remaining
usages of AnyValue, and the only usage of AnyRequest is now in
the reference dependency tracking, which can be refactored to use
a similar strategy of storing per-request maps as well.
... instead of passing around an ActiveRequest.
Just as with the result cache, instead of a single DenseMap with
type-erased AnyRequest keys, we can use per-request maps for a
nice performance improvement.
@slavapestov slavapestov marked this pull request as ready for review January 8, 2021 05:14
@slavapestov slavapestov requested a review from a team as a code owner January 8, 2021 05:14
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

swift-ci commented Jan 8, 2021

Build failed
Swift Test Linux Platform
Git Sha - b10dda6

@swift-ci
Copy link
Contributor

swift-ci commented Jan 8, 2021

Build failed
Swift Test OS X Platform
Git Sha - b10dda6

@DougGregor
Copy link
Member

@swift-ci please test

@swift-ci
Copy link
Contributor

swift-ci commented Jan 9, 2021

Build failed
Swift Test Linux Platform
Git Sha - b10dda6

@slavapestov
Copy link
Contributor Author

@swift-ci please test Linux

@swift-ci
Copy link
Contributor

swift-ci commented Jan 9, 2021

Build failed
Swift Test OS X Platform
Git Sha - b10dda6

@slavapestov
Copy link
Contributor Author

@swift-ci Please test macOS

@DougGregor DougGregor merged commit e81ed9f into swiftlang:release/5.4 Jan 11, 2021
@AnthonyLatsis AnthonyLatsis added swift 5.4 🍒 release cherry pick Flag: Release branch cherry picks labels Jan 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🍒 release cherry pick Flag: Release branch cherry picks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants