Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build IndexStore together with the host toolchain #1456

Closed
wants to merge 564 commits into from

Conversation

MaxDesiatov
Copy link

IndexStore is required for swift build --enable-test-discovery to work.

@MaxDesiatov MaxDesiatov added the continuous integration Improvements to continuous integration label Jul 14, 2020
@MaxDesiatov MaxDesiatov changed the title Add IndexStore and SourceKit-LSP to host toolchain Build IndexStore together with the host toolchain Jul 20, 2020
dan-zheng and others added 26 commits August 19, 2020 17:05
Previously, the flag was omitted, causing function types which were
otherwise the same to have the same id, leading to caching woes.

Here, the issue is fixed by adding the boolean flag to the id, ensuring
that types which differ only in that flag are still understood to be
different.
Codesign the Error binary before executing
…ance cache

to use it.

ConcurrentReadableHashMap is lock-free for readers, with writers using a lock to
ensure mutual exclusion amongst each other. The intent is to eventually replace
all uses ConcurrentMap with ConcurrentReadableHashMap.

ConcurrentReadableHashMap provides for relatively quick lookups by using a hash
table. Rearders perform an atomic increment/decrement in order to inform writers
that there are active readers. The design attempts to minimize wasted memory by
storing the actual elements out-of-line, and having the table store indices into
a separate array of elements.

The protocol conformance cache now uses ConcurrentReadableHashMap, which
provides faster lookups and less memory use than the previous ConcurrentMap
implementation. The previous implementation caches
ProtocolConformanceDescriptors and extracts the WitnessTable after the cache
lookup. The new implementation directly caches the WitnessTable, removing an
extra step (potentially a quite slow one) from the fast path.

The previous implementation used a generational scheme to detect when negative
cache entries became obsolete due to new dynamic libraries being loaded, and
update them in place. The new implementation just clears the entire cache when
libraries are loaded, greatly simplifying the code and saving the memory needed
to track the current generation in each negative cache entry. This means we need
to re-cache all requested conformances after loading a dynamic library, but
loading libraries at runtime is rare and slow anyway.

rdar://problem/67268325
…d swiftpm

Along with options to disable the mandatory clean using: `--skip-clean-swift-driver` and `--skip-clean-swiftpm`.

This will ensure that every invocation of build-script will, by default, clean up all build artifacts of these projects and re-build them from scratch.
This is needed for all builds because today arbitrary changes to the compiler can lead to us being unable to incrementally build components that are themselves written in Swift. This causes now-frequent failures in incremental build bots, and is a scenario that is encountered by developers. (For example see: rdar://65006593)

The proper long-term solution is to enable library evolution for these projects. Until this is done, the only safe thing to do is to always rebuild them.

Resolves rdar://65006593
"do" blocks will always go through buildBlock().
… scan

This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.

This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.
[PrintAsObjC] Forward-declare bridged types
[CodeCompletion] Avoid prioritizing unavailable in LookupVisibleDecls
When @compatibility_alias is used with an ObjC generic class, this ends up importing as a generic typealias. PrintAsObjC previously didn’t handle declarations involving these types correctly; it would fail an assertion in asserts compilers, and potentially print an incorrect compatibility header in non-asserts compilers.

This PR makes it so that PrintAsObjC can now correctly use generic typealiases imported from Objective-C modules. It is, of course, still not possible to declare a generic typealias in Swift that will be printed into the Objective-C header.

Fixes rdar://67256866.
…ted types

Extensions to implementation-only types are accepted at type-checking
only if they don't define any public members. However if they declared a
conformance to a public type they were also printed in the
swiftinterface, making it unparsable because of an unknown type.

Still accept such extensions but don't print them.

rdar://problem/67516588
CodaFi and others added 25 commits August 28, 2020 14:07
…in-sil-setup

Setup code for Clang types in SIL.
Add error reporting when looking up types by demangled name.
…ctor expressions that contain code completions.
…ompletion exprs and type check them for code completion.

Fixes up some tests marked as non-ideal to give the ideal result now too.
…ayed parsing as well

We were previously only doing it when parsing up front.
CSGen: Infer generic arguments for explicit closure result types
…247649fefab99d7dc7b

[ownership] Move SemanticARCOpts into a separate folder in preparation for splitting into multiple small pseudo-passes.
…t-async

 [Concurrency] Import Objective-C methods with completion handlers as async
[Parse][IDE] Various parser fixes for code completion
…merge

# Conflicts:
#	stdlib/public/stubs/ThreadLocalStorage.cpp
#	utils/build_swift/build_swift/driver_arguments.py
@MaxDesiatov MaxDesiatov closed this Oct 2, 2020
@MaxDesiatov MaxDesiatov deleted the maxd/indexstore branch October 2, 2020 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
continuous integration Improvements to continuous integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet