Merge 2020-08-08 into tensorflow

docs/ABI/Mangling.rst

@@ -1047,3 +1047,63 @@ Some kinds need arguments, which precede ``Tf``.
 If the first character of the string literal is a digit ``[0-9]`` or an
 underscore ``_``, the identifier for the string literal is prefixed with an
 additional underscore ``_``.
+
+Conventions for foreign symbols
+-------------------------------
+
+Swift interoperates with multiple other languages - C, C++, Objective-C, and
+Objective-C++. Each of these languages defines their own mangling conventions,
+so Swift must take care to follow them. However, these conventions do not cover
+Swift-specific symbols like Swift type metadata for foreign types, so Swift uses
+its own mangling scheme for those symbols.
+
+Importing C and C++ structs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Types imported from C and C++ are imported as if they are located in the ``__C``
+module, regardless of the actual Clang module that they are coming from. This
+can be observed when mangling a Swift function that accepts a C/C++ struct as a
+parameter:
+
+C++ module ``CxxStructModule``:
+
+.. code-block:: c++
+
+  struct CxxStruct {};
+
+  inline void cxxFunction(CxxStruct s) {}
+
+Swift module ``main`` that imports ``CxxStructModule``:
+
+.. code-block:: swift
+
+  import CxxStructModule
+
+  public func swiftFunction(_ s: CxxStruct) {}
+
+Resulting symbols (showing only Itanium-mangled C++ symbols for brevity):
+
+.. code::
+
+  _Z11cxxFunction9CxxStruct // -> cxxFunction(CxxStruct)
+  s4main13swiftFunctionyySo9CxxStructVF // -> main.swiftFunction(__C.CxxStruct) -> ()
+
+The reason for ignoring the Clang module and always putting C and C++ types into
+``__C`` at the Swift ABI level is that the Clang module is not a part of the C
+or C++ ABI. When owners of C and C++ Clang modules decide what changes are
+ABI-compatible or not, they will likely take into account C and C++ ABI, but not
+the Swift ABI. Therefore, Swift ABI can only encode information about a C or C++
+type that the C and C++ ABI already encodes in order to remain compatible with
+future versions of libraries that evolve according to C and C++ ABI
+compatibility principles.
+
+The C/C++ compiler does not generate Swift metadata symbols and value witness
+tables for C and C++ types. To make a foreign type usable in Swift in the same
+way as a native type, the Swift compiler must generate these symbols.
+Specifically, each Swift module that uses a given C or C++ type generates the
+necessary Swift symbols. For the example above the Swift compiler will generate following
+nominal type descriptor symbol for ``CxxStruct`` while compiling the ``main`` module:
+
+.. code::
+
+  sSo9CxxStructVMn // -> nominal type descriptor for __C.CxxStruct

docs/ContinuousIntegration.md

@@ -106,9 +106,11 @@ macOS platform               | @swift-ci Please smoke benchmark              | S
 Linux platform               | @swift-ci Please test Linux platform          | Swift Test Linux Platform (smoke test)<br>Swift Test Linux Platform
 Linux platform               | @swift-ci Please clean test Linux platform    | Swift Test Linux Platform (smoke test)<br>Swift Test Linux Platform
 macOS platform               | @swift-ci Please ASAN test                    | Swift ASAN Test OS X Platform
-Ubuntu 20.04                | @swift-ci Please test Ubuntu 20.04 platform    | Swift Test Ubuntu 20.04 Platform
-CentOS 8                    | @swift-ci Please test CentOS 8 platform    | Swift Test CentOS 8 Platform
-Amazon Linux 2              | @swift-ci Please test Amazon Linux 2 platform    | Swift Test Amazon Linux 2 Platform
+Ubuntu 18.04                 | @swift-ci Please test Ubuntu 18.04 platform   | Swift Test Ubuntu 18.04 Platform
+Ubuntu 20.04                 | @swift-ci Please test Ubuntu 20.04 platform   | Swift Test Ubuntu 20.04 Platform
+CentOS 7                     | @swift-ci Please test CentOS 7 platform       | Swift Test CentOS 7 Platform
+CentOS 8                     | @swift-ci Please test CentOS 8 platform       | Swift Test CentOS 8 Platform
+Amazon Linux 2               | @swift-ci Please test Amazon Linux 2 platform | Swift Test Amazon Linux 2 Platform
 
 The core principles of validation testing is that:
 

docs/DifferentiableProgramming.md

@@ -893,8 +893,15 @@ extension Perceptron {
 
 ### `@differentiable` function types
 
-A subtype of normal function types with a different runtime representation,
-which stores metadata that allows their values to be differentiated anywhere.
+Differentiable functions are first-class values, identified by a
+`@differentiable` attribute in the function type. A `@differentiable` function
+type is a subtype of its corresponding normal function type (i.e. without a
+`@differentiable` attribute) with an extended ABI, which stores metadata that
+allows their values to be differentiated anywhere the function is passed. A
+`@differentiable(linear)` function type is a subtype of its corresponding
+`@differentiable` function type. A normal function can be implicitly converted
+to a `@differentiable` or `@differentiable(linear)` function with appropriate
+compile-time checks.
 
 ```swift
 func addOne(_ x: Float) -> Float { x + 1 }
@@ -920,8 +927,9 @@ func _(_ x: Float) -> (value: Float,
 
 ### Differential operators
 
-Standard library differentiation APIs that take `@differentiable` functions and
-return derivative functions or compute derivative values.
+Differential operators are APIs defined in the standard library that take
+`@differentiable` functions and return derivative functions or compute
+derivative values.
 
 ```swift
 // In the standard library:
@@ -2318,7 +2326,7 @@ As shown in the
 subsection, a `@differentiable` function value's runtime representation contains
 the original function along with extra information that allows the function to
 be differentiated (or transposed, if it is `@differentiable(linear)`). A
-@differentiable or `@differentiable(linear)` function value can be called like a
+`@differentiable` or `@differentiable(linear)` function value can be called like a
 non-`@differentiable` function. A `@differentiable(linear)` function value can
 be implicitly converted to a `@differentiable` one, which can be implicitly
 converted to a non-`@differentiable` one.

docs/README.md

@@ -105,7 +105,7 @@ documentation, please create a thread on the Swift forums under the
   - [RequestEvaluator.md](/docs/RequestEvaluator.md):
     Describes the request evaluator architecture, which is used for
     lazy type-checking and efficient caching.
-  - [Literal.md](/docs/Literal.md):
+  - [Literals.md](/docs/Literals.md):
     Describes type-checking and inference specifically for literals.
 - [Serialization.rst](/docs/Serialization.rst):
   Gives an overview of the LLVM bitcode format used for swiftmodules.
@@ -117,7 +117,7 @@ documentation, please create a thread on the Swift forums under the
     Provides an overview of the implementation of SIL in the compiler.
   - [OptimizerDesign.md](/docs/OptimizerDesign.md):
     Describes the design of the optimizer pipeline.
-  - [HighLevelSILOptimizations.rst](docs/HighLevelSILOptimizations.rst):
+  - [HighLevelSILOptimizations.rst](/docs/HighLevelSILOptimizations.rst):
     Describes how the optimizer understands the semantics of high-level
     operations on currency data types and optimizes accordingly.
     Includes a thorough discussion of the `@_semantics` attribute.

include/swift/AST/DiagnosticEngine.h

@@ -24,6 +24,8 @@
 #include "swift/AST/TypeLoc.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/Support/Allocator.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/VersionTuple.h"
 
 namespace swift {
@@ -742,8 +744,26 @@ namespace swift {
     void setLocalization(std::string locale, std::string path) {
       assert(!locale.empty());
       assert(!path.empty());
-      localization =
-          std::make_unique<diag::YAMLLocalizationProducer>(locale, path);
+      llvm::SmallString<128> filePath(path);
+      llvm::sys::path::append(filePath, locale);
+      llvm::sys::path::replace_extension(filePath, ".db");
+
+      // If the serialized diagnostics file not available,
+      // fallback to the `YAML` file.
+      if (llvm::sys::fs::exists(filePath)) {
+        if (auto file = llvm::MemoryBuffer::getFile(filePath)) {
+          localization = std::make_unique<diag::SerializedLocalizationProducer>(
+              std::move(file.get()));
+        }
+      } else {
+        llvm::sys::path::replace_extension(filePath, ".yaml");
+        // In case of missing localization files, we should fallback to messages
+        // from `.def` files.
+        if (llvm::sys::fs::exists(filePath)) {
+          localization =
+              std::make_unique<diag::YAMLLocalizationProducer>(filePath.str());
+        }
+      }
     }
 
     void ignoreDiagnostic(DiagID id) {

include/swift/AST/DiagnosticsCommon.def

@@ -178,5 +178,12 @@ WARNING(cross_imported_by_both_modules, none,
         "please report this bug to the maintainers of these modules",
         (Identifier, Identifier, Identifier))
 
+//------------------------------------------------------------------------------
+// MARK: dependencies scanner diagnostics
+//------------------------------------------------------------------------------
+
+ERROR(scanner_find_cycle, none,
+      "dependency scanner detected dependency cycle: '%0'", (StringRef))
+
 #define UNDEFINE_DIAGNOSTIC_MACROS
 #include "DefineDiagnosticMacros.h"

include/swift/AST/DiagnosticsSema.def

@@ -88,6 +88,13 @@ ERROR(could_not_find_subscript_member_did_you_mean,none,
       "did you mean to use the subscript operator?",
       (Type))
 
+ERROR(could_not_find_subscript_member_tuple, none,
+      "cannot access element using subscript for tuple type %0; "
+      "use '.' notation instead", (Type))
+ERROR(could_not_find_subscript_member_tuple_did_you_mean_use_dot, none,
+      "cannot access element using subscript for tuple type %0; "
+      "did you mean to use '.%1'?", (Type, StringRef))
+
 ERROR(could_not_find_enum_case,none,
       "enum type %0 has no case %1; did you mean %2?",
       (Type, DeclNameRef, DeclName))
@@ -2835,7 +2842,8 @@ NOTE(codable_extraneous_codingkey_case_here,none,
 NOTE(codable_non_conforming_property_here,none,
      "cannot automatically synthesize %0 because %1 does not conform to %0", (Type, TypeLoc))
 NOTE(codable_non_decoded_property_here,none,
-     "cannot automatically synthesize %0 because %1 does not have a matching CodingKey and does not have a default value", (Type, Identifier))
+     "cannot automatically synthesize %0 because %1 does not have a matching "
+     "CodingKey and does not have a default value", (Type, Identifier))
 NOTE(codable_codingkeys_type_is_not_an_enum_here,none,
      "cannot automatically synthesize %0 because 'CodingKeys' is not an enum", (Type))
 NOTE(codable_codingkeys_type_does_not_conform_here,none,
@@ -4064,10 +4072,13 @@ ERROR(throw_in_nonexhaustive_catch,none,
       "error is not handled because the enclosing catch is not exhaustive", ())
 
 ERROR(throwing_call_in_illegal_context,none,
-      "call can throw, but errors cannot be thrown out of %0",
-      (StringRef))
+      "call can throw, but errors cannot be thrown out of "
+      "%select{<<ERROR>>|a default argument|a property initializer|a global variable initializer|an enum case raw value|a catch pattern|a catch guard expression|a defer body}0",
+      (unsigned))
 ERROR(throw_in_illegal_context,none,
-      "errors cannot be thrown out of %0", (StringRef))
+      "errors cannot be thrown out of "
+      "%select{<<ERROR>>|a default argument|a property initializer|a global variable initializer|an enum case raw value|a catch pattern|a catch guard expression|a defer body}0",
+      (unsigned))
 
 ERROR(throwing_operator_without_try,none,
       "operator can throw but expression is not marked with 'try'", ())
@@ -4716,13 +4727,24 @@ ERROR(availability_decl_unavailable, none,
       "%select{ in %3|}2%select{|: %4}4",
       (unsigned, DeclName, bool, StringRef, StringRef))
 
+WARNING(availability_decl_unavailable_warn, none,
+        "%select{getter for |setter for |}0%1 is unavailable"
+        "%select{ in %3|}2%select{|: %4}4",
+        (unsigned, DeclName, bool, StringRef, StringRef))
+
 #define REPLACEMENT_DECL_KIND_SELECT "select{| instance method| property}"
 ERROR(availability_decl_unavailable_rename, none,
       "%select{getter for |setter for |}0%1 has been "
       "%select{renamed to|replaced by}2%" REPLACEMENT_DECL_KIND_SELECT "3 "
       "'%4'%select{|: %5}5",
       (unsigned, DeclName, bool, unsigned, StringRef, StringRef))
 
+WARNING(availability_decl_unavailable_rename_warn, none,
+        "%select{getter for |setter for |}0%1 has been "
+        "%select{renamed to|replaced by}2%" REPLACEMENT_DECL_KIND_SELECT "3 "
+        "'%4'%select{|: %5}5",
+        (unsigned, DeclName, bool, unsigned, StringRef, StringRef))
+
 NOTE(availability_marked_unavailable, none,
      "%select{getter for |setter for |}0%1 has been explicitly marked "
      "unavailable here", (unsigned, DeclName))

include/swift/AST/Expr.h

@@ -861,9 +861,6 @@ class StringLiteralExpr : public LiteralExpr {
     /// A UTF-8 string.
     UTF8,
 
-    /// A UTF-16 string.
-    UTF16,
-
     /// A single UnicodeScalar, passed as an integer.
     OneUnicodeScalar
   };
@@ -5259,6 +5256,7 @@ class KeyPathExpr : public Expr {
       OptionalWrap,
       Identity,
       TupleElement,
+      DictionaryKey,
     };
 
   private:
@@ -5367,6 +5365,16 @@ class KeyPathExpr : public Expr {
                        propertyType,
                        loc);
     }
+
+    /// Create a component for a dictionary key (#keyPath only).
+    static Component forDictionaryKey(DeclNameRef UnresolvedName,
+                                      Type valueType,
+                                      SourceLoc loc) {
+      return Component(nullptr, UnresolvedName, nullptr, {}, {},
+                       Kind::DictionaryKey,
+                       valueType,
+                       loc);
+    }
 
     /// Create a component for a subscript.
     static Component forSubscript(ASTContext &ctx,
@@ -5457,6 +5465,7 @@ class KeyPathExpr : public Expr {
       case Kind::Property:
       case Kind::Identity:
       case Kind::TupleElement:
+      case Kind::DictionaryKey:
         return true;
 
       case Kind::UnresolvedSubscript:
@@ -5481,6 +5490,7 @@ class KeyPathExpr : public Expr {
       case Kind::Property:
       case Kind::Identity:
       case Kind::TupleElement:
+      case Kind::DictionaryKey:
         return nullptr;
       }
       llvm_unreachable("unhandled kind");
@@ -5500,6 +5510,7 @@ class KeyPathExpr : public Expr {
       case Kind::Property:
       case Kind::Identity:
       case Kind::TupleElement:
+      case Kind::DictionaryKey:
         llvm_unreachable("no subscript labels for this kind");
       }
       llvm_unreachable("unhandled kind");
@@ -5522,6 +5533,7 @@ class KeyPathExpr : public Expr {
       case Kind::Property:
       case Kind::Identity:
       case Kind::TupleElement:
+      case Kind::DictionaryKey:
         return {};
       }
       llvm_unreachable("unhandled kind");
@@ -5533,6 +5545,7 @@ class KeyPathExpr : public Expr {
     DeclNameRef getUnresolvedDeclName() const {
       switch (getKind()) {
       case Kind::UnresolvedProperty:
+      case Kind::DictionaryKey:
         return Decl.UnresolvedName;
 
       case Kind::Invalid:
@@ -5563,6 +5576,7 @@ class KeyPathExpr : public Expr {
       case Kind::OptionalForce:
       case Kind::Identity:
       case Kind::TupleElement:
+      case Kind::DictionaryKey:
         llvm_unreachable("no decl ref for this kind");
       }
       llvm_unreachable("unhandled kind");
@@ -5582,6 +5596,7 @@ class KeyPathExpr : public Expr {
         case Kind::Identity:
         case Kind::Property:
         case Kind::Subscript:
+        case Kind::DictionaryKey:
           llvm_unreachable("no field number for this kind");
       }
       llvm_unreachable("unhandled kind");

include/swift/AST/GenericSignatureBuilder.h

@@ -751,7 +751,7 @@ class GenericSignatureBuilder {
   /// \param resolutionKind How to perform the resolution.
   ///
   /// \param wantExactPotentialArchetype Whether to return the precise
-  /// potential archetype described by the type (vs. just the equivalance
+  /// potential archetype described by the type (vs. just the equivalence
   /// class and resolved type).
   ResolvedType maybeResolveEquivalenceClass(
                                       Type type,

include/swift/AST/IRGenOptions.h

@@ -139,6 +139,12 @@ struct PointerAuthOptions : clang::PointerAuthOptions {
   PointerAuthSchema ResilientClassStubInitCallbacks;
 };
 
+enum class JITDebugArtifact : unsigned {
+  None,   ///< None
+  LLVMIR, ///< LLVM IR
+  Object, ///< Object File
+};
+
 /// The set of options supported by IR generation.
 class IRGenOptions {
 public:
@@ -326,6 +332,8 @@ class IRGenOptions {
   Optional<llvm::VersionTuple> AutolinkRuntimeCompatibilityLibraryVersion;
   Optional<llvm::VersionTuple> AutolinkRuntimeCompatibilityDynamicReplacementLibraryVersion;
 
+  JITDebugArtifact DumpJIT = JITDebugArtifact::None;
+
   IRGenOptions()
       : DWARFVersion(2), OutputKind(IRGenOutputKind::LLVMAssembly),
         Verify(true), OptMode(OptimizationMode::NotSet),