Re-apply r267784, r267824 and r267830.

I have updated the compiler-rt tests.

llvm-svn: 267903

Author: pcc

clang/docs/ControlFlowIntegrity.rst

@@ -25,13 +25,25 @@ As currently implemented, all schemes rely on link-time optimization (LTO);
 so it is required to specify ``-flto``, and the linker used must support LTO,
 for example via the `gold plugin`_.
 
-To allow the checks to be implemented efficiently, the program must be
-structured such that certain object files are compiled with CFI
+To allow the checks to be implemented efficiently, the program must
+be structured such that certain object files are compiled with CFI
 enabled, and are statically linked into the program. This may preclude
-the use of shared libraries in some cases. Experimental support for
-:ref:`cross-DSO control flow integrity <cfi-cross-dso>` exists that
-does not have these requirements. This cross-DSO support has unstable
-ABI at this time.
+the use of shared libraries in some cases.
+
+The compiler will only produce CFI checks for a class if it can infer hidden
+LTO visibility for that class. LTO visibility is a property of a class that
+is inferred from flags and attributes. For more details, see the documentation
+for :doc:`LTO visibility <LTOVisibility>`.
+
+The ``-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}`` flags
+require that a ``-fvisibility=`` flag also be specified. This is because the
+default visibility setting is ``-fvisibility=default``, which would disable
+CFI checks for classes without visibility attributes. Most users will want
+to specify ``-fvisibility=hidden``, which enables CFI checks for such classes.
+
+Experimental support for :ref:`cross-DSO control flow integrity
+<cfi-cross-dso>` exists that does not require classes to have hidden LTO
+visibility. This cross-DSO support has unstable ABI at this time.
 
 .. _gold plugin: http://llvm.org/docs/GoldPlugin.html
 
@@ -233,11 +245,6 @@ A :doc:`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
 source files, functions and types using the ``src``, ``fun`` and ``type``
 entity types.
 
-In addition, if a type has a ``uuid`` attribute and the blacklist contains
-the type entry ``attr:uuid``, CFI checks are suppressed for that type. This
-allows all COM types to be easily blacklisted, which is useful as COM types
-are typically defined outside of the linked program.
-
 .. code-block:: bash
 
     # Suppress checking for code in a file.
@@ -247,8 +254,6 @@ are typically defined outside of the linked program.
     fun:*MyFooBar*
     # Ignore all types in the standard library.
     type:std::*
-    # Ignore all types with a uuid attribute.
-    type:attr:uuid
 
 .. _cfi-cross-dso:
 
@@ -260,6 +265,11 @@ flow integrity mode, which allows all CFI schemes listed above to
 apply across DSO boundaries. As in the regular CFI, each DSO must be
 built with ``-flto``.
 
+Normally, CFI checks will only be performed for classes that have hidden LTO
+visibility. With this flag enabled, the compiler will emit cross-DSO CFI
+checks for all classes, except for those which appear in the CFI blacklist
+or which use a ``no_sanitize`` attribute.
+
 Design
 ======
 

clang/docs/LTOVisibility.rst

@@ -0,0 +1,111 @@
+==============
+LTO Visibility
+==============
+
+*LTO visibility* is a property of an entity that specifies whether it can be
+referenced from outside the current LTO unit. A *linkage unit* is a set of
+translation units linked together into an executable or DSO, and a linkage
+unit's *LTO unit* is the subset of the linkage unit that is linked together
+using link-time optimization; in the case where LTO is not being used, the
+linkage unit's LTO unit is empty. Each linkage unit has only a single LTO unit.
+
+The LTO visibility of a class is used by the compiler to determine which
+classes the virtual function call optimization and control flow integrity
+features apply to. These features use whole-program information, so they
+require the entire class hierarchy to be visible in order to work correctly.
+
+If any translation unit in the program uses either of the virtual function
+call optimization or control flow integrity features, it is effectively an
+ODR violation to define a class with hidden LTO visibility in multiple linkage
+units. A class with public LTO visibility may be defined in multiple linkage
+units, but the tradeoff is that the virtual function call optimization and
+control flow integrity features can only be applied to classes with hidden LTO
+visibility. A class's LTO visibility is treated as an ODR-relevant property
+of its definition, so it must be consistent between translation units.
+
+In translation units built with LTO, LTO visibility is based on symbol
+visibility or, on the Windows platform, the dllimport and dllexport
+attributes. When targeting non-Windows platforms, classes with a visibility
+other than hidden visibility receive public LTO visibility. When targeting
+Windows, classes with dllimport or dllexport attributes receive public LTO
+visibility. All other classes receive hidden LTO visibility. Classes with
+internal linkage (e.g. classes declared in unnamed namespaces) also receive
+hidden LTO visibility.
+
+A class defined in a translation unit built without LTO receives public
+LTO visibility regardless of its object file visibility, linkage or other
+attributes.
+
+This mechanism will produce the correct result in most cases, but there are
+two cases where it may wrongly infer hidden LTO visibility.
+
+1. As a corollary of the above rules, if a linkage unit is produced from a
+   combination of LTO object files and non-LTO object files, any hidden
+   visibility class defined in both a translation unit built with LTO and
+   a translation unit built without LTO must be defined with public LTO
+   visibility in order to avoid an ODR violation.
+
+2. Some ABIs provide the ability to define an abstract base class without
+   visibility attributes in multiple linkage units and have virtual calls
+   to derived classes in other linkage units work correctly. One example of
+   this is COM on Windows platforms. If the ABI allows this, any base class
+   used in this way must be defined with public LTO visibility.
+
+Classes that fall into either of these categories can be marked up with the
+``[[clang::lto_visibility_public]]`` attribute. To specifically handle the
+COM case, classes with the ``__declspec(uuid())`` attribute receive public
+LTO visibility. On Windows platforms, clang-cl's ``/MT`` and ``/MTd``
+flags statically link the program against a prebuilt standard library;
+these flags imply public LTO visibility for every class declared in the
+``std`` and ``stdext`` namespaces.
+
+Example
+=======
+
+The following example shows how LTO visibility works in practice in several
+cases involving two linkage units, ``main`` and ``dso.so``.
+
+.. code-block:: none
+
+    +-----------------------------------------------------------+  +----------------------------------------------------+
+    | main (clang++ -fvisibility=hidden):                       |  | dso.so (clang++ -fvisibility=hidden):              |
+    |                                                           |  |                                                    |
+    |  +-----------------------------------------------------+  |  |  struct __attribute__((visibility("default"))) C { |
+    |  | LTO unit (clang++ -fvisibility=hidden -flto):       |  |  |    virtual void f();                               |
+    |  |                                                     |  |  |  }                                                 |
+    |  |  struct A { ... };                                  |  |  |  void C::f() {}                                    |
+    |  |  struct [[clang::lto_visibility_public]] B { ... }; |  |  |  struct D {                                        |
+    |  |  struct __attribute__((visibility("default"))) C {  |  |  |    virtual void g() = 0;                           |
+    |  |    virtual void f();                                |  |  |  };                                                |
+    |  |  };                                                 |  |  |  struct E : D {                                    |
+    |  |  struct [[clang::lto_visibility_public]] D {        |  |  |    virtual void g() { ... }                        |
+    |  |    virtual void g() = 0;                            |  |  |  };                                                |
+    |  |  };                                                 |  |  |  __attribute__(visibility("default"))) D *mkE() {  |
+    |  |                                                     |  |  |    return new E;                                   |
+    |  +-----------------------------------------------------+  |  |  }                                                 |
+    |                                                           |  |                                                    |
+    |  struct B { ... };                                        |  +----------------------------------------------------+
+    |                                                           |
+    +-----------------------------------------------------------+
+
+We will now describe the LTO visibility of each of the classes defined in
+these linkage units.
+
+Class ``A`` is not defined outside of ``main``'s LTO unit, so it can have
+hidden LTO visibility. This is inferred from the object file visibility
+specified on the command line.
+
+Class ``B`` is defined in ``main``, both inside and outside its LTO unit. The
+definition outside the LTO unit has public LTO visibility, so the definition
+inside the LTO unit must also have public LTO visibility in order to avoid
+an ODR violation.
+
+Class ``C`` is defined in both ``main`` and ``dso.so`` and therefore must
+have public LTO visibility. This is correctly inferred from the ``visibility``
+attribute.
+
+Class ``D`` is an abstract base class with a derived class ``E`` defined
+in ``dso.so``.  This is an example of the COM scenario; the definition of
+``D`` in ``main``'s LTO unit must have public LTO visibility in order to be
+compatible with the definition of ``D`` in ``dso.so``, which is observable
+by calling the function ``mkE``.

clang/docs/UsersManual.rst

@@ -1056,17 +1056,8 @@ are listed below.
 .. option:: -fwhole-program-vtables
 
    Enable whole-program vtable optimizations, such as single-implementation
-   devirtualization and virtual constant propagation. Requires ``-flto``.
-
-   By default, the compiler will assume that all type hierarchies are
-   closed except those in the ``std`` namespace, the ``stdext`` namespace
-   and classes with the ``__declspec(uuid())`` attribute.
-
-.. option:: -fwhole-program-vtables-blacklist=path
-
-   Allows the user to specify the path to a list of additional classes to
-   blacklist from whole-program vtable optimizations. This list is in the
-   :ref:`CFI blacklist <cfi-blacklist>` format.
+   devirtualization and virtual constant propagation, for classes with
+   :doc:`hidden LTO visibility <LTOVisibility>`. Requires ``-flto``.
 
 .. option:: -fno-assume-sane-operator-new
 

clang/docs/index.rst

@@ -31,6 +31,7 @@ Using Clang as a Compiler
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
+   LTOVisibility
    SafeStack
    Modules
    MSVCCompatibility

clang/include/clang/Basic/Attr.td

@@ -1611,6 +1611,12 @@ def WeakRef : InheritableAttr {
   let Documentation = [Undocumented];
 }
 
+def LTOVisibilityPublic : InheritableAttr {
+  let Spellings = [CXX11<"clang", "lto_visibility_public">];
+  let Subjects = SubjectList<[Record]>;
+  let Documentation = [LTOVisibilityDocs];
+}
+
 def AnyX86Interrupt : InheritableAttr, TargetSpecificAttr<TargetAnyX86> {
   // NOTE: If you add any additional spellings, ARMInterrupt's,
   // MSP430Interrupt's and MipsInterrupt's spellings must match.

clang/include/clang/Basic/AttrDocs.td

@@ -2380,3 +2380,10 @@ The ``ifunc`` attribute may only be used on a function declaration.  A function
 Not all targets support this attribute.  ELF targets support this attribute when using binutils v2.20.1 or higher and glibc v2.11.1 or higher.  Non-ELF targets currently do not support this attribute.
   }];
 }
+
+def LTOVisibilityDocs : Documentation {
+  let Category = DocCatType;
+  let Content = [{
+See :doc:`LTOVisibility`.
+  }];
+}

clang/include/clang/Driver/CC1Options.td

@@ -282,6 +282,9 @@ def fprofile_instrument_path_EQ : Joined<["-"], "fprofile-instrument-path=">,
 def fprofile_instrument_use_path_EQ :
     Joined<["-"], "fprofile-instrument-use-path=">,
     HelpText<"Specify the profile path in PGO use compilation">;
+def flto_visibility_public_std:
+    Flag<["-"], "flto-visibility-public-std">,
+    HelpText<"Use public LTO visibility for classes in std and stdext namespaces">;
 
 //===----------------------------------------------------------------------===//
 // Dependency Output Options

clang/include/clang/Driver/Options.td

@@ -1152,9 +1152,6 @@ def fwhole_program_vtables : Flag<["-"], "fwhole-program-vtables">, Group<f_Grou
   Flags<[CC1Option]>,
   HelpText<"Enables whole-program vtable optimization. Requires -flto">;
 def fno_whole_program_vtables : Flag<["-"], "fno-whole-program-vtables">, Group<f_Group>;
-def fwhole_program_vtables_blacklist_EQ : Joined<["-"], "fwhole-program-vtables-blacklist=">,
-  Group<f_Group>, Flags<[CC1Option]>,
-  HelpText<"Path to a blacklist file for whole-program vtable optimization">;
 def fwrapv : Flag<["-"], "fwrapv">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Treat signed integer overflow as two's complement">;
 def fwritable_strings : Flag<["-"], "fwritable-strings">, Group<f_Group>, Flags<[CC1Option]>,

clang/include/clang/Frontend/CodeGenOptions.def

@@ -187,6 +187,10 @@ CODEGENOPT(EmitLLVMUseLists, 1, 0) ///< Control whether to serialize use-lists.
 CODEGENOPT(WholeProgramVTables, 1, 0) ///< Whether to apply whole-program
                                       ///  vtable optimization.
 
+/// Whether to use public LTO visibility for entities in std and stdext
+/// namespaces. This is enabled by clang-cl's /MT and /MTd flags.
+CODEGENOPT(LTOVisibilityPublicStd, 1, 0)
+
 /// The user specified number of registers to be used for integral arguments,
 /// or 0 if unspecified.
 VALUE_CODEGENOPT(NumRegisterParameters, 32, 0)

clang/include/clang/Frontend/CodeGenOptions.h

@@ -199,9 +199,6 @@ class CodeGenOptions : public CodeGenOptionsBase {
   /// \brief A list of all -fno-builtin-* function names (e.g., memset).
   std::vector<std::string> NoBuiltinFuncs;
 
-  /// List of blacklist files for the whole-program vtable optimization feature.
-  std::vector<std::string> WholeProgramVTablesBlacklistFiles;
-
 public:
   // Define accessors/mutators for code generation options of enumeration type.
 #define CODEGENOPT(Name, Bits, Default)