I apologise in advance for the size of this check-in. At Intel we do

understand that this is not friendly, and are working to change our
internal code-development to make it easier to make development
features available more frequently and in finer (more functional)
chunks. Unfortunately we haven't got that in place yet, and unpicking
this into multiple separate check-ins would be non-trivial, so please
bear with me on this one. We should be better in the future.

Apologies over, what do we have here?

GGC 4.9 compatibility
--------------------
* We have implemented the new entrypoints used by code compiled by GCC
4.9 to implement the same functionality in gcc 4.8. Therefore code
compiled with gcc 4.9 that used to work will continue to do so.
However, there are some other new entrypoints (associated with task
cancellation) which are not implemented. Therefore user code compiled
by gcc 4.9 that uses these new features will not link against the LLVM
runtime. (It remains unclear how to handle those entrypoints, since
the GCC interface has potentially unpleasant performance implications
for join barriers even when cancellation is not used)

--- new parallel entry points ---
new entry points that aren't OpenMP 4.0 related
These are implemented fully :-
      GOMP_parallel_loop_dynamic()
      GOMP_parallel_loop_guided()
      GOMP_parallel_loop_runtime()
      GOMP_parallel_loop_static()
      GOMP_parallel_sections()
      GOMP_parallel()

--- cancellation entry points ---
Currently, these only give a runtime error if OMP_CANCELLATION is true
because our plain barriers don't check for cancellation while waiting
        GOMP_barrier_cancel()
        GOMP_cancel()
        GOMP_cancellation_point()
        GOMP_loop_end_cancel()
        GOMP_sections_end_cancel()

--- taskgroup entry points ---
These are implemented fully.
      GOMP_taskgroup_start()
      GOMP_taskgroup_end()

--- target entry points ---
These are empty (as they are in libgomp)
     GOMP_target()
     GOMP_target_data()
     GOMP_target_end_data()
     GOMP_target_update()
     GOMP_teams()

Improvements in Barriers and Fork/Join
--------------------------------------
* Barrier and fork/join code is now in its own file (which makes it
easier to understand and modify).
* Wait/release code is now templated and in its own file; suspend/resume code is also templated
* There's a new, hierarchical, barrier, which exploits the
cache-hierarchy of the Intel(r) Xeon Phi(tm) coprocessor to improve
fork/join and barrier performance.

***BEWARE*** the new source files have *not* been added to the legacy
Cmake build system. If you want to use that fixes wil be required.

Statistics Collection Code
--------------------------
* New code has been added to collect application statistics (if this
is enabled at library compile time; by default it is not). The
statistics code itself is generally useful, the lightweight timing
code uses the X86 rdtsc instruction, so will require changes for other
architectures.
The intent of this code is not for users to tune their codes but
rather 
1) For timing code-paths inside the runtime
2) For gathering general properties of OpenMP codes to focus attention
on which OpenMP features are most used. 

Nested Hot Teams
----------------
* The runtime now maintains more state to reduce the overhead of
creating and destroying inner parallel teams. This improves the
performance of code that repeatedly uses nested parallelism with the
same resource allocation. Set the new KMP_HOT_TEAMS_MAX_LEVEL
envirable to a depth to enable this (and, of course, OMP_NESTED=true
to enable nested parallelism at all).

Improved Intel(r) VTune(Tm) Amplifier support
---------------------------------------------
* The runtime provides additional information to Vtune via the
itt_notify interface to allow it to display better OpenMP specific
analyses of load-imbalance.

Support for OpenMP Composite Statements
---------------------------------------
* Implement new entrypoints required by some of the OpenMP 4.1
composite statements.

Improved ifdefs
---------------
* More separation of concepts ("Does this platform do X?") from
platforms ("Are we compiling for platform Y?"), which should simplify
future porting.


ScaleMP* contribution
---------------------
Stack padding to improve the performance in their environment where
cross-node coherency is managed at the page level.

Redesign of wait and release code
---------------------------------
The code is simplified and performance improved.

Bug Fixes
---------
    *Fixes for Windows multiple processor groups.
    *Fix Fortran module build on Linux: offload attribute added.
    *Fix entry names for distribute-parallel-loop construct to be consistent with the compiler codegen.
    *Fix an inconsistent error message for KMP_PLACE_THREADS environment variable.

llvm-svn: 219214

Author: jcownie-intel

openmp/CREDITS.txt

@@ -9,6 +9,7 @@ beautification by scripts.  The fields are: name (N), email (E), web-address
 (S).
 
 N: Carlo Bertolli
+W: http://ibm.com
 D: IBM contributor to PowerPC support in CMake files and elsewhere.
 
 N: Sunita Chandrasekaran
@@ -28,6 +29,11 @@ D: Created the runtime.
 N: Matthias Muller
 D: Contributor to testsuite from OpenUH
 
+N: Tal Nevo
+E: tal@scalemp.com
+D: ScaleMP contributor to improve runtime performance there.
+W: http://scalemp.com
+
 N: Pavel Neytchev
 D: Contributor to testsuite from OpenUH
 

openmp/LICENSE.txt

@@ -14,7 +14,7 @@ software contained in this directory tree is included below.
 University of Illinois/NCSA
 Open Source License
 
-Copyright (c) 1997-2013 Intel Corporation
+Copyright (c) 1997-2014 Intel Corporation
 
 All rights reserved.
 
@@ -51,7 +51,7 @@ SOFTWARE.
 
 ==============================================================================
 
-Copyright (c) 1997-2013 Intel Corporation
+Copyright (c) 1997-2014 Intel Corporation
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

openmp/runtime/Build_With_CMake.txt

@@ -137,8 +137,7 @@ libiomp5 version can be 5 or 4.
 OpenMP version can be either 40 or 30.
 
 -Dmic_arch=knc|knf
-Intel(R) MIC Architecture.  Can be 
-knf (Knights Ferry) or knc (Knights Corner).
+Intel(R) MIC Architecture, can be knf or knc.
 This value is ignored if os != mic
 
 -Dmic_os=lin|bsd

openmp/runtime/CMakeLists.txt

@@ -238,8 +238,18 @@ set(USE_BUILDPL_RULES false CACHE BOOL "Should the build follow build.pl rules/r
 # - these predefined linker flags should work for Windows, Mac, and True Linux for the most popular compilers/linkers
 set(USE_PREDEFINED_LINKER_FLAGS true CACHE BOOL "Should the build use the predefined linker flags in CommonFlags.cmake?")
 
+# - On multinode systems, larger alignment is desired to avoid false sharing
+set(USE_INTERNODE_ALIGNMENT false CACHE BOOL "Should larger alignment (4096 bytes) be used for some locks and data structures?")
+
+# - libgomp drop-in compatibility
+if(${LINUX} AND NOT ${PPC64})
+    set(USE_VERSION_SYMBOLS true CACHE BOOL "Should version symbols be used? These provide binary compatibility with libgomp.")
+else()
+    set(USE_VERSION_SYMBOLS false CACHE BOOL "Should version symbols be used? These provide binary compatibility with libgomp.")
+endif()
+
 # - TSX based locks have __asm code which can be troublesome for some compilers.  This feature is also x86 specific.
-if({${IA32} OR ${INTEL64})
+if(${IA32} OR ${INTEL64})
     set(USE_ADAPTIVE_LOCKS true CACHE BOOL "Should TSX-based lock be compiled (adaptive lock in kmp_lock.cpp).  These are x86 specific.")
 else()
     set(USE_ADAPTIVE_LOCKS false CACHE BOOL "Should TSX-based lock be compiled (adaptive lock in kmp_lock.cpp).  These are x86 specific.")

openmp/runtime/README.txt

@@ -37,7 +37,7 @@ omp_root:    The path to the top-level directory containing the top-level
 	     current working directory.
 
 omp_os:      Operating system.  By default, the build will attempt to 
-	     detect this. Currently supports "linux", "freebsd", "macos", and
+	     detect this. Currently supports "linux", "freebsd", "macos", and 
 	     "windows".
 
 arch:        Architecture. By default, the build will attempt to 
@@ -72,36 +72,44 @@ There is also an experimental CMake build system. This is *not* yet
 supported for production use and resulting binaries have not been checked
 for compatibility.
 
+On OS X* machines, it is possible to build universal (or fat) libraries which
+include both IA-32 architecture and Intel(R) 64 architecture objects in a
+single archive; just build the 32 and 32e libraries separately, then invoke 
+make again with a special argument as follows:
+
+make compiler=clang build_args=fat
+
 Supported RTL Build Configurations
 ==================================
 
 Supported Architectures: IA-32 architecture, Intel(R) 64, and 
 Intel(R) Many Integrated Core Architecture
 
-              --------------------------------------------  
-              |   icc/icl     |    gcc      |   clang    |
---------------|---------------|--------------------------|
-| Linux* OS   |   Yes(1,5)    |  Yes(2,4)   | Yes(4,6,7) |
-| FreeBSD*    |   No          |  No         | Yes(4,6,7) |
-| OS X*       |   Yes(1,3,4)  |  No         | Yes(4,6,7) |
-| Windows* OS |   Yes(1,4)    |  No         | No         |
-----------------------------------------------------------
+              ----------------------------------------------
+              |   icc/icl     |    gcc      |   clang      |
+--------------|---------------|----------------------------|
+| Linux* OS   |   Yes(1,5)    |  Yes(2,4)   | Yes(4,6,7)   |
+| FreeBSD*    |   No          |  No         | Yes(4,6,7,8) |
+| OS X*       |   Yes(1,3,4)  |  No         | Yes(4,6,7)   |
+| Windows* OS |   Yes(1,4)    |  No         | No           |
+------------------------------------------------------------
 
 (1) On IA-32 architecture and Intel(R) 64, icc/icl versions 12.x are 
     supported (12.1 is recommended).
-(2) gcc version 4.6.2 is supported.
+(2) GCC* version 4.6.2 is supported.
 (3) For icc on OS X*, OS X* version 10.5.8 is supported.
 (4) Intel(R) Many Integrated Core Architecture not supported.
 (5) On Intel(R) Many Integrated Core Architecture, icc/icl versions 13.0 
     or later are required.
-(6) clang version 3.3 is supported.
-(7) clang currently does not offer a software-implemented 128 bit extended 
+(6) Clang* version 3.3 is supported.
+(7) Clang* currently does not offer a software-implemented 128 bit extended 
     precision type.  Thus, all entry points reliant on this type are removed
     from the library and cannot be called in the user program.  The following
     functions are not available:
     __kmpc_atomic_cmplx16_*
     __kmpc_atomic_float16_*
     __kmpc_atomic_*_fp
+(8) Community contribution provided AS IS, not tested by Intel.
 
 Front-end Compilers that work with this RTL
 ===========================================

openmp/runtime/cmake/BuildPLRules.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 ###############################################################################
 # This file contains additional build rules that correspond to build.pl's rules
 # Building libiomp5.dbg is linux only, Windows will build libiomp5md.dll.pdb

openmp/runtime/cmake/Clang/AsmFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds Clang (clang/clang++) specific compiler dependent flags
 # The flag types are:
 #   1) Assembly flags

openmp/runtime/cmake/Clang/CFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds Clang (clang/clang++) specific compiler dependent flags
 # The flag types are:
 #   1) C/C++ Compiler flags
@@ -19,6 +30,7 @@ function(append_compiler_specific_c_and_cxx_flags input_c_flags input_cxx_flags)
     endif()
     append_c_and_cxx_flags("-Wno-unused-value") # Don't warn about unused values
     append_c_and_cxx_flags("-Wno-switch") # Don't warn about switch statements that don't cover entire range of values
+    append_c_and_cxx_flags("-Wno-deprecated-register") # Don't warn about using register keyword
     set(${input_c_flags}   ${${input_c_flags}}   "${local_c_flags}" PARENT_SCOPE)
     set(${input_cxx_flags} ${${input_cxx_flags}} "${local_cxx_flags}" PARENT_SCOPE)
 endfunction()

openmp/runtime/cmake/CommonFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds the common flags independent of compiler
 # The flag types are: 
 #   1) Assembly flags          (append_asm_flags_common)
@@ -71,22 +82,21 @@ function(append_linker_flags_common input_ld_flags input_ld_flags_libs)
     set(local_ld_flags)
     set(local_ld_flags_libs)
 
-    #################################
-    # Windows linker flags
-    if(${WINDOWS}) 
+    if(${USE_PREDEFINED_LINKER_FLAGS})
+
+        #################################
+        # Windows linker flags
+        if(${WINDOWS}) 
 
-    ##################
-    # MAC linker flags
-    elseif(${MAC})
-        if(${USE_PREDEFINED_LINKER_FLAGS})
+        ##################
+        # MAC linker flags
+        elseif(${MAC})
             append_linker_flags("-single_module")
             append_linker_flags("-current_version ${version}.0")
             append_linker_flags("-compatibility_version ${version}.0")
-        endif()
-    #####################################################################################
-    # Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) linker flags
-    elseif(${MIC})
-        if(${USE_PREDEFINED_LINKER_FLAGS})
+        #####################################################################################
+        # Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) linker flags
+        elseif(${MIC})
             append_linker_flags("-Wl,-x")
             append_linker_flags("-Wl,--warn-shared-textrel") #  Warn if the linker adds a DT_TEXTREL to a shared object.
             append_linker_flags("-Wl,--as-needed")
@@ -98,13 +108,11 @@ function(append_linker_flags_common input_ld_flags input_ld_flags_libs)
             if(${STATS_GATHERING})
                 append_linker_flags_library("-Wl,-lstdc++") # link in standard c++ library (stats-gathering needs it)
             endif()
-        endif()
-    #########################
-    # Unix based linker flags
-    else()
-        # For now, always include --version-script flag on Unix systems.
-        append_linker_flags("-Wl,--version-script=${src_dir}/exports_so.txt") # Use exports_so.txt as version script to create versioned symbols for ELF libraries
-        if(${USE_PREDEFINED_LINKER_FLAGS})
+        #########################
+        # Unix based linker flags
+        else()
+            # For now, always include --version-script flag on Unix systems.
+            append_linker_flags("-Wl,--version-script=${src_dir}/exports_so.txt") # Use exports_so.txt as version script to create versioned symbols for ELF libraries
             append_linker_flags("-Wl,-z,noexecstack") #  Marks the object as not requiring executable stack.
             append_linker_flags("-Wl,--as-needed")    #  Only adds library dependencies as they are needed. (if libiomp5 actually uses a function from the library, then add it)
             if(NOT ${STUBS_LIBRARY})
@@ -117,8 +125,9 @@ function(append_linker_flags_common input_ld_flags input_ld_flags_libs)
                     append_linker_flags_library("-Wl,-ldl") # link in libdl (dynamic loader library)
                 endif()
             endif()
-        endif() # if(${USE_PREDEFINED_LINKER_FLAGS})
-    endif() # if(${OPERATING_SYSTEM}) ...
+        endif() # if(${OPERATING_SYSTEM}) ...
+
+    endif() # USE_PREDEFINED_LINKER_FLAGS
 
     set(${input_ld_flags}      "${${input_ld_flags}}"      "${local_ld_flags}"      "${USER_LD_FLAGS}"     PARENT_SCOPE)
     set(${input_ld_flags_libs} "${${input_ld_flags_libs}}" "${local_ld_flags_libs}" "${USER_LD_LIB_FLAGS}" PARENT_SCOPE)

openmp/runtime/cmake/Definitions.cmake

@@ -42,6 +42,10 @@ function(append_cpp_flags input_cpp_flags)
     endif()
     append_definitions("-D INTEL_ITTNOTIFY_PREFIX=__kmp_itt_")
 
+    if(${USE_VERSION_SYMBOLS})
+        append_definitions("-D KMP_USE_VERSION_SYMBOLS")
+    endif()
+
     #####################
     # Windows definitions
     if(${WINDOWS})
@@ -133,6 +137,11 @@ function(append_cpp_flags input_cpp_flags)
         append_definitions("-D KMP_USE_ADAPTIVE_LOCKS=0")
         append_definitions("-D KMP_DEBUG_ADAPTIVE_LOCKS=0")
     endif()
+    if(${USE_INTERNODE_ALIGNMENT})
+        append_definitions("-D KMP_USE_INTERNODE_ALIGNMENT=1")
+    else()
+        append_definitions("-D KMP_USE_INTERNODE_ALIGNMENT=0")
+    endif()
     set(${input_cpp_flags} "${${input_cpp_flags}}" "${local_cpp_flags}" "${USER_CPP_FLAGS}" "$ENV{CPPFLAGS}" PARENT_SCOPE)
 endfunction()
 

openmp/runtime/cmake/GNU/AsmFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds GNU (gcc/g++) specific compiler dependent flags
 # The flag types are:
 #   1) Assembly flags

openmp/runtime/cmake/GNU/CFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds GNU (gcc/g++) specific compiler dependent flags
 # The flag types are:
 #   2) C/C++ Compiler flags

openmp/runtime/cmake/GNU/FortranFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds GNU (gcc/g++) specific compiler dependent flags
 # The flag types are:
 #   1) Fortran Compiler flags

openmp/runtime/cmake/Intel/AsmFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds Intel(R) C Compiler / Intel(R) C++ Compiler / Intel(R) Fortran Compiler (icc/icpc/icl.exe/ifort) dependent flags
 # The flag types are:
 #   1) Assembly flags

openmp/runtime/cmake/Intel/CFlags.cmake

@@ -1,3 +1,14 @@
+#
+#//===----------------------------------------------------------------------===//
+#//
+#//                     The LLVM Compiler Infrastructure
+#//
+#// This file is dual licensed under the MIT and the University of Illinois Open
+#// Source Licenses. See LICENSE.txt for details.
+#//
+#//===----------------------------------------------------------------------===//
+#
+
 # This file holds Intel(R) C Compiler / Intel(R) C++ Compiler / Intel(R) Fortran Compiler (icc/icpc/icl.exe/ifort) dependent flags
 # The flag types are:
 #   2) C/C++ Compiler flags
@@ -41,7 +52,6 @@ function(append_compiler_specific_c_and_cxx_flags input_c_flags input_cxx_flags)
         endif()
     else()
         append_c_and_cxx_flags("-Wsign-compare") # warn on sign comparisons
-        append_c_and_cxx_flags("-Werror") # Changes all warnings to errors.
         append_c_and_cxx_flags("-Qoption,cpp,--extended_float_types") # Enabled _Quad type.
         append_c_and_cxx_flags("-fno-exceptions") # Exception handling table generation is disabled.
         append_c_and_cxx_flags("-x c++") # Compile C files as C++ files