Merge pull request #1374 from mesonbuild/simd

Add support for SIMD detection

docs/markdown/Module-reference.md

@@ -1,4 +1,6 @@
-Meson has a selection of modules to make common requirements easy to use. Modules can be thought of like the standard library of a programming language. Currently Meson provides the following modules.
+Meson has a selection of modules to make common requirements easy to use.
+Modules can be thought of like the standard library of a programming language.
+Currently Meson provides the following modules.
 
 * [Gnome](Gnome-module.md)
 * [i18n](i18n-module.md)
@@ -8,3 +10,11 @@ Meson has a selection of modules to make common requirements easy to use. Module
 * [Python3](Python-3-module.md)
 * [RPM](RPM-module.md)
 * [Windows](Windows-module.md)
+
+In addition there are unstable modules. These are meant for testing new
+functionality but note that they do *not* provide a stable API. It can
+change in arbitrary ways between releases. The modules might also be removed
+without warning in future releases.
+
+ * [SIMD](Simd-module.md)
+

docs/markdown/Release-notes-for-0.42.0.md

@@ -58,3 +58,10 @@ Rust's [linkage reference][rust-linkage].
 
 Both the address- and undefined behavior sanitizers can now be used
 simultaneously by passing `-Db_sanitize=address,undefined` to Meson.
+
+## Unstable SIMD module
+
+A new experimental module to compile code with many different SIMD
+instruction sets and selecting the best one at runtime. This module
+is unstable, meaning its API is subject to change in later releases.
+It might also be removed altogether.

docs/markdown/Simd-module.md

@@ -0,0 +1,70 @@
+# Unstable SIMD module
+
+This module provides helper functionality to build code with SIMD instructions.
+Available since 0.42.0.
+
+**Note**: this module is unstable. It is only provided as a technology preview.
+Its API may change in arbitrary ways between releases or it might be removed
+from Meson altogether.
+
+## Usage
+
+This module is designed for the use case where you have an algorithm with one
+or more SIMD implementation and you choose which one to use at runtime.
+
+The module provides one method, `check`, which is used like this:
+
+    rval = simd.check('mysimds',
+      mmx : 'simd_mmx.c',
+      sse : 'simd_sse.c',
+      sse2 : 'simd_sse2.c',
+      sse3 : 'simd_sse3.c',
+      ssse3 : 'simd_ssse3.c',
+      sse41 : 'simd_sse41.c',
+      sse42 : 'simd_sse42.c',
+      avx : 'simd_avx.c',
+      avx2 : 'simd_avx2.c',
+      neon : 'simd_neon.c',
+      compiler : cc)
+
+Here the individual files contain the accelerated versions of the functions
+in question. The `compiler` keyword argument takes the compiler you are
+going to use to compile them. The function returns an array with two values.
+The first value is a bunch of libraries that contain the compiled code. Any
+SIMD code that the compiler can't compile (for example, Neon instructions on
+an x86 machine) are ignored. You should pass this value to the desired target
+using `link_with`. The second value is a `configuration_data` object that
+contains true for all the values that were supported. For example if the
+compiler did support sse2 instructions, then the object would have `HAVE_SSE2`
+set to 1.
+
+Generating code to detect the proper instruction set at runtime is
+straightforward. First you create a header with the configuration object and
+then a chooser function that looks like this:
+
+    void (*fptr)(type_of_function_here)  = NULL;
+
+    #if HAVE_NEON
+    if(fptr == NULL && neon_available()) {
+        fptr = neon_accelerated_function;
+    }
+    #endif
+    #if HAVE_AVX2
+    if(fptr == NULL && avx2_available()) {
+        fptr = avx_accelerated_function;
+    }
+    #endif
+
+    ...
+
+    if(fptr == NULL) {
+        fptr = default_function;
+    }
+
+Each source file provides two functions, the `xxx_available` function to query
+whether the CPU currently in use supports the instruction set and
+`xxx_accelerated_function` that is the corresponding accelerated
+implementation.
+
+At the end of this function the function pointer points to the fastest
+available implementation and can be invoked to do the computation.

docs/sitemap.txt

@@ -27,14 +27,15 @@ index.md
 		Build-options.md
 		Subprojects.md
 		Modules.md
+			Gnome-module.md
 			i18n-module.md
 			Pkgconfig-module.md
 			Python-3-module.md
 			Qt4-module.md
 			Qt5-module.md
 			RPM-module.md
+			Simd-module.md
 			Windows-module.md
-			Gnome-module.md
 		Java.md
 		Vala.md
 		IDE-integration.md

mesonbuild/compilers/c.py

@@ -25,6 +25,8 @@
     msvc_buildtype_args,
     msvc_buildtype_linker_args,
     msvc_winlibs,
+    vs32_instruction_set_args,
+    vs64_instruction_set_args,
     ClangCompiler,
     Compiler,
     CompilerArgs,
@@ -810,7 +812,7 @@ class VisualStudioCCompiler(CCompiler):
     std_warn_args = ['/W3']
     std_opt_args = ['/O2']
 
-    def __init__(self, exelist, version, is_cross, exe_wrap):
+    def __init__(self, exelist, version, is_cross, exe_wrap, is_64):
         CCompiler.__init__(self, exelist, version, is_cross, exe_wrap)
         self.id = 'msvc'
         # /showIncludes is needed for build dependency tracking in Ninja
@@ -820,6 +822,7 @@ def __init__(self, exelist, version, is_cross, exe_wrap):
                           '2': ['/W3'],
                           '3': ['/W4']}
         self.base_options = ['b_pch'] # FIXME add lto, pgo and the like
+        self.is_64 = is_64
 
     # Override CCompiler.get_always_args
     def get_always_args(self):
@@ -1005,3 +1008,15 @@ def get_link_whole_for(self, args):
         if not isinstance(args, list):
             args = [args]
         return ['/WHOLEARCHIVE:' + x for x in args]
+
+    def get_instruction_set_args(self, instruction_set):
+        if self.is_64:
+            return vs64_instruction_set_args.get(instruction_set, None)
+        if self.version.split('.')[0] == '16' and instruction_set == 'avx':
+            # VS documentation says that this exists and should work, but
+            # it does not. The headers do not contain AVX intrinsics
+            # and the can not be called.
+            return None
+        return vs32_instruction_set_args.get(instruction_set, None)
+
+

mesonbuild/compilers/compilers.py

@@ -228,6 +228,43 @@ def is_library(fname):
                                                           True),
                 }
 
+gnulike_instruction_set_args = {'mmx': ['-mmmx'],
+                                'sse': ['-msse'],
+                                'sse2': ['-msse2'],
+                                'sse3': ['-msse3'],
+                                'ssse3': ['-mssse3'],
+                                'sse41': ['-msse4.1'],
+                                'sse42': ['-msse4.2'],
+                                'avx': ['-mavx'],
+                                'avx2': ['-mavx2'],
+                                'neon': ['-mfpu=neon'],
+                                }
+
+vs32_instruction_set_args = {'mmx': ['/arch:SSE'], # There does not seem to be a flag just for MMX
+                             'sse': ['/arch:SSE'],
+                             'sse2': ['/arch:SSE2'],
+                             'sse3': ['/arch:AVX'], # VS leaped from SSE2 directly to AVX.
+                             'sse41': ['/arch:AVX'],
+                             'sse42': ['/arch:AVX'],
+                             'avx': ['/arch:AVX'],
+                             'avx2': ['/arch:AVX2'],
+                             'neon': None,
+}
+
+# The 64 bit compiler defaults to /arch:avx.
+vs64_instruction_set_args = {'mmx': ['/arch:AVX'],
+                             'sse': ['/arch:AVX'],
+                             'sse2': ['/arch:AVX'],
+                             'sse3': ['/arch:AVX'],
+                             'ssse3': ['/arch:AVX'],
+                             'sse41': ['/arch:AVX'],
+                             'sse42': ['/arch:AVX'],
+                             'avx': ['/arch:AVX'],
+                             'avx2': ['/arch:AVX2'],
+                             'neon': None,
+                             }
+
+
 def sanitizer_compile_args(value):
     if value == 'none':
         return []
@@ -755,6 +792,12 @@ def get_link_whole_for(self, args):
             return []
         raise EnvironmentException('Language %s does not support linking whole archives.' % self.get_display_language())
 
+    # Compiler arguments needed to enable the given instruction set.
+    # May be [] meaning nothing needed or None meaning the given set
+    # is not supported.
+    def get_instruction_set_args(self, instruction_set):
+        return None
+
     def build_unix_rpath_args(self, build_dir, from_dir, rpath_paths, install_rpath):
         if not rpath_paths and not install_rpath:
             return []
@@ -933,6 +976,10 @@ def get_gui_app_args(self):
             return ['-mwindows']
         return []
 
+    def get_instruction_set_args(self, instruction_set):
+        return gnulike_instruction_set_args.get(instruction_set, None)
+
+
 class ClangCompiler:
     def __init__(self, clang_type):
         self.id = 'clang'
@@ -983,7 +1030,7 @@ def get_soname_args(self, prefix, shlib_name, suffix, path, soversion, is_shared
 
     def has_multi_arguments(self, args, env):
         return super().has_multi_arguments(
-            ['-Werror=unknown-warning-option'] + args,
+            ['-Werror=unknown-warning-option', '-Werror=unused-command-line-argument'] + args,
             env)
 
     def has_function(self, funcname, prefix, env, extra_args=None, dependencies=None):
@@ -1010,6 +1057,9 @@ def get_link_whole_for(self, args):
             return result
         return ['-Wl,--whole-archive'] + args + ['-Wl,--no-whole-archive']
 
+    def get_instruction_set_args(self, instruction_set):
+        return gnulike_instruction_set_args.get(instruction_set, None)
+
 
 # Tested on linux for ICC 14.0.3, 15.0.6, 16.0.4, 17.0.1
 class IntelCompiler:

mesonbuild/compilers/cpp.py

@@ -173,10 +173,10 @@ def has_multi_arguments(self, args, env):
 
 
 class VisualStudioCPPCompiler(VisualStudioCCompiler, CPPCompiler):
-    def __init__(self, exelist, version, is_cross, exe_wrap):
+    def __init__(self, exelist, version, is_cross, exe_wrap, is_64):
         self.language = 'cpp'
         CPPCompiler.__init__(self, exelist, version, is_cross, exe_wrap)
-        VisualStudioCCompiler.__init__(self, exelist, version, is_cross, exe_wrap)
+        VisualStudioCCompiler.__init__(self, exelist, version, is_cross, exe_wrap, is_64)
         self.base_options = ['b_pch'] # FIXME add lto, pgo and the like
 
     def get_options(self):

mesonbuild/environment.py

@@ -534,8 +534,12 @@ def _detect_c_or_cpp_compiler(self, lang, evar, want_cross):
                 # Visual Studio prints version number to stderr but
                 # everything else to stdout. Why? Lord only knows.
                 version = search_version(err)
+                if not err or not err.split('\n')[0]:
+                    m = 'Failed to detect MSVC compiler arch: stderr was\n{!r}'
+                    raise EnvironmentException(m.format(err))
+                is_64 = err.split('\n')[0].endswith(' x64')
                 cls = VisualStudioCCompiler if lang == 'c' else VisualStudioCPPCompiler
-                return cls(compiler, version, is_cross, exe_wrap)
+                return cls(compiler, version, is_cross, exe_wrap, is_64)
             if '(ICC)' in out:
                 # TODO: add microsoft add check OSX
                 inteltype = ICC_STANDARD

mesonbuild/interpreter.py

@@ -161,6 +161,7 @@ def __init__(self):
                              'set_quoted': self.set_quoted_method,
                              'has': self.has_method,
                              'get': self.get_method,
+                             'merge_from': self.merge_from_method,
                              })
 
     def is_used(self):
@@ -221,6 +222,16 @@ def get(self, name):
     def keys(self):
         return self.held_object.values.keys()
 
+    def merge_from_method(self, args, kwargs):
+        if len(args) != 1:
+            raise InterpreterException('Merge_from takes one positional argument.')
+        from_object = args[0]
+        if not isinstance(from_object, ConfigurationDataHolder):
+            raise InterpreterException('Merge_from argument must be a configuration data object.')
+        from_object = from_object.held_object
+        for k, v in from_object.values.items():
+            self.held_object.values[k] = v
+
 # Interpreter objects can not be pickled so we must have
 # these wrappers.
 
@@ -1479,6 +1490,10 @@ def func_import(self, node, args, kwargs):
         if len(args) != 1:
             raise InvalidCode('Import takes one argument.')
         modname = args[0]
+        if modname.startswith('unstable-'):
+            plainname = modname.split('-', 1)[1]
+            mlog.warning('Module %s has no backwards or forwards compatibility and might not exist in future releases.' % modname)
+            modname = 'unstable_' + plainname
         if modname not in self.environment.coredata.modules:
             try:
                 module = importlib.import_module('mesonbuild.modules.' + modname)

mesonbuild/modules/unstable_simd.py

@@ -0,0 +1,72 @@
+# Copyright 2017 The Meson development team
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .. import mesonlib, compilers, mlog
+
+from . import ExtensionModule
+
+class SimdModule(ExtensionModule):
+
+    def __init__(self):
+        super().__init__()
+        self.snippets.add('check')
+        # FIXME add Altivec and AVX512.
+        self.isets = ('mmx',
+                      'sse',
+                      'sse2',
+                      'sse3',
+                      'ssse3',
+                      'sse41',
+                      'sse42',
+                      'avx',
+                      'avx2',
+                      'neon',
+                      )
+
+    def check(self, interpreter, state, args, kwargs):
+        result = []
+        if len(args) != 1:
+            raise mesonlib.MesonException('Check requires one argument, a name prefix for checks.')
+        prefix = args[0]
+        if not isinstance(prefix, str):
+            raise mesonlib.MesonException('Argument must be a string.')
+        if 'compiler' not in kwargs:
+            raise mesonlib.MesonException('Must specify compiler keyword')
+        compiler = kwargs['compiler'].compiler
+        if not isinstance(compiler, compilers.compilers.Compiler):
+            raise mesonlib.MesonException('Compiler argument must be a compiler object.')
+        cdata = interpreter.func_configuration_data(None, [], {})
+        conf = cdata.held_object
+        for iset in self.isets:
+            if iset not in kwargs:
+                continue
+            iset_fname = kwargs[iset] # Migth also be an array or Files. static_library will validate.
+            args = compiler.get_instruction_set_args(iset)
+            if args is None:
+                mlog.log('Compiler supports %s:' % iset, mlog.red('NO'))
+                continue
+            if len(args) > 0:
+                if not compiler.has_multi_arguments(args, state.environment):
+                    mlog.log('Compiler supports %s:' % iset, mlog.red('NO'))
+                    continue
+            mlog.log('Compiler supports %s:' % iset, mlog.green('YES'))
+            conf.values['HAVE_' + iset.upper()] = ('1', 'Compiler supports %s.' % iset)
+            libname = prefix + '_' + iset
+            lib_kwargs = {'sources': iset_fname,
+                          compiler.get_language() + '_args': args}
+            result.append(interpreter.func_static_lib(None, [libname], lib_kwargs))
+        return [result, cdata]
+
+def initialize():
+    return SimdModule()