-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
modules: The 'features' module #11307
base: master
Are you sure you want to change the base?
Conversation
63ddf0d
to
a3fa20b
Compare
6f686ff
to
5257b40
Compare
Looks promising, thanks @seiko2plus! One question I have is what CPU features are of interest here aside from SIMD ones? From the discussion at #11033 I thought you were going to try and fit this into the existing For others, @seiko2plus posted this test program to try this out earlier (here): project('test', 'c')
feature = import('feature')
cpu_features = feature.cpu_features()
targets = [['AVX2_FMA3', [cpu_features['AVX2'], cpu_features['FMA3']]],
['AVX512_COMMON', cpu_features['AVX512_COMMON']],
['AVX512_SKX', cpu_features['AVX512_SKX']]]
foreach target : targets
t = feature.test(target[1])
if t[0]
message('target name:', target[0])
data = t[1]
message('Implicit features:', data['implicit'])
message('Compiler arguments:', data['args'])
message('Headers:', data['headers'])
message('Need to detect on runtime:', data['detect'])
message('Supported #definitions:', data['defines'])
message('Unsupported #definitions:', data['undefines'])
message('Compiler support this target by ', data['support'])
message('========================================')
endif
endforeach @seiko2plus this diff fixes the minor issue I had with feature detection here: --- a/mesonbuild/modules/feature/x86_features.py
+++ b/mesonbuild/modules/feature/x86_features.py
@@ -42,7 +42,7 @@ if T.TYPE_CHECKING:
def test_code_main(code: str) -> str:
return dedent(f'''\
- int main(int, char **argv)
+ int main(int unused, char **argv)
{{
char *src = argv[1];
{dedent(code)} |
The current meson SIMD module lacks flexibility and modifies it to support our needs in a way to provide static and dynamic dispatching (baseline/dispatch) that fits "any project needs" will require more effort than I expected. So I just split the work into two modules rather than a single module covering all the operations, the first module is the module 'feature' which provides what is necessary to be coded by python, and the second module 'compiler_opt' will be left for future step basically it will require to convert the following two meson scripts (not completed yet) into python module:
We still can modify the current meson SIMD module to allow only the use of the CPU features in this module without any extra modifications but we're not going to use it in NumPy, for many considerations are hard to explain in one comment.
Do you mean aside from SIMD features? Any CPU features in general, but main concern features that will be required for optimizing certain operations. For example, POPCNT and LZCNT. But if you mean aside from the current meson SIMD module? Flexibility is the main reason, imagine if a project wants to modify or add a new feature or a new architecture. What if there is a necessity to make fast or custom modifications to an existing feature? some examples: So assuming adding feature X implies feature
|
I think that'd be a vendorable package containing only
I meant SIMD features in general. Which is still the main use case here I think. But yes,
Nice examples you have given here already, thanks. |
It certainly would not be the only meson-using project that used a subproject containing sources which are intended strictly for use as a vendorable package, together with the |
qggq |
f5bd2ac
to
bed3012
Compare
629e17f
to
650e7e8
Compare
This module aims to provide a more dynamic way to control common features between compilers that would generally require special headers, arguments, or compile-time test cases, quite ideal to manage CPU features and solve compatibility issues.
and brings X86 CPU features
- Removes CPU features implmentation and move them NumPy meson scripts - Removes attr 'header', since it can be handeled within a configuration file - re-implment method test, make it more simple - Implment method `multi_targets` Handels multi_targets via meson script was pretty anoyning from python make it much simpler
650e7e8
to
74da606
Compare
@@ -50,6 +50,7 @@ | |||
from unittests.subprojectscommandtests import SubprojectsCommandTests | |||
from unittests.windowstests import WindowsTests | |||
from unittests.platformagnostictests import PlatformAgnosticTests | |||
from unittests.featurestests import FeaturesTests |
Check notice
Code scanning / CodeQL
Unused import Note
e013e8a
to
7f66aff
Compare
@eli-schwartz, Hi Eli, I am curious if there a method to directly generate documentation for the supported methods from the Python scripts themselves. |
This vendors the current state of mesonbuild/meson#11307, which is the "feature module" for the multi-target build support that NumPy needs to build its SIMD support (by Sayed Adel). That PR is, as vendored, based on top of the state of the Meson master branch in commit 9d88d0d5c, which is a dev commit in between the 1.2.0 and 1.2.1 tags from 18 July 2023. No modifications have been made to any of the vendored files.
Historically, no. All the documentation is either handwritten or produced as structured yaml in docs/yaml/. We do have a way to generate module documentation from structured yaml, but it's currently disabled because those were never properly ported (doing this would be nice, but also probably too complicated to hold up the 'feature' module over). |
We need this in order to add the "feature" module for SIMD support, which is at mesonbuild/meson#11307.
Thank you for the clarification, its seem I will go for handwritten but I just realized different formats when it comes to methods signatures and module objects. Could you take a look at current handwritten format that this module use, and suggest format for documenting module objects? |
We need this in order to add the "feature" module for SIMD support, which is at mesonbuild/meson#11307.
You could use the python module as partial inspiration: https://mesonbuild.com/Python-module.html#python_installation-object It is a subheader with a list of methods in further subheaders, which aren't substantially different from module functions. |
A bit of vague high-level commentary:
|
I think this module would also probably merit a tutorial walkthrough, e.g. For bonus points it would compare and contrast both this module and the existing (also unstable) SIMD module. This module is a lot more powerful, so it would be interesting to see how rewriting a SIMD module example would look here -- and by the same token, offer guidance on migrating over to the new module so that we could perhaps deprecate the SIMD module. |
We need this in order to add the "feature" module for SIMD support, which is at mesonbuild/meson#11307.
Also see #11885 which also proposes a module/features module |
I don't know that it's import, I can always rename the module in 11885 if we ever have agreement on adding that module, it could be named |
I'd be in favor of renaming it |
@@ -0,0 +1 @@ | |||
../init_features |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
symlinks will be problematic on Windows platform.
Seems like this is pretty close to getting merged. Would be great to get in for 1.6. |
This module aims to provide a more dynamic way to control common
features between compilers that would generally require special
headers, arguments, or compile-time test cases, quite ideal
to manage CPU features and solve compatibility issues.
TODO:
[ ] add x86 features[ ] add Arm features[ ] add PPC64 features[ ] add IBMZ features