Allow users to target Py_LIMITED_API via python's extension_module #11745

amcn · 2023-04-27T14:05:31Z

What does this PR do?

This PR adds the ability for a user to target the [Python Limited API] directly in the build system by supplying a new, optional keyword argument to the extension_module function on the python_installation object. For example:

pymod = import('python')
pyinst = pymod.find_installation()
ext_mod = pyinst.extension_module('my_module',
  'my_module_source.c',
  limited_api: '3.6',
)

Under the hood, Meson takes the '3.6' argument and converts it to a hex value corresponding to the equivalent PY_VERSION_HEX. It then sets the value of the Py_LIMITED_API preprocessor directive to this value.

There is an extra complexity on Windows: when targeting Python's limited API on Windows one must link to python3.dll and not python3.X.dll. In order to allow enable this, the dependency function on python_installation has gained the limited_api boolean keyword argument. When supplied, this causes the dependency to link to python3.lib and python3.dll on Windows instead of the unstable API versions. This new kwarg only has an effect on Windows.

So a full example, including Windows:

pymod = import('python')
pyinst = pymod.find_installation()
ext_mod = pyinst.extension_module('my_module',
  'my_module_source.c',
  limited_api: '3.6',
  dependencies: [ pyinst.dependency(limited_api: true), other_deps ]
)

Motivation

Targeting the Python limited API allows an extension to be loaded in any version of the Python interpreter that supports that API, simplifying compatibility.

Notes

I had to teach the testsuite how to match for installed python modules which target the limited API. The testsuite file matching works by checking the file extension and limited API modules have a different extension.

Let me know what you think.

amcn · 2023-04-27T14:35:18Z

Fixed CI test issue by removing project's meson_version kwarg.

amcn · 2023-04-27T14:36:55Z

For any future reviewers, there is a discussion here about this, which might be useful for context.

dnicolodi · 2023-04-27T15:32:04Z

Thanks for working on this, it is very welcome addition. However, I think that requiring to manually specify the dependency to the Python shared library is a step backward. Since quite a few meson versions (at least since 0.63.3) there is no need to add the dependency explicitly: Meson inspect the Python interpreter and determines if and how the Python shared library should be linked. I think the support for limited_api should be integrated with that and do the right thing automatically, namely pick the right DLL to link to on Windows.

amcn · 2023-04-27T17:01:01Z

Thanks for working on this, it is very welcome addition. However, I think that requiring to manually specify the dependency to the Python shared library is a step backward. Since quite a few meson versions (at least since 0.63.3) there is no need to add the dependency explicitly: Meson inspect the Python interpreter and determines if and how the Python shared library should be linked. I think the support for limited_api should be integrated with that and do the right thing automatically, namely pick the right DLL to link to on Windows.

I share your opinion that this feels odd to have to manually specify the dependency. I wasn't sure how else to do it without having the logic spill into the lower levels of Meson. I had considered adding that logic in but figured I would try this solution first.

I will work to remove this requirement to specify the dependency directly. Thank you for your feedback.

eli-schwartz · 2023-04-27T14:48:16Z

mesonbuild/modules/python.py

+
+        major = int(decimal_match.group(1))
+        minor = int(decimal_match.group(2))
+        decimal_version = f'{major}.{minor}'


Isn't the decimal version the initial value we require? We don't need to recalculate this or return it.

(That is to say, assuming we actually validate it fully.)

This has been removed. I was returned decimal_version here as a cleaned up version of the input string. If a user supplied " 03.4 " then decimal_version would be '3.4'. This was then used in an error message. This logic has been simplified.

eli-schwartz · 2023-04-27T20:46:05Z

mesonbuild/modules/python.py


        if 'gnu_symbol_visibility' not in kwargs and \
                (self.is_pypy or mesonlib.version_compare(self.version, '>=3.9')):
            kwargs['gnu_symbol_visibility'] = 'inlineshidden'

        return self.interpreter.func_shared_module(None, args, kwargs)

+    def _parse_limited_api_version(self, limited_api: str) -> (str, str):
+
+        decimal_match = limited_api_decimal_format.match(limited_api)


AFAICT there is no convenient way to handle this without regex, oh well...

But we should still use fullmatch here. Without that, it's valid to pass limited_api: '3.9 all the way'.

This uses fullmatch now.

eli-schwartz · 2023-04-27T20:56:11Z

mesonbuild/modules/python.py

@@ -59,8 +61,11 @@ class FindInstallationKw(ExtractRequired):
        modules: T.List[str]
        pure: T.Optional[bool]

+limited_api_decimal_format = re.compile(r'\b0*(\d)\.(\d+)\b')


And if we use fullmatch, we don't need to bother with \b either.

Also while we're here, please avoid using \d.

>>> re.match(r'\d', '𑶢') <re.Match object; span=(0, 1), match='𑶢'> >>> '𑶢'.isdigit() True >>> int('𑶢') 2

And also,

>>> _parse_limited_api_version('𑶣.𑶨') ('3.8', '0x03080000')

Use exactly what we mean and want -- [0-9], no more, no less.

I have updated the regex to be more explicit (avoiding \d) and getting rid of \b.

amcn · 2023-04-29T18:04:08Z

I have removed the limited_api kwarg from python_installation.dependency. Building on Windows now no longer works because the Python dependency on Windows links to python3.x.lib/dll instead of python3.lib/dll.

I am currently working on a solution but I would like to get a sense of if my intended solution is a good one or if there is a simpler way of doing things: my solution is to make a copy of the supplied Python dependency object, modify the link_args of this copy to instead link to the limited api lib and to then add this copy to the link deps of the extension module.

This seems like a hack to me, but I'm not sure how else to go about it?

@eli-schwartz mentioned during the earlier discussion that this functionality could be a module option. I wasn't sure what that was until recently. If I were to make this functionality a module option would that mean that all Python installation/dependency objects found via a module with this option enabled would instead link to the limited API? Would that prevent users from declaring two different extension_module, one targeting the limited api and one targeting the full api?

It's fun hacking on this but I hope all these questions are not a burden. I apologize if so.

amcn · 2023-05-29T10:04:57Z

Any updates on this?

I've rebased this PR against upstream master and will rework this functionality however the maintainers wish in order to get it ready. Feedback very welcome.

tristan957 · 2023-06-07T21:10:01Z

I have removed the limited_api kwarg from python_installation.dependency. Building on Windows now no longer works because the Python dependency on Windows links to python3.x.lib/dll instead of python3.lib/dll.

See #11851

tristan957 · 2023-06-07T21:10:20Z

@eli-schwartz do you have time to come back and review this?

eli-schwartz · 2023-06-07T21:47:26Z

Getting back to this:

However, I think that requiring to manually specify the dependency to the Python shared library is a step backward. Since quite a few meson versions (at least since 0.63.3) there is no need to add the dependency explicitly: Meson inspect the Python interpreter and determines if and how the Python shared library should be linked. I think the support for limited_api should be integrated with that and do the right thing automatically, namely pick the right DLL to link to on Windows.

I share your opinion that this feels odd to have to manually specify the dependency. I wasn't sure how else to do it without having the logic spill into the lower levels of Meson. I had considered adding that logic in but figured I would try this solution first.

I am currently working on a solution but I would like to get a sense of if my intended solution is a good one or if there is a simpler way of doing things: my solution is to make a copy of the supplied Python dependency object, modify the link_args of this copy to instead link to the limited api lib and to then add this copy to the link deps of the extension module.

I wonder if maybe instead, we can add the limited_api kwarg to py3.dependency() (and add it as part of the cache key in get_dep_identifier)? Then, also add it as a kwarg to extension_module and have that perform the automatic dependency lookup by passing that kwarg to _dependency_method_impl.

We don't have to handle the case where users manually passed their own dependency object to extension_module -- because if they have done so, they are probably trying to be very exact with what they want, or else they should simply not do that for new functionality. We don't worry about the possibility that they get the embed kwarg wrong, either.

eli-schwartz · 2023-06-07T21:53:51Z

@eli-schwartz mentioned during the earlier discussion that this functionality could be a module option. I wasn't sure what that was until recently. If I were to make this functionality a module option would that mean that all Python installation/dependency objects found via a module with this option enabled would instead link to the limited API? Would that prevent users from declaring two different extension_module, one targeting the limited api and one targeting the full api?

My thought here is that we may want a module option to suppress use of the limited_api, that is to say, modules which don't say they use the limited API would continue to not use it -- but modules which can use the limited API, default to using it and allow disabling this by setting a meson option like -Dpython.allow_limited_api=false.

We can't opt modules in to using the limited API if they don't support it. We can opt modules out if they do support it, and this can result in more optimized code at the cost of, well, using more than just the limited API. :)

...

Anyways, this idea can be ignored for an MVP and added later on, if it's too complicated.

tristan957 · 2023-06-13T15:09:21Z

@bruchar1 you may be interested in testing this PR

amcn · 2023-06-13T15:23:21Z

Apologies for not responding sooner, work has been busy recently. I admit that it's been a while since I looked at this PR, so I will re-acquaint myself with its changes and give my response to recent comments and feedback as soon as possible.

My thought here is that we may want a module option to suppress use of the limited_api, that is to say, modules which don't say they use the limited API would continue to not use it -- but modules which can use the limited API, default to using it and allow disabling this by setting a meson option like -Dpython.allow_limited_api=false.

We can't opt modules in to using the limited API if they don't support it. We can opt modules out if they do support it, and this can result in more optimized code at the cost of, well, using more than just the limited API. :)

Forgive me if I'm wrong here but would this mean that, if not explicitly disabled via this module option, the limited API kwarg would always be set, either to some default value or to whatever the user supplied, and therefore Py_LIMITED_API would always be defined?

Is that right?

bruchar1 · 2023-06-13T15:50:21Z

mesonbuild/modules/python.py

+
+        if limited_api_version != '':
+
+            has_cython_files = any(source_file.endswith('.pyx') for source_file in args)


source_file is not necessarily a str. It could be a File, a CustomTarget, etc. You cannot use .endswith directly on it.

Thanks for finding this!

I'm prototyping a fix to this and I am wondering if something like this would get the job done:

def check_for_cython_files(args): for arg in args: if isinstance(arg, mesonlib.File) and arg.endswith('.pyx'): return True elif isinstance(arg, build.BuildTarget) and check_for_cython_files(arg.get_sources()): return True return False

Although maybe there exists functionality already for doing something like this that I'm not aware of?

I admit that the Cython support for this is considerably less well tested currently than raw C/C++. Cython's support for the Python Limited API is currently limited to the upcoming 3.0 release, which I wasn't able to find packaged anywhere that allowed easy testing. It might therefore be a good idea to just remove this functionality until this version of Cython is more widely available. I have no strong feelings either way, and will do whatever is amenable to maintainers.

I think you want get_outputs, actually. A BuildTarget won't be supported at all here -- those are executable() or library() return values, and get_sources would be the inputs to the executable, rather than the inputs to py.extension_module. The concern here is CustomTarget instead, which might be used to produce .c or .pyx files at build time. You do still have to check for str.endswith as well.

It may of course not be worth making this check at all -- what's the outcome of a failure here?

It may of course not be worth making this check at all -- what's the outcome of a failure here?

The check exists to detect if we have any cython files as input so that we can then check to see if the version of Cython supports the limited API. If it doesn't we can then emit a warning. So the outcome of failure here (defined as failing to correctly detect the presence of cython files in the input to this extension module) would mean that a warning is not emitted when it should be.

Is the value of this warning worth the complexity of this task?

Let's just say that I would in no way consider this any kind of blocker, especially if it's possible to get a false positive.

eli-schwartz · 2023-06-13T15:57:50Z

Pseudocode:

kwargval = get_function_kwarg('limited_api')
moduleval = get_cli_option('allow_limited_api')

if moduleval is True and kwargval is not None:
    finalval: str = kwargval
else:
    finalval = None

the limited API kwarg would always be set, either to some default value or to whatever the user supplied, and therefore Py_LIMITED_API would always be defined?

So the default value would be "do not define Py_LIMITED_API"

bruchar1 · 2023-06-13T16:13:15Z

@bruchar1 you may be interested in testing this PR

Beside the bug I mentionned with cython detection, it seems to work with my cffi project on Windows.

Should .abi3 suffix be added to the generated extension module?

It would be great if limited api could be automatically detected, at least for cffi extensions...
I wonder if a kind of compiler.get_define() call could be performed internally for that?

eli-schwartz · 2023-06-13T17:30:12Z

Should .abi3 suffix be added to the generated extension module?

Is that not what limited_api_suffix already does?

eli-schwartz · 2023-06-13T17:33:40Z

It would be great if limited api could be automatically detected, at least for cffi extensions...

No. This would be very complicated and then only work for source files checked into git which also trigger full meson reconfiguration whenever the source file was edited.

I wonder if a kind of compiler.get_define() call could be performed internally for that?

How would this work? Usually get_define tries to #include a header.

bruchar1 · 2023-06-13T17:37:45Z

Should .abi3 suffix be added to the generated extension module?

Is that not what limited_api_suffix already does?

Oh. The limited api suffix on Windows is .pyd. So yes, this is all right finally.

mattip · 2023-07-19T05:59:20Z

mesonbuild/dependencies/python.py

@@ -43,6 +43,7 @@ class PythonIntrospectionDict(TypedDict):
        paths: T.Dict[str, str]
        platform: str
        suffix: str
+        limited_api_suffix: str


The documentation calls this limited_api, here it is limited_api_suffix. They should be aligned.

This is part of the introspection data gathered from the interpreter the extension is being built for, it does not indicate whether the extension is built for the limited API or not. I think it is fine as it is.

mattip · 2023-07-19T06:00:25Z

mesonbuild/dependencies/python.py

@@ -196,7 +198,7 @@ def __init__(self, name: str, environment: 'Environment',
        if self.link_libpython:
            # link args
            if mesonlib.is_windows():
-                self.find_libpy_windows(environment)
+                self.find_libpy_windows(environment, False)


can you use the keyword variant for clarity: limited_api=False

mattip · 2023-07-19T06:02:14Z

mesonbuild/modules/python.py

+        # https://doc.pypy.org/en/latest/cpython_differences.html#permitted-abi-tags-in-extensions
+        if self.is_pypy:
+            self.limited_api_suffix = self.suffix
+


Nice! I came here to mention this and it was already done.

mattip

This looks good and solves some thorny problems. Thanks! A couple of meaningless comments, mostly about why meson needs to deal with import libs at all when using MSVC.

mattip · 2023-08-03T10:53:31Z

mesonbuild/dependencies/python.py

@@ -277,6 +279,8 @@ def get_windows_link_args(self) -> T.Optional[T.List[str]]:
                    else:
                        libpath = Path(f'python{vernum}.dll')
                else:
+                    if limited_api:
+                        vernum = vernum[0]


This will work until the major version goes above 9. Nit: you could use verdot.split(".")[0] instead, but probably other things will break before this becomes a problem.

mattip · 2023-08-03T11:03:16Z

mesonbuild/modules/python.py

+            # When compiled under MSVC, Python's PC/pyconfig.h forcibly inserts pythonMAJOR.MINOR.lib
+            # into the linker path when not running in debug mode via a series #pragma comment(lib, "")
+            # directives. We manually override these here as this interferes with the intended
+            # use of the 'limited_api' kwarg


It is a shame you need this. I think the intention of the header was deal with linking the correct import lib, and meson should ideally not specify any link arguments to deal with the import lib at all. But it seems the header does not properly link to the "limited api + debug" import lib. I opened python/cpython#107585 to clarify that

Ideally meson would read a sysconfig setting specifying which import lib to link to, and use that. Doing this via MSVC specific pragmas is painful, weird, and bad, for a variety of reasons but perhaps the simplest and most persuasive one is that it cripples meson's ability to notice a bug in CPython and add a quirk like this to work around the problem.

You should see the crimes against computing that pybind11 does to allow building a python extension with debug symbols for user code without forcing to link against the debug version of python311.dll -- meson simply spawns a scary warning if you try to mix them.

Hot take: the computing industry would be better off if that MSVC pragma had not been invented.

I also regret that this was necessary but it does seem that CPython's pyconfig.h is missing a bit of logic here.

Back when I was starting this work I recall finding a PR in CPython which aimed to fix the logic in PC/pyconfig.h, allowing a correct and peaceful coexistence between the Py_LIMITED_API and _DEBUG macros, but after searching for it again just now I can't seem to find it. That's frustrating: if I'm unable to find it again then I will work on a fix for CPython.

That being said, even if a fix makes it into CPython, I think Meson should probably still aim to support users with older versions of CPython that are already in the wild, right?

mattip · 2023-08-03T11:13:02Z

mesonbuild/scripts/python_info.py

+if sys.version_info >= (3, 2):
+    try:
+        from importlib.machinery import EXTENSION_SUFFIXES
+        limited_api_suffix = EXTENSION_SUFFIXES[1]


There is a new API design for sysconfig, I commented on the compile-specific issue

amcn · 2023-08-07T11:17:07Z

Pseudocode:
kwargval = get_function_kwarg('limited_api')
moduleval = get_cli_option('allow_limited_api')

if moduleval is True and kwargval is not None:
    finalval: str = kwargval
else:
    finalval = None
the limited API kwarg would always be set, either to some default value or to whatever the user supplied, and therefore Py_LIMITED_API would always be defined?

So the default value would be "do not define Py_LIMITED_API"

Oof, I was looking at what else needs to be addressed and realised that I haven't yet addressed this. I am working on this now and have two questions:

Should the default value of python.allow_limited_api be True or False?
What should Meson do when the user has set the value of python.allow_limited_api to False and the user has also given us a limited_api kwarg in an extension_module? I imagine either a warning, or an error, but I'm not sure what would be considered more "mesonic".

eli-schwartz · 2023-08-07T12:43:37Z

True
Silently ignore it. The meson.build file says that the extension_module is compatible with and intended to work well with the limited API, but the user has said "I'm locally building and installing, I don't plan to upload to PyPI and I'm going to rebuild it if I update my python version, so build me an optimized module that uses unstable, faster APIs with performance inlining".

amcn · 2023-08-07T13:47:23Z

True

Silently ignore it. The meson.build file says that the extension_module is compatible with and intended to work well with the limited API, but the user has said "I'm locally building and installing, I don't plan to upload to PyPI and I'm going to rebuild it if I update my python version, so build me an optimized module that uses unstable, faster APIs with performance inlining".

Thanks for your feedback. This has been implemented and I've pushed the change up to this PR. Let me know what you think.

eli-schwartz · 2023-08-07T15:45:44Z

mesonbuild/modules/python.py

+            # into the linker path when not running in debug mode via a series #pragma comment(lib, "")
+            # directives. We manually override these here as this interferes with the intended
+            # use of the 'limited_api' kwarg
+            c_compiler = detect_c_compiler(self.interpreter.environment, MachineChoice.BUILD)


Instead of detect_c_compiler we should assume the user has specified one via the project languages or add_languages(), and look it up in environment.coredata.compilers[for_machine].

Implemented, thanks for this feedback in particular. I remember being unsure if this was a reliable way to get this information or not.

eli-schwartz · 2023-08-07T16:01:56Z

mesonbuild/modules/python.py

+            new_c_args = mesonlib.extract_as_list(kwargs, 'c_args')
+            new_c_args.append(f'-DPy_LIMITED_API={limited_api_version_hex}')
+            kwargs['c_args'] = new_c_args


We should try appending it for cpp_args too.

Implemented, thanks for the feedback.

eli-schwartz · 2023-08-07T16:09:17Z

mesonbuild/modules/python.py

+            if mesonlib.version_compare(limited_api_version, '<3.2'):
+                raise InvalidArguments(f'Python Limited API version invalid: {limited_api_version} (must be greater than 3.2)')
+            if mesonlib.version_compare(limited_api_version, '>' + pydep.version):
+                raise InvalidArguments(f'Python Limited API version too high: {limited_api_version} (detected {pydep.version})')


I'd probably move this into the converter function.

Implemented, thanks again for the feedback.

eli-schwartz · 2023-08-07T17:19:42Z

mesonbuild/modules/python.py

+            # use of the 'limited_api' kwarg
+            for_machine = self.interpreter.machine_from_native_kwarg(kwargs)
+            compilers = self.interpreter.environment.coredata.compilers[for_machine]
+            if any(compiler.get_id() == 'msvc' for _, compiler in compilers.items()):


Suggested change

if any(compiler.get_id() == 'msvc' for _, compiler in compilers.items()):

if any(compiler.get_id() == 'msvc' for compiler in compilers.values()):

Thanks, implemented!

eli-schwartz · 2023-08-07T22:00:49Z

mesonbuild/modules/python.py

@@ -62,8 +62,7 @@ class ExtensionModuleKw(SharedModuleKw):

        subdir: NotRequired[T.Optional[str]]

-
-mod_kwargs = {'subdir'}
+mod_kwargs = {'subdir', 'limited_api'}


Can you leave the empty line in there?

Sure, done.

eli-schwartz · 2023-08-07T22:20:03Z

mesonbuild/modules/python.py

+            if any(compiler.get_id() == 'msvc' for compiler in compilers.values()):
+                pydep_copy = copy.copy(pydep)
+                pydep_copy.find_libpy_windows(self.env, limited_api=True)
+                if not pydep_copy.found():
+                    raise mesonlib.MesonException('Python dependency supporting limited API not found')
+
+                new_deps.remove(pydep)
+                new_deps.append(pydep_copy)


I was trying to think whether there was value to making this part of py_inst.dependency(limited: ...) but after thinking about it a bit more I think the way you've done it is fine.

I think it's fairly unlikely that anyone will ever want to touch the limited API for reasons other than building an extension_module, so we do not need to make this directly exposed.

eli-schwartz · 2023-08-07T22:27:44Z

docs/markdown/Python-module.md

@@ -101,6 +101,11 @@ the addition of the following:
  `/usr/lib/site-packages`. When subdir is passed to this method,
  it will be appended to that location. This keyword argument is
  mutually exclusive with `install_dir`
+- `limited_api`: *since 1.3.0* A string containing the Python version
+  of the [Py_LIMITED_API](C-https://docs.python.org/3/c-api/stable.html) that


This link looks odd, is the C- part a typo?

Ah yes, this is definitely a typo on my part. I've fixed this.

eli-schwartz · 2023-08-08T03:09:27Z

mesonbuild/scripts/python_info.py

@@ -76,4 +90,5 @@ def links_against_libpython():
  'is_venv': sys.prefix != variables['base_prefix'],
  'link_libpython': links_against_libpython(),
  'suffix': suffix,
+  'limited_api_suffix': limited_api_suffix


Please end this with a comma :D that way the next person to update this doesn't have to change two lines.

Although we are dumping json, this file isn't actually json, which means we can do sensible things like end an entry with a comma even when commas aren't mandatory -- biggest gripe against json, a trailing comma is outright illegal. :(

amcn · 2023-08-08T09:21:10Z

I had a bit of a nightmare last night about this Windows linking hack that I'm proposing in this PR and so have added an extra bit to the test case to ensure that the hack actually does what I intend: linking to the right version of the library on Windows and also not impacting future invocations of extension_module

Let me know if this is just paranoia on my part.

tristan957 · 2023-08-08T14:34:23Z

test cases/python/10 extmodule limited api disabled/meson.build

@@ -0,0 +1,10 @@
+project('Python limited api disabled', 'c',
+  default_options : ['buildtype=release', 'werror=true', 'python.allow_limited_api=False'])


If you have to do another push based on other comments. Don't use a capital F for false.

amcn · 2023-08-11T12:01:10Z

How do the Meson maintainers feel about the current state of this PR? Is there a consensus that it's ready or that it needs a bit more work before it's ready?

Apologies if I have missed any feedback that has been left so far.

eli-schwartz · 2023-08-11T12:05:27Z

I'm feeling pretty good about it and will do a final review today. Can you fix the minor typo pointed out above while you wait?

This commit adds a new keyword arg to extension_module() that enables a user to target the Python Limited API, declaring the version of the limited API that they wish to target. Two new unittests have been added to test this functionality.

amcn · 2023-08-11T12:41:38Z

I'm feeling pretty good about it and will do a final review today. Can you fix the minor typo pointed out above while you wait?

Done.

eli-schwartz · 2023-08-15T00:02:46Z

Missed reviewing this over the weekend, sorry. :D

Thanks for this fantastic work.

amcn · 2023-08-15T11:33:08Z

Many thanks to all reviewers for their feedback!

I aim to follow up on the Cython stuff when a version of Cython that supports the limited API is no longer in beta.

mattip · 2024-01-04T07:30:45Z

I aim to follow up on the Cython stuff when a version of Cython that supports the limited API is no longer in beta.

FWIW, the limited API in cython is triggered by setting -DCYTHON_LIMITED_API=1.

amcn · 2024-02-14T01:24:20Z

I aim to follow up on the Cython stuff when a version of Cython that supports the limited API is no longer in beta.

FWIW, the limited API in cython is triggered by setting -DCYTHON_LIMITED_API=1.

Thanks for the nudge, I've started to implement this now. I will push a PR soon and invite feedback.

amcn requested a review from jpakkane as a code owner April 27, 2023 14:05

amcn changed the title ~~Allow users to targe Py_LIMITED_API via python's extension_module~~ Allow users to target Py_LIMITED_API via python's extension_module Apr 27, 2023

eli-schwartz requested changes Apr 27, 2023

View reviewed changes

eli-schwartz added the modules:python Issues specific to the python module label Apr 28, 2023

amcn requested a review from eli-schwartz May 5, 2023 10:17

amcn force-pushed the python_limited_api branch 3 times, most recently from 789c15c to 8729b43 Compare May 29, 2023 10:01

bruchar1 reviewed Jun 13, 2023

View reviewed changes

eli-schwartz mentioned this pull request Jul 18, 2023

When building a python c-extension using the limited_api, windows should link with python3.lib #12008

Closed

mattip reviewed Jul 19, 2023

View reviewed changes

mattip reviewed Aug 3, 2023

View reviewed changes

mattip mentioned this pull request Aug 3, 2023

Windows + MSVC: using debug + limited API does not inject a pragma to link to python3_d.lib python/cpython#107585

Open

amcn force-pushed the python_limited_api branch from 0099efd to 5928a51 Compare August 7, 2023 13:45

eli-schwartz reviewed Aug 7, 2023

View reviewed changes

tristan957 mentioned this pull request Aug 7, 2023

modules/python: fix debug support for Python3 on Windows #6129

Closed

amcn force-pushed the python_limited_api branch 2 times, most recently from fb9780d to 93dccfd Compare August 7, 2023 17:16

eli-schwartz reviewed Aug 7, 2023

View reviewed changes

amcn force-pushed the python_limited_api branch from 93dccfd to a3ae2a6 Compare August 7, 2023 17:21

eli-schwartz reviewed Aug 7, 2023

View reviewed changes

eli-schwartz reviewed Aug 8, 2023

View reviewed changes

amcn force-pushed the python_limited_api branch 2 times, most recently from 58bc6d1 to 88ad193 Compare August 8, 2023 09:20

tristan957 reviewed Aug 8, 2023

View reviewed changes

amcn force-pushed the python_limited_api branch from 88ad193 to 86049de Compare August 11, 2023 12:38

eli-schwartz merged commit c730807 into mesonbuild:master Aug 15, 2023
35 checks passed

amcn deleted the python_limited_api branch August 15, 2023 11:33

amcn mentioned this pull request Feb 20, 2024

Cython: emit CYTHON_LIMITED_API #12882

Open


		if limited_api_version != '':

		has_cython_files = any(source_file.endswith('.pyx') for source_file in args)

	if any(compiler.get_id() == 'msvc' for _, compiler in compilers.items()):
	if any(compiler.get_id() == 'msvc' for compiler in compilers.values()):

		@@ -0,0 +1,10 @@
		project('Python limited api disabled', 'c',
		default_options : ['buildtype=release', 'werror=true', 'python.allow_limited_api=False'])

Allow users to target Py_LIMITED_API via python's extension_module #11745

Allow users to target Py_LIMITED_API via python's extension_module #11745

Conversation

amcn commented Apr 27, 2023

What does this PR do?

Motivation

Notes

amcn commented Apr 27, 2023 • edited

amcn commented Apr 27, 2023

dnicolodi commented Apr 27, 2023

amcn commented Apr 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amcn commented Apr 29, 2023

amcn commented May 29, 2023

tristan957 commented Jun 7, 2023

tristan957 commented Jun 7, 2023

eli-schwartz commented Jun 7, 2023

eli-schwartz commented Jun 7, 2023

tristan957 commented Jun 13, 2023

amcn commented Jun 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eli-schwartz commented Jun 13, 2023

bruchar1 commented Jun 13, 2023

eli-schwartz commented Jun 13, 2023

eli-schwartz commented Jun 13, 2023

bruchar1 commented Jun 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattip left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amcn commented Aug 7, 2023

eli-schwartz commented Aug 7, 2023

amcn commented Aug 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amcn commented Aug 8, 2023

Choose a reason for hiding this comment

amcn commented Aug 11, 2023

eli-schwartz commented Aug 11, 2023

amcn commented Aug 11, 2023

eli-schwartz commented Aug 15, 2023

amcn commented Aug 15, 2023

mattip commented Jan 4, 2024

amcn commented Feb 14, 2024

amcn commented Apr 27, 2023 •

edited