Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

language_level and Cython.Distutils.extension.Extension #2995

Open
kdschlosser opened this issue Jun 12, 2019 · 0 comments

Comments

Projects
None yet
1 participant
@kdschlosser
Copy link

commented Jun 12, 2019

I do not know if this would fall into the realm of a bug or maybe just something of a convenience feature that might be added. That can be determined by the maintainers/authors.

Background of language_level (for those that do not know.

To set the language_level adding
#cython: language_level={PYTHON MAJOR VERSION NUMBER}
to the top of the pyx file will work if only compiling specifically for that python version.

if you want to programmatically set the language level this can be done.
cythonize(EXTENSIONS, compiler_directives={'language_level' : "{PYTHON MAJOR VERSION NUMBER}"))

In Cython there are 2 convenience classes Cython.Distutils.extension.Extension and Cython.Distutils.build_ext.build_ext. While there are great for integrating into the extension building portions of distutils/setuptools it does not allow for setting the language level without duplicating the code for the build_ext class and adding the language level support to it.

I thought that the cython_directives in the Extension class would be used to do this exact thing.. It does not work if language_level is added to it. There is no option for the build_ext or the Extension for "compiler_directives"

I would like to request that the build_ext class gets extended to add the user option "language_level" this option can be set by the user. and if the option is not set by the user then in the finalize_options method of the class do an equality check for None. if it is None then set it to the corresponding python version.

Here is an example of what the code would look like in Cython.Distutils.build_ext

import sys

if 'setuptools' in sys.modules:
    try:
        from setuptools.command.build_ext import build_ext as _build_ext
    except ImportError:
        # We may be in the process of importing setuptools, which tries
        # to import this.
        from distutils.command.build_ext import build_ext as _build_ext
else:
    from distutils.command.build_ext import build_ext as _build_ext


class new_build_ext(_build_ext, object):
    user_options = [
        (
            'language-level=',
            None,
            'Sets the python language syntax to use "2", "3", "3str".' 
        )
    ] + _build_ext.user_options

    def initialize_options(self):
        self.language_level = None
        super(new_build_ext, self).initialize_options()
        
    def finalize_options(self):
        if self.distribution.ext_modules:
            if self.language_level is None:
                self.language_level = str(sys.version_info[0])

            assert self.language_level in ('2', '3', '3str'), 'Incorrect Cython language level ("{0}")'.format(self.language_level)
           
            nthreads = getattr(self, 'parallel', None)  # -j option in Py3.5+
            nthreads = int(nthreads) if nthreads else None
            
            from Cython.Build.Dependencies import cythonize
            
            self.distribution.ext_modules[:] = cythonize(
                self.distribution.ext_modules, 
                nthreads=nthreads, 
                force=self.force,
                compiler_directives=dict(language_level=self.language_level)
            )
        super(new_build_ext, self).finalize_options()

# This will become new_build_ext in the future.
from .old_build_ext import old_build_ext as build_ext

I do not know if there is a way to set the language level using build_ext or Extension. I did a lot of digging and searching but came up empty handed. This is one of the reasons I did not make a PR for this and also because I do not know all of the inner workings of Cython and there could be use cases where the above would not work.

On a tangent but still related to build_ext.

setuptools has a nice feature called setup_requires. which only installs a package for the setup programs use and gets deleted if the setup program was run from pip or easy_install. it does not install it into the users site-packages folder.

If we wanted to use this feature with Cython the code below explains how to do this. This is only going to work if you use Extension from either setuptools or from distutils. otherwise you would need to add the extensions to ext_modules as outlined in the comments.

you need to subclass setuptools.command.build.build and override the initialize_options method.

It would look similar to the code below.

import setuptools
import setuptools.extension
import setuptools.command.build

class build(setuptools.command.build.build):

    def initialize_options(self):
        import Cython.Distutils.build_ext

        self.distribution.cmdclass['build_ext'] = Cython.Distutils.build_ext.build_ext

        # if you wanted to use the Extension class from Cython
        # from Cython.Distutils.extension import Extension
        # ext = Extension(....)
        # self.distribution.ext_modules = [ext]

        super(build, self).initialize_options()

# and the call to setup

setuptools.setup(
    name='some package name',
    setup_requires=['Cython'],
    ext_modules=[setuptools.extension.Extension(.....)],
    cmdclass=dict(
        build=build,
        # this is done in build.initialize_options 
        # build_ext=Cython.Distutils.build_ext.build_ext
    )
)

I am hoping that information along these lines with a code example can get added to the documentation. Use case example would be installing a package from pypi where an extension gets compiled using Cython. After the package has been installed there is no further use of Cython after that.

All examples of using Cython are all under the assumption that Cython is already installed. There is no documentation on how to run a setup program without it being installed.

All code blocks in this issue are pseudo code and have not been tested, they are simply for example. There could be typos and things I may not have accounted for.

Thanks again for taking the time to read/review this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.