New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation bug: installing package without cython #1941

Open
leftys opened this Issue Oct 17, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@leftys

leftys commented Oct 17, 2017

Docs say:

Another option is to make Cython a setup dependency of your system and use Cython’s build_ext module which runs cythonize as part of the build process:

setup(
    setup_requires=[
        'cython>=0.x',
    ],
    extensions = [Extension("*", ["*.pyx"])],
    cmdclass={'build_ext': Cython.Build.build_ext},
    ...
)

But this doesn't work, because you have to import Cython before the setup_requirements get installed. Can the documentation be fixed? It would be nice to provide another example how to package cython to be installed without cython installed beforehand.

@leftys

This comment has been minimized.

Show comment
Hide comment
@scoder

This comment has been minimized.

Show comment
Hide comment
@scoder

scoder Oct 17, 2017

Contributor

It might actually Just Work without the build_ext. Since setup_requires is a setuptools option, and recent setuptools versions know about Cython modules, I wouldn't be surprised if leaving out the cmd_class fixed it.

OTOH, I generally discourage depending on Cython in the normal user build process. Instead, ship the generated .c files in the sdist. Makes it much easier to reason about the code that users are executing on their side. Cython code is "translate once, build everywhere".

Contributor

scoder commented Oct 17, 2017

It might actually Just Work without the build_ext. Since setup_requires is a setuptools option, and recent setuptools versions know about Cython modules, I wouldn't be surprised if leaving out the cmd_class fixed it.

OTOH, I generally discourage depending on Cython in the normal user build process. Instead, ship the generated .c files in the sdist. Makes it much easier to reason about the code that users are executing on their side. Cython code is "translate once, build everywhere".

@scoder scoder added the help wanted label Oct 31, 2017

@derNarr

This comment has been minimized.

Show comment
Hide comment
@derNarr

derNarr Feb 7, 2018

Since Nov. 2016 it seems to work to import the Extension class from setuptools and pass it a pyx-file. If you do not import Cython in your setup.py and defines cython as a setup requirement (setup_requires=['cython']), setuptools will figure out the rest for you. This should work with setuptools version 18.0 or higher.

Minimal example

Minimal example for setup.py:

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext

setup(
    ...
    setup_requires=['cython'],
    ext_modules=[Extension("*", ["*.pyx"])],
    cmdclass={'build_ext': build_ext}
)

Example with cython, numpy and openmp

If you need numpy headers and openmp support it becomes slightly more complicated. The best way I found is the following setup.py layout:

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext as _build_ext

# bootstrap numpy
# https://stackoverflow.com/questions/19919905/how-to-bootstrap-numpy-installation-in-setup-py
class build_ext(_build_ext):
    def finalize_options(self):
        _build_ext.finalize_options(self)
        # Prevent numpy from thinking it is still in its setup process:
        __builtins__.__NUMPY_SETUP__ = False
        import numpy
        self.include_dirs.append(numpy.get_include())

ext_modules = [
    Extension(
        "*",
        ["*.pyx"],
        extra_compile_args=['-fopenmp'],
        extra_link_args=['-fopenmp'],
    )
]

setup(
    ...
    setup_requires=['numpy', 'cython'],
    ext_modules=ext_modules,
    cmdclass={'build_ext': build_ext}
)

Alternatives

During investigating this "problem", I found this template for a setup.py, which looks interesting and seems to solve many of the suggested and similar problems:

https://github.com/Technologicat/setup-template-cython

Sources

https://github.com/douban/libmc/pull/45/files
https://stackoverflow.com/a/38057196
https://stackoverflow.com/questions/19919905/how-to-bootstrap-numpy-installation-in-setup-py

derNarr commented Feb 7, 2018

Since Nov. 2016 it seems to work to import the Extension class from setuptools and pass it a pyx-file. If you do not import Cython in your setup.py and defines cython as a setup requirement (setup_requires=['cython']), setuptools will figure out the rest for you. This should work with setuptools version 18.0 or higher.

Minimal example

Minimal example for setup.py:

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext

setup(
    ...
    setup_requires=['cython'],
    ext_modules=[Extension("*", ["*.pyx"])],
    cmdclass={'build_ext': build_ext}
)

Example with cython, numpy and openmp

If you need numpy headers and openmp support it becomes slightly more complicated. The best way I found is the following setup.py layout:

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext as _build_ext

# bootstrap numpy
# https://stackoverflow.com/questions/19919905/how-to-bootstrap-numpy-installation-in-setup-py
class build_ext(_build_ext):
    def finalize_options(self):
        _build_ext.finalize_options(self)
        # Prevent numpy from thinking it is still in its setup process:
        __builtins__.__NUMPY_SETUP__ = False
        import numpy
        self.include_dirs.append(numpy.get_include())

ext_modules = [
    Extension(
        "*",
        ["*.pyx"],
        extra_compile_args=['-fopenmp'],
        extra_link_args=['-fopenmp'],
    )
]

setup(
    ...
    setup_requires=['numpy', 'cython'],
    ext_modules=ext_modules,
    cmdclass={'build_ext': build_ext}
)

Alternatives

During investigating this "problem", I found this template for a setup.py, which looks interesting and seems to solve many of the suggested and similar problems:

https://github.com/Technologicat/setup-template-cython

Sources

https://github.com/douban/libmc/pull/45/files
https://stackoverflow.com/a/38057196
https://stackoverflow.com/questions/19919905/how-to-bootstrap-numpy-installation-in-setup-py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment