Page isn't finished: https://packaging.python.org/extensions/

[alexlenail]

I'm particularly interested in the last header: https://packaging.python.org/extensions/#publishing-binary-extensions which is completely empty =0

[ncoghlan]

The two main options I'm currently aware of for this are:

1. Use enscons as your build system (rather than distutils or setuptools): https://pypi.python.org/pypi/enscons/

That gives you a full-fledged build system to use (Scons), with some additions for also emitting Python packages as a build output, rather than having to figure out how to teach distutils/setuptools to build your extension modules.

2. Tell distutils/setuptools how to build the extension: https://docs.python.org/3/extending/building.html#building-c-and-c-extensions-with-distutils

This option is OK for simple extension modules, including CFFI wrappers as described in http://cffi.readthedocs.io/en/latest/cdef.html

So a possible subsection structure that might work:

• Publishing simple extension modules with setuptools

  • Pull the example from the CPython documentation into here and adapt it to use setuptools instead

• Publishing CFFI extension modules with setuptools

  • Pull the setuptools specific snippet from the CFFI docs into here

• Publishing complex extension modules with enscons

  • Explain that while possible, building complex extension modules directly with distutils or setuptools tends to be fragile and error-prone, so it's no longer a recommended approach
  • Point to enscons as a project that can be either used directly for projects that are willing and able to adopt SCons as their build system, or else used as the basis for wrapping another build system in an sdist-compatible build interface

• Providing pre-built wheel files through PyPI

  • Explain the binary compatibility tagging system
  • Explain the stable ABI and the ability to preserve binary extension compatibility across releases

Other (also incomplete) sections that could potentially be referenced:

• Application deployment: https://packaging.python.org/deployment/
• Windows support through Appveyor: https://packaging.python.org/appveyor/

[ncoghlan]

I added a link to the CPython tutorial on writing extension modules by hand: 532268a

I also added a link from the publishing section back to this issue.

[ned-deily]

Perhaps another topic to include in this section, if not discussed elsewhere, is how to package extensions modules built with multiple ABIs, in particular, non-debug vs debug builds. For Python 3.x, PEP 3149 and follow-ons provide ABI-unique extension module file names so that they can be installed in the same directory. (Python 2 does not attempt to address this.)

[asmodehn]

While looking around for this kind of information, I also found CMake python integration (see here and there)

Another (related) point:
While it is easier for the developer to build binary extension from the maintainer side, I find it "more friendly for the user" (ie. more pythonic/open) and easier to distribute, if the user installing the package also builds the extension. I believe I have seen some package taking this approach, but I haven't found any documentation about it.

[ncoghlan]

@asmodehn Yes, end users being free to build the extension for themselves is why we discourage wheel-only distribution - we prefer publishers to offer both an sdist (for folks that want to build from source) and wheels (for folks that may not even have the appropriate compilation toolchains available).

[ncoghlan]

scikit-build is another option for CMake integration: #381