A simple build system in python to build C extensions
Fetching latest commit…
Cannot retrieve the latest commit at this time
.. vim syntax=rst What is Yaku ============ Yaku is a simple system (think make) in python, aimed at building python extensions, although it could be used for other tasks if python is your thing Main features ============= The main features of Yaku are: - Simplicity above all (core code ideally ~ 1000 LOC - currently ~ 1500 LOC) - Easy to extend: - adding handler for new source types (.pyx for cython, .f for Fortran, .i for SWIG) may be done in one importable module, without the need for monkey-patching or subclasses as currently required by distutils - adding new compilers or new tools is also easier - Simple dependency handling: whenever a source file content changes, all targets which depend on it will be considered out of date - Although not yet implemented, parallel builds may be added relatively simply - Can be used as a library to be controled by an external python process. Yaku raison-d'etre is to be the build tool for bento What Yaku is not ================ Yaku is not a replacement for real build systems like scons or waf. To keep it simple, I made a few design choices which significantly impact what it can do: - Yaku has no notion of source directory - you have to build things from the original source directory - Once Yaku's DAG is built, it is assumed to never change, and adding new dependencies on the fly is not possible. For example, if you have a .c.in -> .c task, it won't be possible for yaku to scan changes in the .c file (additional headers, etc...) - Yaku does not aim at being very fast. It does not uses fancy algorithms, and is unlikely to work very well for thousand of source files, although it does work ok for packages as large as scipy. Yaku's design is mostly a dumb-downed rip-off of waf, with a focus on making it usable as a library. When I started yaku, using waf as a library was a bit difficult, but the new 1.6 version improved things quite a bit, and I may just replace yaku by waf at that point. Why Yaku ? ========== Currently, the vast majority of python C extensions are built using distutils. Distutils is an imperative, inflexible way of building extensions. Instead, one could use a real build system like the aforementioned waf or scons. But both are rather large codebases, with their own release schedule. Yaky aims at replacing the build infrastructure from distutils. It is used as the default build engine for bento, but I hope it may become a simple, default engine for other distributions tools like setuptools/distribute. Yaku vs distutils ================= With distutils: - Handling new kind of sources (SWIG, Cython, etc...) requires extending distutils Commands through subclassing, which cause various issues when you need to integrate distutils extensions through various sources. - Distutils has no notion of dependency handling - Adding new compiler or new type of targets (ctypes, etc...) is a black art that few people master - Distutils codebase is horrible, and needlessly complex. Yaku is based around a Task concept similar to make rules, and has a callback-like system to register new source files types (.pyx for cython or pyrex, .i for swig, etc...). - Each task has a signature, which is used to determine whether a task should be executed. The signature depends on the source file content, the target content, the build environment as well as the dependencies content (md5 checksum), so that changes can be reliably detected. - It uses dependencies defined in each task to build things in the correct order. - Task are automatically built for files with registered file extension, and new hooks for extension can be added. - Automatic dependency handling (scanning C/C++ files to detect its dependencies) may be added later Note that by default, yaku currently uses the same compilation options as distutils, but that may change in the future given the inconsistencies in various bits to get options from (sysconfig, msvc compiler classes/instances, etc...). What's with the name ? ====================== Yaku means "bake" or "grill" in Japanese. It also happens that yaki-niku ("grilled meat") is one of my favorite food in Japan.