Split buildPythonPackage into setup hooks

This commit splits the `buildPythonPackage` into multiple setup hooks.

Generally, Python packages are built from source to wheels using `setuptools`.
The wheels are then installed with `pip`. Tests were often called with
`python setup.py test` but this is less common nowadays. Most projects
now use a different entry point for running tests, typically `pytest`
or `nosetests`.

Since the wheel format was introduced more tools were built to generate these,
e.g. `flit`. Since PEP 517 is provisionally accepted, defining a build-system
independent format (`pyproject.toml`), `pip` can now use that format to
execute the correct build-system.

In the past I've added support for PEP 517 (`pyproject`) to the Python
builder, resulting in a now rather large builder. Furthermore, it was not possible
to reuse components elsewhere. Therefore, the builder is now split into multiple
setup hooks.

The `setuptoolsCheckHook` is included now by default but in time it should
be removed from `buildPythonPackage` to make it easier to use another hook
(curently one has to pass in `dontUseSetuptoolsCheck`).

doc/languages-frameworks/python.section.md

@@ -540,7 +540,8 @@ and the aliases
 #### `buildPythonPackage` function
 
 The `buildPythonPackage` function is implemented in
-`pkgs/development/interpreters/python/build-python-package.nix`
+`pkgs/development/interpreters/python/mk-python-derivation`
+using setup hooks.
 
 The following is an example:
 ```nix
@@ -797,6 +798,22 @@ such as `ignoreCollisions = true` or `postBuild`. If you need them, you have to
 Python 2 namespace packages may provide `__init__.py` that collide. In that case `python.buildEnv`
 should be used with `ignoreCollisions = true`.
 
+#### Setup hooks
+
+The following are setup hooks specifically for Python packages. Most of these are
+used in `buildPythonPackage`.
+
+- `flitBuildHook` to build a wheel using `flit`.
+- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
+- `pipInstallHook` to install wheels.
+- `pytestCheckHook` to run tests with `pytest`.
+- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
+- `pythonImportsCheckHook` to check whether importing the listed modules works.
+- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
+- `setuptoolsBuildHook` to build a wheel using `setuptools`.
+- `setuptoolsCheckHook` to run tests with `python setup.py test`.
+- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed with the `pipInstallHook`.
+
 ### Development mode
 
 Development or editable mode is supported. To develop Python packages

pkgs/development/interpreters/python/hooks/default.nix

@@ -0,0 +1,95 @@
+# Hooks for building Python packages.
+{ python
+, callPackage
+, makeSetupHook
+}:
+
+let
+  pythonInterpreter = python.pythonForBuild.interpreter;
+  pythonSitePackages = python.sitePackages;
+  pythonCheckInterpreter = python.interpreter;
+  setuppy = ../run_setup.py;
+in rec {
+
+  flitBuildHook = callPackage ({ flit }:
+    makeSetupHook {
+      name = "flit-build-hook";
+      deps = [ flit ];
+      substitutions = {
+        inherit pythonInterpreter;
+      };
+    } ./flit-build-hook.sh) {};
+
+  pipBuildHook = callPackage ({ pip }:
+    makeSetupHook {
+      name = "pip-build-hook.sh";
+      deps = [ pip ];
+      substitutions = {
+        inherit pythonInterpreter pythonSitePackages;
+      };
+    } ./pip-build-hook.sh) {};
+
+  pipInstallHook = callPackage ({ pip }:
+    makeSetupHook {
+      name = "pip-install-hook";
+      deps = [ pip ];
+      substitutions = {
+        inherit pythonInterpreter pythonSitePackages;
+      };
+    } ./pip-install-hook.sh) {};
+
+  pytestCheckHook = callPackage ({ pytest }:
+    makeSetupHook {
+      name = "pytest-check-hook";
+      deps = [ pytest ];
+      substitutions = {
+        inherit pythonCheckInterpreter;
+      };
+    } ./pytest-check-hook.sh) {};
+
+  pythonCatchConflictsHook = callPackage ({ setuptools }:
+    makeSetupHook {
+      name = "python-catch-conflicts-hook";
+      deps = [ setuptools ];
+      substitutions = {
+        inherit pythonInterpreter;
+        catchConflicts=../catch_conflicts/catch_conflicts.py;
+      };
+    } ./python-catch-conflicts-hook.sh) {};
+
+  pythonImportsCheckHook = callPackage ({}:
+    makeSetupHook {
+      name = "python-imports-check-hook.sh";
+      substitutions = {
+        inherit pythonCheckInterpreter;
+      };
+    } ./python-imports-check-hook.sh) {};
+
+  pythonRemoveBinBytecodeHook = callPackage ({ }:
+    makeSetupHook {
+      name = "python-remove-bin-bytecode-hook";
+    } ./python-remove-bin-bytecode-hook.sh) {};
+
+  setuptoolsBuildHook = callPackage ({ setuptools, wheel }:
+    makeSetupHook {
+      name = "setuptools-setup-hook";
+      deps = [ setuptools wheel ];
+      substitutions = {
+        inherit pythonInterpreter pythonSitePackages setuppy;
+      };
+    } ./setuptools-build-hook.sh) {};
+
+  setuptoolsCheckHook = callPackage ({ setuptools }:
+    makeSetupHook {
+      name = "setuptools-check-hook";
+      deps = [ setuptools ];
+      substitutions = {
+        inherit pythonCheckInterpreter setuppy;
+      };
+    } ./setuptools-check-hook.sh) {};
+
+  wheelUnpackHook = callPackage ({ }:
+    makeSetupHook {
+      name = "wheel-unpack-hook.sh";
+    } ./wheel-unpack-hook.sh) {};
+}

pkgs/development/interpreters/python/hooks/flit-build-hook.sh

@@ -0,0 +1,15 @@
+# Setup hook for flit
+echo "Sourcing flit-build-hook"
+
+flitBuildPhase () {
+    echo "Executing flitBuildPhase"
+    preBuild
+    @pythonInterpreter@ -m flit build --format wheel
+    postBuild
+    echo "Finished executing flitBuildPhase"
+}
+
+if [ -z "$dontUseFlitBuild" ] && [ -z "$buildPhase" ]; then
+    echo "Using flitBuildPhase"
+    buildPhase=flitBuildPhase
+fi