Add package signing.

* Add GnuPG based signing for Arch, Fedora, Manjaro and OpenSUSE * Add OpenSSL based signing for Void Linux * Ignore Debian/Ubuntu which don't meaningfully support signing except at a repository level.
bwoodsend · Nov 22, 2023 · 34c0dde · 34c0dde
1 parent 0506db6
commit 34c0dde
Show file tree

Hide file tree

Showing 46 changed files with 1,222 additions and 111 deletions.
diff --git a/.gitignore b/.gitignore
@@ -122,3 +122,5 @@ ENV/
 # Generated
 docs/source/history.rst
 docs/source/schema.rst
+
+tests/gpg-home/.#*
diff --git a/docs/source/arch.rst b/docs/source/arch.rst
@@ -33,3 +33,39 @@ package is no longer findable. Once this happens, your existing built packages
 are effectively useless and you need to rebuild and release then encourage your
 users to run ``pacman -Syu`` (upgrade all packages) before installing/upgrading
 your package in case they still have the previous version of Python installed.
+
+
+Package Signing
+...............
+
+Arch packages are optionally signed using a GnuPG_ detached signature. See
+:ref:`gpg_signing` for the signing itself.
+
+**To consume** your signed package, downstream users will need to install your
+public key into their ``pacman`` key stores. You can get your key to them in two
+ways:
+
+1. The recommended way is to upload it to Arch's preferred keyserver::
+
+    gpg --armor --export 3CB69E1833270B714034B7558CA85BF8D96DB4E9
+    # Copy/paste the output to http://keyserver.ubuntu.com/#submitKey
+
+  Note that there will be around a one hour propagation delay before the next
+  steps can work. Installers of your package should be instructed to import your
+  key from the keyserver as follows::
+
+    sudo pacman-key --init
+    sudo pacman-key --recv-keys 3CB69E1833270B714034B7558CA85BF8D96DB4E9
+    sudo pacman-key --lsign-key 3CB69E1833270B714034B7558CA85BF8D96DB4E9
+
+2. Alternatively, you can boycott the keyserver and put the public key somewhere
+   on your website. Run::
+
+    gpg --armor --export 3CB69E1833270B714034B7558CA85BF8D96DB4E9 > 3CB69E1833270B714034B7558CA85BF8D96DB4E9.asc
+
+  Then put the ``.asc`` file somewhere downloadable on your website, with
+  instructions to your users to run::
+
+    sudo pacman-key --init
+    curl https://your.website/downloads/3CB69E1833270B714034B7558CA85BF8D96DB4E9.asc | sudo pacman-key --add -
+    sudo pacman-key --lsign-key 3CB69E1833270B714034B7558CA85BF8D96DB4E9
diff --git a/docs/source/fedora.rst b/docs/source/fedora.rst
@@ -76,3 +76,34 @@ megabytes, subsequent updates require only a few megabytes. The complexity of
 * ``zchunk`` performs so badly under ``qemu`` architecture emulation that even a
   minimal package takes around 20 minutes to build. Building for non-native
   architectures is not recommended nor tested.
+
+
+.. _fedora_signing:
+
+Package Signing
+...............
+
+Fedora packages are optionally signed using an embedded GnuPG_ signature. See
+:ref:`gpg_signing` for the signing itself.
+
+**To consume** your signed package, downstream users should (although ``rpm``
+strangely does nothing to prevent you from installing packages with untrusted
+signatures) install your public key into their ``rpm`` key stores.
+
+Export your public key::
+
+    gpg --armor --export 3CB69E1833270B714034B7558CA85BF8D96DB4E9 > 3CB69E1833270B714034B7558CA85BF8D96DB4E9.asc
+
+Then put the ``.asc`` file somewhere downloadable on your website. Users can
+then import the key using::
+
+    curl -O https://your.website/downloads/3CB69E1833270B714034B7558CA85BF8D96DB4E9.asc
+    sudo rpm --import 3CB69E1833270B714034B7558CA85BF8D96DB4E9.asc
+
+The should now be able to verify your package using::
+
+    rpm -K your-package-0.1.0-1.fc39.noarch.rpm
+
+Note that DNF will not block installation if the key is not imported. The only
+indicator that something is wrong would be the message ``digests SIGNATURES NOT
+OK`` from ``rpm -K``.
diff --git a/docs/source/gpg.rst b/docs/source/gpg.rst
@@ -0,0 +1,38 @@
+.. _gpg_signing:
+
+=============
+GnuPG Signing
+=============
+
+Some distributions, namely Arch, Fedora, Manjaro and OpenSUSE, optionally use
+GnuPG_ to sign their packages. Other distributions either use their own wrappers
+around OpenSSL, for which the signing process is documented under :ref:`each
+distribution's quirks page <building for>`, or don't meaningfully support
+signing.
+
+.. note::
+
+    Before embarking on signing, bear in mind that, without a web of trust based
+    or in-person public key verification, a signature is more or less a
+    meaningless exercise, providing less security than HTTPS.
+
+To sign your packages:
+
+* Generate an RSA signing key for yourself or your organisation using ``gpg
+  --generate-key``.
+
+* Run ``gpg --list-secret-keys`` to find the key key ID (a 40 character
+  hexadecimal string) of the key you just generated.
+
+* Pass that key ID to the ``--gpg-signing-id`` flag when building (replace
+  ``arch`` with whatever distribution you're building for)::
+
+    polycotylus arch --gpg-signing-id 3CB69E1833270B714034B7558CA85BF8D96DB4E9
+
+If your GnuPG key has a password, you will be prompted to enter it during the
+build. There is currently no automation friendly way to pass the password through
+`polycotylus` to GnuPG_.
+
+**To consume** your signed package, downstream users will need to install your
+public key into their package manager's key stores. The process is different on
+each distribution – see :ref:`each distribution's quirks page <building for>`.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -12,6 +12,7 @@
     requirements
     example-library
     example-gui/index
+    gpg
 
 .. _`building for`:
 

diff --git a/docs/source/opensuse.rst b/docs/source/opensuse.rst
@@ -40,3 +40,10 @@ Caveats
 .......
 
 * Building for OpenSUSE is not supported with Podman_.
+
+
+Package Signing
+...............
+
+OpenSUSE's signing process is the same as Fedora's. See :ref:`Fedora package
+signing <fedora_signing>`.
diff --git a/docs/source/rst_prolog.txt b/docs/source/rst_prolog.txt
@@ -21,5 +21,6 @@
 .. _Poetry: https://python-poetry.org/
 .. _SPDX: https://spdx.org/licenses/
 .. _qemu: https://www.qemu.org/
+.. _GnuPG: https://gnupg.org/
 
 .. highlight:: bash
diff --git a/docs/source/void.rst b/docs/source/void.rst
@@ -18,3 +18,59 @@ following::
 
     polycotylus void:glibc  # The default, equivalent to `polycotylus void`
     polycotylus void:musl
+
+
+Package Signing
+...............
+
+Void Linux packages use a detached RSA signature. To generate an RSA key run either::
+
+    openssl genrsa -des3 -out privkey.pem 4096
+
+Or if you want builds to be automatable, generate a password-less key using::
+
+    openssl genrsa -out privkey.pem 4096
+
+Then to build a package with signing enabled, pass the path to the private key
+to the ``--void-signing-certificate`` option::
+
+    polycotylus void --void-signing-certificate privkey.pem
+
+There are several files of interest produced:
+
+.. code-block:: console
+
+    $ tree .polycotylus/void
+    .polycotylus/void
+    ├── 64:0c:81:d1:54:d6:6d:88:b4:49:4a:4e:c6:0a:1c:26.plist  (1)
+    ├── Dockerfile
+    ├── hostdir
+    │   └── sources
+    │       └── dumb_text_viewer-0.1.0
+    │           └── 0.1.0.tar.gz
+    ├── musl
+    │   ├── dumb_text_viewer-0.1.0_1.x86_64-musl.xbps          (2)
+    │   ├── dumb_text_viewer-0.1.0_1.x86_64-musl.xbps.sig2     (3)
+    │   └── x86_64-musl-repodata                               (4)
+    └── srcpkgs
+        └── dumb_text_viewer
+            └── template
+
+1. Your public key, in the format that XBPS uses
+2. The package itself
+3. The detached signature for the package
+4. The repository index, containing an embedded signature
+
+
+To consume a package
+--------------------
+
+The preferred way for a user to import your public key is just to install a
+signed package, which will display the signing key's fingerprint and ask if the
+user wants to import the key. That fingerprint is the basename of the
+``.polycotylus/void/{md5sum}.plist`` file – put that fingerprint somewhere
+downloaders of your package can find it and know that it belongs to you.
+
+An automation friendly, albeit rarely used alternative is to copy the public key
+into XBPS's key-store. Do this simply by having the user put the
+``{md5sum}.plist`` file into their ``/var/db/xbps/keys/`` directory.
diff --git a/polycotylus/__main__.py b/polycotylus/__main__.py
@@ -75,6 +75,8 @@ def __call__(self, parser, namespace, key, option_string=None):
 parser.add_argument("--list-localizations", action=ListLocalizationAction,
                     choices=["language", "region", "modifier"])
 parser.add_argument("--architecture")
+parser.add_argument("--gpg-signing-id")
+parser.add_argument("--void-signing-certificate")
 parser.add_argument("--post-mortem", action="store_true",
                     help="Enter an in-container interactive shell whenever an "
                     "error occurs in a docker container")
@@ -90,7 +92,13 @@ def cli(argv=None):
         polycotylus._docker.post_mortem = options.post_mortem
 
         cls = polycotylus.distributions[options.distribution]
-        self = cls(polycotylus.Project.from_root("."), options.architecture)
+        signing_id = None
+        if issubclass(cls, polycotylus._base.GPGBased):
+            signing_id = options.gpg_signing_id
+        elif issubclass(cls, polycotylus.Void):  # pragma: no branch
+            signing_id = options.void_signing_certificate
+        self = cls(polycotylus.Project.from_root("."), options.architecture,
+                   signing_id)
         self.generate()
         artifacts = self.build()
         self.test(artifacts["main"])
@@ -100,6 +108,11 @@ def cli(argv=None):
     print(f"Built {len(set(artifacts.values()))} artifact{'s' if len(artifacts) != 1 else ''}:")
     for (variant, path) in artifacts.items():
         print(f"{variant}: {path}")
+    return artifacts
+
+
+def _console_script():  # pragma: no cover
+    cli()
 
 
 if __name__ == "__main__":

diff --git a/polycotylus/_arch.py b/polycotylus/_arch.py
@@ -6,12 +6,14 @@
 from functools import lru_cache
 import contextlib
 import shutil
+import os
+from pathlib import Path
 
 from polycotylus import _misc, _docker
-from polycotylus._base import BaseDistribution
+from polycotylus._base import BaseDistribution, GPGBased
 
 
-class Arch(BaseDistribution):
+class Arch(GPGBased, BaseDistribution):
     base_image = "archlinux:base"
     python_prefix = "/usr"
     python_extras = {
@@ -139,9 +141,11 @@ def pkgbuild(self):
         """)
         return out
 
+    patch_gpg_locale = ""
+
     def dockerfile(self):
         dependencies = self.dependencies + self.build_dependencies + self.test_dependencies
-        return self._formatter(f"""
+        out = self._formatter(f"""
             FROM {self.base_image} AS base
 
             RUN {self.mirror.install_command}
@@ -153,10 +157,17 @@ def dockerfile(self):
             FROM base as build
             RUN echo 'PACKAGER="{self.project.maintainer_slug}"' >> /etc/makepkg.conf
             RUN pacman -Syu --noconfirm --needed base-devel {shlex.join(dependencies)}
+            {self.patch_gpg_locale}
 
             FROM base AS test
             RUN pacman -Syu --noconfirm --needed {shlex.join(self.test_dependencies)}
-    """)
+        """)
+        if self.signing_id:
+            out += self._formatter(f"""
+                RUN pacman-key --init
+                RUN echo '{self.public_key}' | base64 -d | pacman-key --add - && pacman-key --lsign '{self.signing_id}'
+            """)
+        return out
 
     def generate(self):
         with contextlib.suppress(FileNotFoundError):
@@ -169,10 +180,19 @@ def generate(self):
         _misc.unix_write(self.distro_root / "PKGBUILD", self.pkgbuild())
 
     def build(self):
+        if self.signing_id:
+            signing_flags = ["--sign", "--key", self.signing_id]
+            gpg_home = os.environ.get("GNUPGHOME", Path.home() / ".gnupg")
+            gpg_volume = [(str(gpg_home), "/home/user/.gnupg")]
+        else:
+            signing_flags = []
+            gpg_volume = []
         with self.mirror:
-            _docker.run(self.build_builder_image(), "makepkg -fs --noconfirm",
-                        volumes=[(self.distro_root, "/io")], root=False,
-                        architecture=self.docker_architecture, tty=True, post_mortem=True)
+            _docker.run(self.build_builder_image(),
+                        ["makepkg", "-fs", "--noconfirm", *signing_flags],
+                        volumes=[(self.distro_root, "/io"), *gpg_volume],
+                        root=False, architecture=self.docker_architecture,
+                        tty=True, post_mortem=True, interactive=True)
         architecture = self.architecture if self.project.architecture != "none" else "any"
         package, = self.distro_root.glob(
             f"{self.package_name}-{self.project.version}-*-{architecture}.pkg.tar.zst")

diff --git a/polycotylus/_base.py b/polycotylus/_base.py
@@ -5,6 +5,8 @@
 import platform
 import json
 from functools import lru_cache
+import subprocess
+import base64
 
 from packaging.requirements import Requirement
 
@@ -19,8 +21,9 @@ class BaseDistribution(abc.ABC):
     supported_architectures = abc.abstractproperty()
     _packages = abc.abstractproperty()
     tag = abc.abstractproperty()
+    signature_property = None
 
-    def __init__(self, project, architecture=None):
+    def __init__(self, project, architecture=None, signature=None):
         self.project = project
         self.architecture = architecture or self.preferred_architecture
         if self.architecture not in self.supported_architectures:
@@ -45,6 +48,8 @@ def __init__(self, project, architecture=None):
                         native package manager.
                     """))
                 _docker.setup_binfmt()
+        if self.signature_property:
+            setattr(self, self.signature_property, signature)
 
     @property
     def distro_root(self):
@@ -333,6 +338,44 @@ def update_artifacts_json(self, packages):
             _misc.unix_write(json_path, json.dumps(artifacts, indent="  "))
 
 
+class GPGBased(abc.ABC):
+    signature_property = "signing_id"
+
+    @property
+    def signing_id(self):
+        return self._signing_id
+
+    @signing_id.setter
+    def signing_id(self, id):
+        if id is None:
+            self._signing_id = None
+            return
+        # Normalise key identifier (name/email/abbreviated fingerprint) to full
+        # length fingerprint (which doubles as a check that the key exists).
+        p = subprocess.run(["gpg", "--with-colons", "--status-fd=2", "--list-secret-keys", id],
+                           stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+        if p.returncode:
+            assert "[GNUPG:] ERROR keylist.getkey 17" in p.stderr, p.stderr + "\nI am a bug in polycotylus, please report me!"
+            raise _exceptions.PolycotylusUsageError(
+                f'No private GPG key found with user ID or fingerprint "{id}"')
+        # For GPG's machine readable (--with-colons) mode, see:
+        # https://github.com/gpg/gnupg/blob/gnupg-2.5-base/doc/DETAILS#format-of-the-colon-listings
+        lines = p.stdout.splitlines()
+        keys = sorted(i.split(":")[4] for i in lines if i.startswith("sec:"))
+        if len(keys) > 1:
+            raise _exceptions.PolycotylusUsageError(
+                f'The GPG signing key identifier "{id}" is ambiguous. It could refer to either of {keys}')
+        fingerprints = [i.split(":")[9] for i in lines if i.startswith("fpr:")]
+        self._signing_id, = [i for i in fingerprints if i.endswith(keys[0])]
+
+    @property
+    def public_key(self):
+        p = subprocess.run(["gpg", "--armor", "--export", self.signing_id],
+                           stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        assert p.returncode == 0, p.stderr.decode()
+        return base64.b64encode(p.stdout).decode("ascii")
+
+
 def _deduplicate(array):
     """Remove duplicates, preserving order of first appearance."""
     return list(dict.fromkeys(array))
diff --git a/polycotylus/_completions/polycotylus.fish b/polycotylus/_completions/polycotylus.fish
@@ -28,6 +28,11 @@ complete -x -c polycotylus -n 'not __fish_seen_subcommand_from $all_variants &&
 complete -x -c polycotylus -n 'not __fish_seen_subcommand_from $all_variants && string match -rq -- u (commandline -t)' -a "$ubuntu_variants"
 complete -x -c polycotylus -n 'not __fish_seen_subcommand_from $all_variants && string match -rq -- v (commandline -t)' -a "$void_variants"
 
+# Suggest GPG signing only for distributions that use it or when no distribution is given.
+complete -c polycotylus -x -l gpg-signing-id -n "not __fish_seen_subcommand_from $all_variants || __fish_seen_subcommand_from arch fedora $fedora_variants manjaro opensuse" -a '(type -q gpg && __fish_complete_gpg_key_id gpg --list-secret-keys)'
+# Likewise with VoidLinux signing certificates.
+complete -c polycotylus -r -l void-signing-certificate -n "not __fish_seen_subcommand_from $all_variants || __fish_seen_subcommand_from void $void_variants"
+
 complete -c polycotylus -f -s q -l quiet -d 'Decrease verbosity'
 complete -c polycotylus -f -l post-mortem -d 'Enter container on error'
 complete -c polycotylus -f -s h -l help -d 'Show help'