Consider creating an OBS instance for Kata packages

[jodh-intel]

We currently use https://build.opensuse.org/project/show/home:katacontainers:release to build and host package binaries for Kata for various distro versions.

It's a good service but suffers from two awkward issues:

• Reliability:

  • The free service is not always accessible (down for maintance, etc).
  • We sometimes see issues with repository metadata inconsistencies.

• Security:

  • It appears that OBS does not use https fully:

    install: Fix kata installation steps. documentation#83 (comment)

    This is bad since users cannot fully trust the installation of the OBS-generated packages (unless they download them manually via https and then install using a package management tool).

    To make this (big) issue clear, we have to specify http:// URLs in our install guides:

    • https://github.com/kata-containers/documentation/blob/master/install/

OBS is an OSS project so in theory we could create our own instance:

• https://github.com/openSUSE/open-build-service

However, that would require hardware for all architectures and resource to manage it.

@annabellebertooch, @jimmytipit - is this something OSF could investigate for the Kata project?

@cboylan - OOI, does zuul offer any OBS-like functionality?

/cc @sameo, @jcvenegas.

[fungi]

On the security bullet item, please read https://whydoesaptnotusehttps.com/ if you haven't already. It's an article written by the project leader of the Debian GNU/Linux distribution (the creators and maintainers of dpkg/apt). Fedora's dnf https://docs.fedoraproject.org/en-US/fedora/f28/system-administrators-guide/package-management/DNF/index.html and yum on RHEL/CentOS https://wiki.centos.org/TipsAndTricks/YumAndRPM#head-8d4b2b50bf89405814ef598b1f0954c042d5cd29 are similarly able to handle package signature verification. Signing packages and relying on package management features which verify those signatures offers much stronger protection to end users, and makes HTTPS fairly pointless for serving package repositories anyway. As a long-time information security professional myself, this doesn't seem like a bug.

[jodh-intel]

Hi @fungi - thanks for the comment. This is useful information. I understand that package signing is an essential part of the process for installing packages from a remote server. However, what I do not understand is why an initially secure site would choose to downgrade the connection to http (performance maybe?) This issue caused a fair amount of confusion.

Regardless, thankfully, the OBS issue has now been resolved fwics - see kata-containers/documentation#213.

[fungi]

Since official Debian mirrors didn't bother with HTTPS (in part because they solved the problem through signed indices of package checksums instead, but also as it's particularly tough with any meaningful security to coordinate consistent certificates across a distributed network of hundreds of volunteer-run mirror sites), the apt/apt-get tool hasn't traditionally shipped with built-in HTTPS support. There is an optional apt-transport-https plugin package which has emerged fairly recently, mainly as an option for interacting with repositories served from sites which aren't allowed to use HTTP by their operators, so in theory it could be made to work... but OBS downgrading HTTPS to HTTP for package downloads seems to be for much the same reason as Debian: complexities of relying on volunteer-run mirror networks. See https://lists.opensuse.org/opensuse-buildservice/2016-08/msg00040.html and the discussion leading up to it about using HTTPS to host their signing key URLs.

Anyway, as long as your install documentation includes the fingerprints of the signing keys and instructs users to check those before adding them to the package manager's trust set, you should be fine. The OpenStack project does something similar with its release artifact signing, having release managers and infrastructure sysadmins sign the keys which its release automation uses to sign release artifacts, distributing those through the global keyserver network, and publishing details of them on our release documentation site: https://releases.openstack.org/#cryptographic-signatures