signing: start discussion on signing

[philips]

This project should "specify a way to attach signatures". There are a few different approaches that could be taken based on prior art:

1 Detached signature sitting alongside each CAS object (this is the approach appc takes)
2. Inline signatures in the manifest list, putting things into the JSON objects themselves won't work, obviously.
3. Completely external signing with a signing server (this is the approach notary takes).

[philips]

@stevvooe Is there a doc somewhere that describes the notary API in detail not just the CLI tool. Reading through the docs I couldn't find anything that someone wishing to verify name example.com/app with hash ffff could implement from top to bottom. This was the closest I could find: https://github.com/docker/notary/blob/master/docs/getting_started.md#inspect-a-docker-hub-repository

[stevvooe]

@philips Check out the advanced documentation. It demonstrates the use of a text file.

For signatures, the key will be to focus on the signing target, be it a primary artifact or a hash. This is the commonality of all these schemes.

In general, we found inline signatures to be problematic, as it changes the artifact. We've mostly deprecated the original inline signatures in Docker HTTP Registry API V2. Using the manifest list is an interesting proposition, but it may be as problematic in practice, as updates need to be coordinated. In general, manifest lists aren't necessarily on the happy path, since they are more likely to be used when distributing multi-platform artifacts.

[wking]

On Wed, Apr 13, 2016 at 11:01:25PM -0700, Brandon Philips wrote:

1 Detached signature sitting alongside each CAS object (this is the
approach appc takes)

This sounds like “non-CAS stuff in the CAS engine”, which seems like a
bad idea ;).

1. Completely external signing with a signing server (this is the
   approach notary takes).

This seems like the best approach to me. It's still going to have the
coordinated-update problem @stevvooe points out for (2) 1, but it's
in an engine that's designed with that in mind, so locking, etc. while
adding a new signature to the list is acceptable (a CAS engine, on the
other hand, has no need to lock during writes, since there's no
mutable state). The distinction between (2) and (3) is just that (3)
is a better separation of concerns (and allows for deploys that have
signed objects but not manifest lists).

[kamalmarhubi]

1. Inline signatures in the manifest list, putting things into the JSON objects themselves won't work, obviously.

For a take on this, take a look at https://cyberphone.github.io/openkeystore/resources/docs/jcs.html

It specifies ordering and scope of subobjects that are signed.

[kamalmarhubi]

As another alternative in this space, there's the approach taken by TUF, which creates an object like:

{
   "signatures": [
    {
     "keyid": "f2d5020d08aea06a0a9192eb6a4f549e17032ebefa1aa9ac167c1e3e727930d6", 
     "method": "ed25519", 
     "sig": "a312b9c3cb4a1b693e8ebac5ee1ca9cc01f2661c14391917dcb111517f72370809
             f32c890c6b801e30158ac4efe0d4d87317223077784c7a378834249d048306"
    }
   ], 
   "signed": {
        signed object here
    }
}

where they use Canonical JSON to form the byte sequence to be signed based on the signed object.

[wking]

On Wed, Apr 20, 2016 at 08:15:37PM -0700, Kamal Marhubi wrote:
“… they use Canonical JSON to form the byte sequence to be signed
based on the signed object.”

That sounds like a reasonable approach as far as signing sub-objects
goes. My only experience with signed and unsigned portions of objects
is from writing an OpenPGP implementation (e.g. see 1), and working
on it made me want to claw out my eyes ;). I'd personally suggest
staying as far away from partially-signed objects as possible, and
just transmitting byte-for-byte copies of signed payloads with (or
referenced by) the signatures.

For a more detailed sketch of how signing might work, see 2. And
for some prior art, see the IPFS specs [3,4].

[stevvooe]

One time artifact generation with unattached signatures is the sweet spot for content addressable, sign-able content. Anything else creates too high of an engineering burden to maintain flexibility under future changes.

@kamalmarhubi We employed something similar in docker, except we used "prettyjws" version of it. The primary problem is that the target of the signature needs to be unpacked from that format and indexed separately. Otherwise, each time you sign it, you get a different id. This makes the JWS format worthless for CAS applications since to get any benefit, you have to disassemble the content.

[kamalmarhubi]

@stevvooe I think that makes sense. Since we're distributing multiple blobs and manifests as part of an image, it sounds simpler to have "sidecar signatures".

[philips]

Moving to v0.2.0 as I don't think anything actionable has come out of this yet.

[philips]

Actually, moved to v1.0.0 as I don't think anyone has really stepped up to own/help here on OCI signing.

[vbatts]

fair. It does make me think, once examples like this bubble up and folks have options to choose from, there may need to be additional media-types/MIME-types for signature objects.

[wking]

On Tue, May 24, 2016 at 02:12:32PM -0700, Vincent Batts wrote:

It does make me think, once examples like this bubble up and folks
have options to choose from, there may need to be additional
media-types/MIME-types for signature objects.

I expect these already exist. For example application/pgp-signature
1, application/jose+json 2, and application/pkcs7-signature 3
are already established.

[philips]

I meant to move this to post-v1.0.0 as no one has stepped up and as it is an optional layer of the spec. Please discuss if anyone objects to this change @opencontainers/image-spec-maintainers.

[stevvooe]

@philips According to the proposal, this is part of the 1.0 specification. While it is optional for implementors, that proposal is unclear whether it is optional to specify the features for a first release of the specification.

Technically speaking, I'm fine with moving this past 1.0. We know how to do signatures and naming in the current object layout without introducing backwards incompatibility.

[runcom]

/cc @mtrmac @rhatdan

[wking]

On Wed, Jun 08, 2016 at 04:39:59PM -0700, Brandon Philips wrote:

I meant to move this to post-v1.0.0 as no one has stepped up and as
it is an optional layer of the spec.

I've pulled some of my previous suggestions on signed name assertions
together in #176. If folks feel like that sketch has legs, let me
know there and I'll turn it into a PR.

[xiekeyang]

@wking , are you working on signature spec? I've read the sign issues and related technology. It seems that merkle-dag way is what you prefer to. Do you need some help to replenish signature spec? Thanks a lot!

[wking]

On Mon, Aug 29, 2016 at 02:56:49AM -0700, xiekeyang wrote:

@wking , are you working on signature spec?

I've sketched out my proposal for signatures and naming in #176.

[xiekeyang]

@wking May I take on this? I think I may implement signature spec and present a draft pr later, referred to issues discussion. Thanks a lot!

[wking]

On Mon, Aug 29, 2016 at 06:36:40PM -0700, xiekeyang wrote:

@wking May I take on this? I think I may implement signature spec
and present a draft pr later.

I'm not a maintainer or PR gatekeeper ;). But I'm if you want to take
a stab at spec'ing out naming and validation, using my proposal or
something else, I expect that it would help move the discussion along.

[xiekeyang]

@wking , @opencontainers/image-spec-maintainers

Here I want to confirm 3 points with you, thanks lots.

1. signature target

Technically, we can sign any object.
But as the purpose of signature is mostly protecting image from be tampered. So it is enough to sign only for image manifest.
I think we should define the signature title as OpenContainers Image Signature Specification, and emphasize, in spec, that signed target is image manifest, right?
Although that user can sign for any oci content.

2. signature list

[1] shows that you suggest to build signature list in spec. I'm not sure the use case on it.
one manifest need likely only one signature. Docker Notary [2] allows thresholds, but in using case it is defined as 1 permanently.
In signature spec, we should set only one signature item for each signed manifest, right?

3. TUF

If oci signature must comply with TUF? Or just match fields definition in Json Web Signature?

[1] #23 (comment)
[2] https://github.com/docker/notary#goals

[wking]

On Tue, Aug 30, 2016 at 06:56:01PM -0700, xiekeyang wrote:

1. signature target

Technically, signature can identify to any object. But as the
purpose is mostly protecting image from be tampered. So it is enough
to sign only for image manifest. I think we should define the
signature title as OpenContainers Image Signature Specification,
and emphasize, in spec, that signed target is image manifest, right?
Although that user can sign for any oci content.

I think it's better to make the definition generic and emphasize that
you can sign any CAS blob, just like the image-layout spec emphasizes
that it can store any CAS blob 1. Link from the generic
signature-list definition to the assertion section will show folks the
kind of things they can use signatures to say (one of which is the
name assertion for “the name of blob $X is $Y”). More on this in 2.

I agree that signed name-assertions will frequently be on individual
manifests, since adding an additional signature to that signature-list
would break any signatures on the manifest-list referencing the signed
manifest. If you allow signatures (but not the name-assertion they
sign) to be stored outside of the CAS tree (#129), then signed names
for manifest-lists become less brittle.

As long as manifest-lists are not signed, you'll want any
manifest-list information (e.g. the OS and arch of the referenced
manifests) to be verifiable from that downstream data. For example,
if you pick the Linux amd64 entry from an untrusted manifest-list and
follow it through a trusted name-assertion to get to the manifest,
you'd want to check the manifest to ensure the OS and arch are
actually Linux and amd64. Untrusted manifest lists would still be
subject to DoS attacks (“haha, you asked for Linux amd64, but I
pointed you at a Windows arm manifest! pbbt!”), but the images they
reference can accept additional signatures and tooling can
autogenerate a new manifest-list.

Publishers should be free to pick whichever of these approaches works
for them. Some may not care about DoS attacks because they trust the
security of the manifest-list-generating tooling to block that sort of
behavior. Some may not care about allowing additional signatures on
their manifests.

2. signature list

1 shows that you suggest to build signature list in spec. I'm not
sure the use case on it.

Multiple signatures allows you to support situations like:

1. Dev releases widget:1.2.3 and signs the Linux amd64 manifest.

2. Tooling builds manifests for additional OSes and arches, signs
   them, and creates an unsigned manifest-list referencing all
   manifests.

3. QA tests the individual manifests and signs off on them as they
   pass the tests, rebuilding the manifest-list after each
   per-manifest signature-list is amended.

   Subject: Re: [RFC] Current container image workflow analysis and initial
   oci-store implementation.
   Date: Fri, 1 Jul 2016 15:50:36 -0700
   Message-ID: 20160701225035.GV21514@odin.tremily.us

[xiekeyang]

@opencontainers/image-spec-maintainers
/cc @wking @stevvooe @philips @vbatts @runcom

I investigate the technology and present my understanding here. It is my draft document, with brief description in each section. If there is any missing point in my comment, please point out directly.
Maybe there are some your wanted description not to be listed here, just point out, I would go on supplementing.
Thanks so much!

The reference document includes,

• TUF spec
• Docker Notary
• GnuPG
• JWS
• and suggestions in relevant issues.

CAS Content Signature Specification

The mainly goal of spec is:

• To demonstrate the authenticity of any image CAS content by image publisher.
• To support image consumers to validate integrity and authenticity of the CAS contents.
  The signature specification is composed of 2 pattens as signature list and signatures to signed object.

Content Signature List

The signature list points to the individual signatures for specific image manifests for one or more platforms.

Content Signature List Property Descriptions

[...]

(these refer to manifest list field desc)

Example Content Signature List

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.signature.list.v1+json",
  "signatures": [
    {
      "mediaType": "application/vnd.oci.image.signature.v1+json",
      "size": 7143,
      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.signature.v1+json",
      "size": 7682,
      "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270",
      "platform": {
        "architecture": "amd64",
        "os": "linux",
        "features": [
          "sse4"
        ]
      }
    }
  ],
  "annotations": {
    "key1": "value1",
    "key2": "value2"
  }
}

Content Signatures

The content signatures is based on TUF. Signatures patten involves one or more signatures, because that image publisher and observer may co-sign the object, and client should validate all of which.
The signing algorithm adapts x509(options of RSA and ECDSA). Publisher should provide private and public keys to generate the signature, when client uses public key to validate.

Content Signatures Property Descriptions

[...]
The signed object should match the meta format described in tuf-spec.

Example Content Signatures

{
    "signatures": [
        {   
            "keyid": "7fc757801b9bab4ec9e35bfe7a6b61668ff6f4c81b5632af19e6c728ab799599",
            "method": "ecdsa",
            "sig": "Q2YkzUU2dTwanLhtiKd+FogRxi33GmFiX9EreHcyvRNjDU+2hPQwpoKSxlILAoXLxhqA9d7ixDmxmZhyXAzjiQ=="
        }   
    ],  
    "signed": {
        <signed object>
    }
}

As for Name Delegation Design issue, tuf presents the way to design name delegation, which is applied in Docker Notary. I've not solid idea yet. Maybe we can start this section after first signature spec presented.

[wking]

On Tue, Sep 06, 2016 at 04:24:38AM -0700, xiekeyang wrote:

It is my draft document, with brief description in each section.

This should probably be a PR so it's easier to keep track of review
and subsequent changes.

[vbatts]

Per the face-2-face at LinuxCon.EU, it was generally agreed that the spec ought to at least mention the points that are signable, even if not declaring a method to do the signature. Like the manifest as primary target to be signed, even though nothing is stopping from detached-signatures on all the blobs.
As a step to establish the best/common practice for signing.

[wking]

On Thu, Oct 06, 2016 at 10:27:14AM -0700, Vincent Batts wrote:

… it was generally agreed that the spec ought to at least mention
the points that are signable, even if not declaring a method to do
the signature. Like the manifest as primary target to be signed,
even though nothing is stopping from detached-signatures on all the
blobs.

I think there are valid cases for signed name-assertions on both the
manifest and the manifest list 1.

You can sign a name-assertion for the config too and can use diff_ids
to validate layers while unpacking them, but having an untrusted
manifest opens you up to gzip-bombs as you decompress the layers to
compare them with your validated diff_ids. The benefit you get (which
may or may not offset the risk, depending on your environment and
policy preferences) is that a signed name-assertion on the config
would still be valid after a third-party recompresses the layers using
some shiny new compression algorithm.

So I don't think there's a clear winner for a generic
best-place-to-sign type, but there may be some benefit into a guide
that spells out the pros and cons of each possible location.

 Especially the three cases at the end of that comment.

[stevvooe]

I can't stress the importance of #22 (comment) by @vbatts. For signatures to work and be compatible across implementations, we need to define two aspects:

1. What is the scope of the statement being signed? Is it the manifest/list/config directly or do we include something with metadata, such as an annotated descriptor?
2. Where are signing subsystems resolved and how are they structured? How do we balance "resolution" versus "abstraction" without sacrificing functionality?

Number 1 must come before number 2 or we risk a vertically integrated, incompatible mess.

To be clear, this will not be successful if this becomes a file-format discussion, as that won't solve the problem.

TL; DR We need to define an interface to the signing world.

[wking]

On Tue, Oct 18, 2016 at 11:59:19AM -0700, Stephen Day wrote:

1. What is the scope of the statement being signed? Is it the
   manifest/list/config directly or do we include something with
   metadata, such as an annotated descriptor?

I don't think there's one clear thing to sign 1. And I don't think
it really matters if you inline the signature-list. Between:

a. Adding information to the original descriptor. For example
manifest-list.manifests[0].signatures.
b. Adding a new application/vnd.oci.image.signed.blob.v1+json type,
with manifest-list.manifests[0] pointing at a signed blob and the
signed blob pointing at the manifest.

(a) is one fewer type to define and one less round-trip when fetching,
but (b) is easy to use with existing descriptors (you don't have to
change the manifes-list or manifest schema to use (b)). Neither of
those differences seem particularly critical to me.