Proposal: Provenance step 1 - Transform images for validation and verification #8093

Closed
vbatts opened this Issue Sep 17, 2014 · 48 comments

Projects

None yet
@vbatts
Contributor
vbatts commented Sep 17, 2014

Background

The current image format does not allow for content addressable images nor
require metadata about the image to reference the content of the layer used by
the metadata. The identifiers for layers are randomly generated requiring
extra book keeping to map layer ids to the content being referred to. In a
highly distributed environment such as the Docker ecosystem, this book keeping
is cumbersome and complicates security.

This relates to #6805 #6959

Proposal Summary

Make images self-describing manifests containing a list of content addressable
layers, run configuration, and a signatures to identify the builder and verify
the image meets the expectations of the installer.

Image Manifest

The image manifest file will contain all the information which is needed to
pull, install, validate and run an image. It will contain a list of layers by
a content addressable id, history, run time configuration, and signatures.
This manifest is generated by the daemon. Initially this generation will happen
when an image is published, and ultimately happen anytime an image is built or
committed. Each manifest is required to be signed by the client creating the
manifest on push or build with additional signatures which can be added post
build to verify the quality of the manifest or validity of the builder. The
history will contain fully backward compatible metadata to allow old style
layer and metadata to be recreated from the manifest.

Signable manifest (or payload) refers to portions of the manifest which
will be signed by builder. The signable manifest is a JSON dictionary
containing the layers, run configuration, and history. The entire signable
manifest will signed, any changes including whitespace will require a new
signature. To aid in readability the signable manifest should be
well-formatted JSON.

Example: (totally subject to change)

{
   "name": "dmcgowan/test-image",
   "tag": "latest",
   "architecture": "amd64",
   "fsLayers": [
      {
         "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      },
      {
         "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
      },
      {
         "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      },
      {
         "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
      }
   ],
   "history": [
      "{\"id\":\"a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721\",\"parent\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"created\":\"2014-06-05T00:05:35.990887725Z\"...",
      "{\"id\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"parent\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"created\":\"2014-06-05T00:05:35.692528634Z\"...",
      "{\"id\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"parent\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"created\":\"2014-06-05T00:05:35.589531476Z\"...",
      "{\"id\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"comment\":\"Imported from -\",\"created\":\"2013-06-13T14:03:50.821769-07:00\"..."
   ],
   "schemaVersion": 1
}

Signed manifest refers to a manifest which includes the signature as well
as the signable manifest. The signed manifest could be represented as either a
JSON Web Signature (JSON serialization, see link), in which the payload is the
base64 encoded signed manifest, or an altered version of the signed manifest
JSON to include the signature as the last element of the JSON dictionary and a
record of the alternations to the original signed manifest included in the
signature. EIther format is fully verifiable and tamper-proof.

http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2

Example: (human readable format)

{
   "name": "dmcgowan/test-image",
   "tag": "latest",
   "architecture": "amd64",
   "blobSums": [
      {
         "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      },
      {
         "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
      },
      {
         "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      },
      {
         "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
      }
   ],
   "history": ["v1 compatible string encoded json for each layer"],
   "schemaVersion": 1,
   "signatures": [
      {
         "header": {
            "jwk": {
               "crv": "P-256",
               "kid": "LYRA:YAG2:QQKS:376F:QQXY:3UNK:SXH7:K6ES:Y5AU:XUN5:ZLVY:KBYL",
               "kty": "EC",
               "x": "Cu_UyxwLgHzE9rvlYSmvVdqYCXY42E9eNhBb0xNv0SQ",
               "y": "zUsjWJkeKQ5tv7S-hl1Tg71cd-CqnrtiiLxSi6N_yc8"
            },
            "alg": "ES256"
         },
         "signature": "m3bgdBXZYRQ4ssAbrgj8Kjl7GNgrKQvmCSY-00yzQosKi-8UBrIRrn3Iu5alj82B6u_jNrkGCjEx3TxrfT1rig",
         "protected": "eyJmb3JtYXRMZW5ndGgiOjYwNjMsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxNC0wOS0xMVQxNzoxNDozMFoifQ"
      }
   ]
}

Content Addressable Layers

Each layer of an image will be referenced by a checksum created from its
contents. This checksum will be used on push and pull to verify contents have
not been tampered and disallow the layer referred to in the manifest to be
changed after signed.

History

For auditability and assurance of the image, there will be a history section.
This history will convey the life of the the image (build steps, ancestry,
prior attestations on parent images, etc.).
It will have a generic form, and it is important to note that its content is
included in the signed payload.

Signature

Every client and daemon will contain both a public key pair which can be used
to sign manifests. So that the user on the host that publishes (or builds) the
image can sign the image manifest, without sharing their keys for all users on
the host.

Verification

Note: Verification framework will be vetted out in a separate Proposal
review, but the following is provided for a complete picture of its role.

Verification of a manifest will be done by checking the public key used to sign
the image manifest against an authorization graph linking keys to users and the
image namespaces. The authority for this graph will be a remote server which
can respond to authorization queries with signed statements which can be cached
and imported locally for future authorizations. These signed statements which
are received and cached contain a chain of trust which verify their
authenticity. A root certificate to verify this chain will ship with Docker to
allow immediate verification of these statements. Certificates are X509
certificates and verification uses an x509 chain included with the signed
statement. The signed statement will be a JSON Web Signature with the contents
a series of graph nodes and edges to be imported and the x509 chain in the
signature header.

http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-4.1.6

Registry V2 API

(Note: this versioning is on the API of talking to a docker registry)

Unlike the present /v1/... which is locked to the root of the URI path, the
/v2/... is expected be the relative root of the path, for easier route
handling.

Manage manifest by tag

GET/PUT/DELETE /v2/images/<imgname>/<tagname>

List tags

GET /v2/images/<imgname>/tags

Download an image layer by content id

  • Performs ACL verification
  • Redirects to a temporary signed URL

GET /v2/images/<imgname>/<sumtype>/<sum>

Upload an image layer

PUT /v2/images/<imgname>/<sumtype>/<sum>

Upload an image layer

PUT /v2/images/<imgname>/<sumtype>

Compatibility

For compatibility with prior versions of docker-registries and docker daemons,
the manifest will store the json metadata used in previous versions in the
history section. This history will allow recreation of the layers in the
previous format and layout. Version 2 registries can synchronize content with
version 1 registries using this content in order to ensure content is still
accessible through the version 1 API.
There will be a couple of phases.

Phase 1

Have v2 capable registry and docker daemon that can:

  • detect and push as v2 where possible
  • pull from v2 where possible
  • validate signature on pull

Phase 2

  • escrow layer checksums on for layers on the daemon
  • produce a manifest at build/commit time (not just on publish)

Phase 3

  • provide a strict mode where only verified signed images can be run
  • flags to validate layers

Strictly V2

In the future when signatures are enforced more strictly, it will become more
difficult to do this synchronization as version 1 will not validate signatures
and creation version 2 manifests from version 1 registries will not have a
signature of the builder.

Attribution

Folks involved in this design so far
@dmcgowan @dmp42 @jlhawn @shykes and @vbatts

@thaJeztah
Member

Very good proposal. Adding more trust to Docker will become more and more important.

Since I'm not an expert on signing/validation, I've got some questions (excuse me if they're silly);

  • will it still be possible to run a v2 registry as a 'static' registry (i.e. static files/directories, with JSON files for meta data)
  • will it be possible to configure the client / daemon to only "trust" images signed with a specific (root) certificate? I.e. Only allow employees to build from images of trusted image maintainers?

One thing that still worries me is the "chain of trust" in the automated builds. Although the maintainer can be trusted, many Dockerfiles make use of external libraries / source files that are downloaded during build and could have been tampered with. I realise this is outside the scope of this proposal, but could be something to give some thought in the future.

@dmcgowan
Member

@thaJeztah provenance is as alluded to in the title going to be multiple steps and phases. The chain of trust, manifest, signature, and v2 registry are the core components being baked in now that will allow for further development of features such as what you mentioned about multiple certificates and requiring different types of certification of an image. The v2 registry api should allow serving 'static' files like you mentioned, but is this referring a registry that is pull only or just a file system implementation?

The initial "chain of trust" is about verifying an image is built by someone who has permission to build images of that name. Likewise future signatures could be added in future proposal to attest to other aspects of an image. Tracking all the sources which makes up a Docker build is being thought about, and content addressable layers is the first step toward that.

@thaJeztah
Member

but is this referring a registry that is pull only or just a file system implementation?

Basically, a simple read-only repository hosted using apache or nginx. I've seen this was possible with the current (v1) and was wondering if this was still possible after this change.

Thanks for explaining the "roadmap" for future additions on this matter. Thank you and the other contributors for taking security and trust at heart. LVGTM 😄

@dmp42
Member
dmp42 commented Sep 17, 2014

Basically, a simple read-only repository hosted using apache or nginx. I've seen this was possible with the current (v1)

I don't think it is possible with v1 - but we are aiming for it with v2 eventually.

@thaJeztah
Member

@dmp42 Just for reference; this is what I was talking about; #4607 and https://github.com/vbatts/d2r

However, let's not delve into static registries too much as it is only slightly related to this proposal and would only clutter up this discussion. 😺

@vbatts
Contributor
vbatts commented Sep 18, 2014

@thaJeztah static/read-only is expected to stay possible.

@tianon
Member
tianon commented Sep 18, 2014

wait, why the embedded signatures? why not just sign the whole file, .asc style, and just pass around both the manifest and any signatures associated with it?

@tianon
Member
tianon commented Sep 18, 2014

also, won't signing the whitespace too break in really fun ways? ie, space vs tab, \n vs \r\n, etc etc etc.

@tianon
Member
tianon commented Sep 18, 2014

does this also mean that image signatures can only be certificate-based and not leverage other existing systems of signing and trust like GPG that are already widely deployed and used?

@dmcgowan
Member

@tianon yes signing whitespace does allow for breaking in fun ways. The JWS spec gives us some protection against this by encoding the payload as base64. The "Pretty" version of it is both legible (not base64 encoded) and verifiable, but at the expense of it being fragile. I don't know if it is finalized which format will be used for the output, it is important for us to have a single file which includes the signature.

@dmcgowan
Member

We are using the JWS format for signatures, these signatures allow for x509 chain of trust verification or verification based solely off the public key. The initial verification of the namespace based on the signature will only use the public key, just like GPG-based trust.

See http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-4.1.3

@thaJeztah
Member

Not directly related, just a (possibly) interesting read: Announcing Keyless SSL (and the discussion on HackerNews

@tianon
Member
tianon commented Sep 18, 2014

So then shouldn't the signed version be completely extraneous whitespace-free?

@tianon
Member
tianon commented Sep 18, 2014

ie, {"a":"b","c":"d"} instead of { "a": "b", "c": "d" }

@tianon
Member
tianon commented Sep 18, 2014

I'm just reading through this and getting the feeling that it's really quite limiting. Maybe each of those "signature" blocks could have a "type" field to make sure there's space for future growth? (like maybe some kind of GPG backend too)

@dmcgowan
Member

As for whitespace, we are offering a format which is whitespace safe, but it is not preferred to use since it is not legible, just as formatting JSON without whitespace is illegible. I don't see a reason to assume that a signed file should be able to have any of its bytes changed (whitespace or not) and still be expected to pass signature validation.

I agree with a type field and I think we are still trying to figure out how multiple signatures will be handled. The GPG backend I think would come in later at verification. Right now we are thinking of libtrust more as a possible backend for GPG, rather than GPG a possible backend for libtrust.

@dmcgowan
Member

@thaJeztah it is an interesting read and although no directly related, there seems to be some similar motivations. The approach used in this proposal is very sensitive to the idea that builders should both be identified by their public key and solely responsible for their private key. At no point does the private key need to be shared with anyone else to build and verify builds.

@vbatts
Contributor
vbatts commented Sep 18, 2014

On Sep 18, 2014 5:21 PM, "Tianon Gravi" notifications@github.com wrote:

I'm just reading through this and getting the feeling that it's really
quite limiting. Maybe each of those "signature" blocks could have a "type"
field to make sure there's space for future growth? (like maybe some kind
of GPG backend too)

A type field trip s already in the works. I'll update the proposal. The
initial one would be builder/publisher, though there will be an attestor
and likely more.

@vbatts
Contributor
vbatts commented Sep 19, 2014

Also, there will be a directory like /etc/docker/ca.d/ for landing local root CAs. This way local builds of images for an enterprise can use an internal delegation for signing images and even promotion through their environments.
For instance, the image must be signed by a trusted builder, and QA can only run the image once also signed by the QA authority, and so on, such that in a "strict" mode Production would not accidentally run an image that had not be signed by a prod authority. Make sense?

@vbatts
Contributor
vbatts commented Sep 19, 2014

@dmcgowan @shykes I realize that one of the objectives is to have the docker binary be all inclusive, but including anything like a root CA in the binary is not going to be acceptable except for demos. A production environment will manage the distribution of trusted CAs land them on disk for the daemon to use.

@proppy
Contributor
proppy commented Sep 19, 2014

@vbatts about this part:

Content Addressable Layers

Each layer of an image will be referenced by a checksum created from its
contents. This checksum will be used on push and pull to verify contents have
not been tampered and disallow the layer referred to in the manifest to be
changed after signed.

Assuming "the content" of a layer is defined by the diff with its parent, do you plan to store attribute change (i.e: modification datetime) separately from actual content changes? So that touch'ing or re-ADD'ing a file doesn't end up duplicate it in the current layer?

(Happy to open or comment on another issue if that's outside of the scope of this proposal).

@vbatts
Contributor
vbatts commented Sep 19, 2014

On Sep 19, 2014 5:15 PM, "Johan Euphrosine" notifications@github.com
wrote:

@vbatts about this part:

Content Addressable Layers

Each layer of an image will be referenced by a checksum created from its
contents. This checksum will be used on push and pull to verify contents have
not been tampered and disallow the layer referred to in the manifest to be
changed after signed.

Assuming "the content" of a layer is defined by the diff with its parent, do you plan to store attribute change (i.e: modification datetime) separately from actual content changes? So that touch'ing or re-ADD'ing a file doesn't end up duplicate it in the current layer?

I intentionally left that a little vague. For the foreseeable future this
hash will be the TarSum of just the filesystem of the layer (not including
the json payload as is sometimes the case). This includes the contents of
the files that are diff'd from a layer's parents.

(Happy to open or comment on another issue if that's outside of the scope of this proposal).

We've been giving the tarsum calculation a lot of attention lately, so
depending on what extent you want to iron put the hashes of the layers, it
is completely relevant here. :-)

@dmcgowan
Member

@vbatts I think it makes sense to have a directory for root CAs. These CAs could either be checked by default, turned on/off individually, or a flag to use them all. The CA we talked about bundling I would not categorize as the same type of CA, since this root CA is used to enforce the global namespace, so it shouldn't differ between installations. Certain operations may be able to turn off enforcement or allow a work around (a local namespace), but the bundled CA should not be replaced with a different CA. How do you propose bundling the root CA if not in the binary, since the binary is the primary method of bundling and installing?

@tianon
Member
tianon commented Sep 22, 2014

Oh @vbatts you gem.

I'm -1 on CAs in general, but I see how that could be attractive. As long as there's an alternative to the rat's nest that x509 is, I'll be happy with this and might actually use it. If I can say "only allow my Docker engine to run image manifests that are signed/built/acked-by XYZ person (specified via some fully unambiguous and easily verified means like a full GPG fingerprint)", that'd be amazing.

I also see an absolute need to be able to disable the default CA, even if it can't be changed in any other way.

@dmcgowan
Member

@tianon the default root CA in this proposal is used is to verify that the builder has access to namespace of the image. The identity of the builder will be derived from the signature, which is created by the user's private key, just like GPG signatures. The verification is done on the fingerprint of the public key. Statements signed by a namespace authority and chaining back to the root will be able to be downloaded/imported which provide identity of public keys to user and users to namespace. Without these statements chaining back to a single root CA the trust graph will be unmanageable (like of web of trust) and difficult to tie in to the existing namespace. While x509 is not ideal and was not our first choice, it is a proven mechanism for extending authority in a secure and manageable way.

@mmdriley
Contributor

It's probably better to use a SHA256 of the layer file as it's stored (the "payload" hash in Docker client parlance) rather than the TarSum.

Two reasons:
(1) TarSum is super-custom and a spec for it hasn't been written as far as I can tell. This makes it really difficult to reason the algorithm or implementations of it correct. For example, in the current TarSum implementation, what happens if I put two instances of the same filename in a .tar file? I think one of them gets ignored.

(2) The storage layer (be it S3, GCS, or local) shouldn't need special insight into files in order to hash them.

@dmcgowan
Member

If I can say "only allow my Docker engine to run image manifests that are signed/built/acked-by XYZ person (specified via some fully unambiguous and easily verified means like a full GPG fingerprint)", that'd be amazing.

@tianon I think it would not be difficult in the future to extend signatures to do this off a GPG fingerprint. This verification would likely be done in addition to the namespace verification (except when using the local namespace). We would like to be able to do all this via the libtrust graph since it is already using fingerprints, an additional check would just need to be made (likely a "strict" mode) that the person who delegated to the public key has been granted access to build images for the engine it is installing to.

@tianon
Member
tianon commented Sep 22, 2014

That sounds like it's really complicated to set up for something that I'd
think would be one of the most common use cases for provenance, so maybe
I'm missing something in the flow. Do you think you could walk me through
the steps that'd be required to accomplish such a setup in the context of
this proposal? ie, I only want to run images that are built by John
Smith's key, which I have a copy of - where do I put it and how would I
configure my daemon to forcibly disallow keys from anything else? Do I
have to dance around with keychains, or is it just dropping files in the
right places and setting up some basic configuration?

@vbatts
Contributor
vbatts commented Sep 22, 2014

On Sep 22, 2014 5:27 PM, "mmdriley" notifications@github.com wrote:

It's probably better to use a SHA256 of the layer file as it's stored
(the "payload" hash in Docker client parlance) rather than the TarSum.

Two reasons:
(1) TarSum is super-custom and a spec for it hasn't been written as far
as I can tell. This makes it really difficult to reason the algorithm or
implementations of it correct. For example, in the current TarSum
implementation, what happens if I put two instances of the same filename in
a .tar file? I think one of them gets ignored.

(2) The storage layer (be it S3, GCS, or local) shouldn't need special
insight into files in order to hash them.


Reply to this email directly or view it on GitHub.

While i agree, the tar file is thrown away by a daemon when it pulls the
image. Therefore you could hardly reassemble a year archive again that
could have the same sha256sum. So then pushing the image to a local
registry would change the s check sum of the new tar produced. That is why
TarSum has its place in the world.
Now having a spec for it is a skills thing. At this point we have two
reference implementations, python and golang.

@dmcgowan
Member

@tianon The common use case is the builder registering their public key with a namespace authority using the docker login command. This allows installations to be restricted based on repo name, rather than specific keys. Since namespace authorities are queryable by installers, nothing other than the image bits should need to be exchanged between builder and installer. In the case where online querying on installation is not an option, the builder can export the signed statement which grants them authority to the namespace. This exported statement is fully verifiable and able to be imported by the user.

The case where involving a namespace authority is not possible or ideal, then the local namespace I mentioned could be used. The rules for this namespace have not been fully defined but will likely involve a different way of checking the identity of the signer, probably in a separate proposal. It sounds reasonable to restrict installation to the local namespace based solely on the key fingerprint, perhaps just asking the user if the key is trusted and caching the result in a file.

Do you think you could walk me through the steps that'd be required to accomplish such a setup in the context of this proposal?

The steps we are hoping are just docker login or tagging into the 'local' namespace plus normal import/export/push/pull.

ie, I only want to run images that are built by John Smith's key, which I have a copy of

We don't want the common case to be users dealing with public key files. For the case of restricting installation to a subset of images that have passed a criteria (such as been signed by a specified key, or verified by a specified x509 chain of trust), then I think it should be as easy as dropping a public key file or certificate chain into a directory. Then either configuring them to be all on by default or turn them on individually, nothing specific is being proposed here.

@wking
wking commented Sep 26, 2014

On Mon, Sep 22, 2014 at 02:17:08PM -0700, Derek McGowan wrote:

the default root CA in this proposal is used is to verify that the
builder has access to namespace of the image. … Statements signed
by a namespace authority and chaining back to the root will be able
to be downloaded/imported which provide identity of public keys to
user and users to namespace. Without these statements chaining back
to a single root CA the trust graph will be unmanageable (like of
web of trust) and difficult to tie in to the existing namespace.

Webs of trust are good, because they acknowledge that each user is
the root of their own trust graph. A namespace-wide CA isn't going to
be signing things on my build box. On the other hand, a given
maintainer isn't going to be signing things built using the Trusted
Build infrastructure (those should be signed with a Trusted Build
key). Do I trust the maintainer and their local build system? Maybe.
I might know them, or have a long history of correspondence with no
security issues. Do I trust the Trusted Build infrastructure? I
don't know it, but there's a growing history behind it apparently
doing the right thing and not injecting malicious code. Do I trust my
QA lead enough to accept signatures by anyone she trusts? How many
degrees of separation am I willing to accept? Do any of these trust
decisions have anything to do with a namespace-wide CA? I don't see
how.

@wking
wking commented Sep 26, 2014

On Fri, Sep 26, 2014 at 05:44:56AM -0700, W. Trevor King wrote:

Webs of trust are good, …

But just because I like them doesn't mean you have to ;). That's why
the #6070 approach lets you slot in whichever signing system you like
best.

@mmdriley
Contributor

The question of key management is important -- and I expect will continue to be discussed at length :) -- but I'd prefer not to let the TarSum vs. SHA256 thread drop just yet. Experience has shown that only the most braindead algorithms should be used when making trust decisions on arbitrary data. Things like normalizing filenames, sorting, parsing, and (de)compressing have no business in an integrity-verification codepath.

Two recent, high-profile examples of trying to do too much in integrity checks:
http://blogs.technet.com/b/srd/archive/2013/12/10/ms13-098-update-to-enhance-the-security-of-authenticode.aspx
https://www.imperialviolet.org/2014/09/26/pkcs1.html

I think we could establish a normative way of constructing the .tar from its constituent files such that the same files will reliably produce the same .tar[.gz]. That puts the complex logic on the trusted side of the integrity check. As one of this year's BlackHat talks opined, Postel was wrong -- we should be conservative in what we produce and in what we accept. :)

@dmp42 dmp42 referenced this issue in docker/docker-registry Oct 7, 2014
Closed

Registry next generation #612

@dmcgowan
Member
dmcgowan commented Oct 9, 2014

@dmp42 @vbatts Proposed changes are updating the blobSums key to be named "fsLayers" and making it a dictionary. The tarsum key will be in a field called "blobSum" under "fsLayers". Likewise making history a dictionary with the json under a "v1Compatibility" key. Any suggestions on how these keys should be named differently?

@dmp42
Member
dmp42 commented Oct 9, 2014

@shykes does that address the concerns you raised yesterday? If so we can move ahead with images signing if the format is frozen.

@wking wking referenced this issue in docker/docker-registry Oct 10, 2014
Closed

NG: irc hands-on meetings #622

@wking
wking commented Oct 10, 2014

On Wed, Sep 17, 2014 at 12:04:15PM -0700, Vincent Batts wrote:

"signatures": [
{
"header": {
"jwk": {
"crv": "P-256",
"kid": "LYRA:YAG2:QQKS:376F:QQXY:3UNK:SXH7:K6ES:Y5AU:XUN5:ZLVY:KBYL",
"kty": "EC",
"x": "Cu_UyxwLgHzE9rvlYSmvVdqYCXY42E9eNhBb0xNv0SQ",
"y": "zUsjWJkeKQ5tv7S-hl1Tg71cd-CqnrtiiLxSi6N_yc8"
},
"alg": "ES256"
},
"signature": "m3bgdBXZYRQ4ssAbrgj8Kjl7GNgrKQvmCSY-00yzQosKi-8UBrIRrn3Iu5alj82B6u_jNrkGCjEx3TxrfT1rig",
"protected": "eyJmb3JtYXRMZW5ndGgiOjYwNjMsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxNC0wOS0xMVQxNzoxNDozMFoifQ"
}
]

In docker-registry#622, @proppy was just asking if/how revocations
will work 1. To allow them with an audit trail, I'd propose a
separate unsigned field in the metadata 2:

"revoked-signatures": [
{
"reason": "explain why you revoked it",
"revoked-signature": <stuff that used to be in "signatures">,
"signatures": [list of signatures for this revocation] }
},

]

It would be hard to add this later if you bake in the current proposal
that signs everything except the current "signatures" entry (another
reason I prefer detached signatures, docker/docker#6070 ;).

@dmcgowan
Member

@wking the current plan for revocations is to have it be based by key and namespace, meaning there will be a revocation record for the signature key and image name. Most this logic is prototyped in libtrust today and we are working on a more concrete design proposal around it that will be easier to review than what we have here.

Revocations are currently part of the trust statements that are used to generate the keychain graph used for verification. You can see the prototyped statement object here in the libtrust codebase.
https://github.com/docker/libtrust/blob/master/trustgraph/statement.go

@wking wking referenced this issue in docker/docker-registry Oct 11, 2014
Closed

Adds prototype NG driver apis #630

@wking
wking commented Oct 11, 2014

On Fri, Oct 10, 2014 at 03:44:25PM -0700, Derek McGowan wrote:

the current plan for revocations is to have it be based by key and
namespace, meaning there will be a revocation record for the
signature key and image name.

So embedded signatures but detached revocations? Or are both embedded
and stored interleaved in the "signatures" entry?

@wking
wking commented Nov 7, 2014

On Wed, Sep 17, 2014 at 12:04:15PM -0700, Vincent Batts wrote:

"history": [
"{"id":"a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721","parent":"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16","created":"2014-06-05T00:05:35.990887725Z"...",
"{"id":"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16","parent":"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229","created":"2014-06-05T00:05:35.692528634Z"...",
"{"id":"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229","parent":"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158","created":"2014-06-05T00:05:35.589531476Z"...",
"{"id":"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158","comment":"Imported from -","created":"2013-06-13T14:03:50.821769-07:00"..."
]

History

For auditability and assurance of the image, there will be a
history section. This history will convey the life of the the
image (build steps, ancestry, prior attestations on parent images,
etc.). It will have a generic form, and it is important to note
that its content is included in the signed payload.

Can't we have content-addressable history and metadata too?

{
"name": "dmcgowan/test-image",
"tag": "latest",
"layer": "sha256:5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03",
signatures": […]
}

and have the layer image hash point to immutable layer metadata:

{
"parent": "sha256+71573b922a87abc3fd1a957f2cfa09d9e16998567dd878a85e12166112751806",
"layer": "tarsum+sha256:e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317",
"size": 0,
"architecture": "amd64",
"os": "linux",
"config": {
"Hostname": "f6f7ebf0f518",
"Cmd": […],

},

}

With ‘parent’ pointing to the parent layer's metadata (not set for
orphan layers) and ‘layer’ pointing to the layer's tarball. That's
how Git works with signed tags (instead of signed manifests) referring
to commits (instead of metadata), which in turn refer to their parent
commit and a tree (instead of a layer tarball).

This would give us almost the same level of security as flattening all
those references into a single manifest (just pick a crypto hash you
trust, for example, whatever you're using in your sigining algorithm
;). It would also let you share content without reproducing the layer
metadata in every manifest. For exmaple, if you have 1k images based
on library/debian:7.7, you won't need copies of the Debian metadata in
each of the consumer image's metadata. Maybe the idea is that we'll
be pushing intermediate layer tarballs without any associated
metadata:

  • final image metadata
  • final layer tarball
  • parent layer tarball
  • granparent layer tarball

But I'd guess the Docker registry is wider than it is deep, so the
savings you gain by not having to push metadata for ancestor images
(~one hash-length per layer) would be less than the space you lose
through duplication (~one image metadata per named or shared parent).

It also makes the bulk of the metadata immutable, which makes
implementing the registry easier (no need for fancy caching
calculations or ETags, you either have metadata
sha256+71573b922a87abc3fd1a957f2cfa09d9e16998567dd878a85e12166112751806
or you don't). The only mutable bits are the signed
name/tag/layer-metadata associations (because you want to be able to
force-push ‘latest’). Even if you weren't getting space saving from
hash-based linking, I think reducing the amount of mutable data would
be worth it.

On the other hand, perhaps we're already too far down the
flattened-manifest road to be worth considering this?

@mmdriley
Contributor

This discussion may not be apropos of provenance, but this issue is becoming the de facto reference for the V2 manifest format so I hope you'll forgive me for asking here.

Is there any world where Docker image references will be qualified by registry?

For example, say I run an instance of docker/docker-registry locally but I want users to chain their images back up to public images without needing to upload them to me or download them from me.

@shykes
Contributor
shykes commented Nov 30, 2014

Matthew, yes, that is the beauty of global content-addressable pool
of layers + a global federated namespace for manifests. Each Docker
installation could be configured with any number of backends for layer
discovery and transfer. So you can create a manifest which refers to layer
"foo" without needing to specify where to get it. Each user will simply
try all available backends in sequence, until one of the yields the desired
layer, or it returns an error saying "sorry, unknown layer".

On Saturday, November 29, 2014, Matthew Riley notifications@github.com
wrote:

This discussion may not be apropos of provenance, but this issue is
becoming the de facto reference for the V2 manifest format so I hope you'll
forgive me for asking here.

Is there any world where Docker image references will be qualified by
registry?

For example, say I run an instance of docker/docker-registry locally but I
want users to chain their images back up to public images without needing
to upload them to me or download them from me.


Reply to this email directly or view it on GitHub
#8093 (comment).

@wking
wking commented Dec 1, 2014

On Thu, Nov 06, 2014 at 08:12:02PM -0800, W. Trevor King wrote:

Can't we have content-addressable history and metadata too? …

On #docker-dev today we hit another use-case that fat-manifests make
difficult (docker/docker-registry#804 and 1 through 2): if you
like some else's image and want to preserve it for future reference,
how should you do that without lightweight tags? If you just use the
original author's tags, you risk them updating the mutable tag later
(or removing the tag entirely). If you re-tag the image, you lose
their signature. Possible approaches:

  • Use thin tags. For example, Alice pushes layer-metadata 3
    sha256:58… and a signed tag to alice/foo:123. You tag sha256:58… as
    bob/foo:123, and now have an immutable (because you choose not to
    update your tag) reference to that image.

  • Explicitly allow (in the API spec, #9015) folks to push the
    identical manifest with a different name. So fetching bob/foo:123
    would give you a manifest with:

    {
    "name": "alice/foo",
    "tag": "123",

    }

    Then you can preserve the original name and signature, but still
    trust the tag to be immutable and persistent (because you choose not
    to update or remove the copy in bob/foo:123). This would work, but
    without a registry endpoint to enable this efficiently, folks
    tagging their own copy would have to download the full image and
    re-push it to prove they were authorized to access the contet.

  • Don't worry about preserving the original signature. This seems to
    defeat the purpose of signatures.

  • Don't provide a means to guarantee immutable future access to a tag.

As a related point, the fat manifests make the definition of aliases
(#8141) awkward 4. How important are they going forward?

@wking wking referenced this issue in docker/docker-registry Dec 2, 2014
Closed

NG: access image via immutable identifier #804

@sourav82
sourav82 commented Dec 3, 2014

Could you please let me know how to implement signed image on private registry?

I've a private registry and want to build an image from a Dockerfile. Wanted to make that image signed and push it to private registry. Need help to implement this.

@dmp42
Member
dmp42 commented Dec 4, 2014

@sourav82 this is a proposal for an upcoming feature. Right now, Docker is not (yet) able to sign images - but people are currently working on this.

@sourav82
sourav82 commented Dec 4, 2014

Thanks dmp42 for your clarification. While going through the link http://blog.docker.com/2014/10/docker-1-3-signed-images-process-injection-security-options-mac-shared-directories/ for Docker 1.3.0 release, I was in the impression it has released the feature. Anyway, thanks a lot.

@wking wking added a commit to wking/docker that referenced this issue Dec 10, 2014
@wking wking registry.json: Add manifest schema, with layerMetadata and signature
From [1]:

> For reference, the relevant manifest fields for the registry are the
> following:
>
> field      description
> name       The name of the image.
> tag        The tag for this version of the image.
> fsLayers   A list of layer descriptors (including tarsum)
> signature  A JWS used to verify the manifest content
>
> For more information about the manifest format, please see
> docker/docker#8093.

And from [2]:

> Image Manifest
>
> The image manifest file will contain all the information which is
> needed to pull, install, validate and run an image. It will contain
> a list of layers by a content addressable id, history, run time
> configuration, and signatures.  This manifest is generated by the
> daemon. Initially this generation will happen when an image is
> published, and ultimately happen anytime an image is built or
> committed. Each manifest is required to be signed by the client
> creating the manifest on push or build with additional signatures
> which can be added post build to verify the quality of the manifest
> or validity of the builder. The history will contain fully backward
> compatible metadata to allow old style layer and metadata to be
> recreated from the manifest.
>
> Signable manifest (or payload) refers to portions of the manifest
> which will be signed by builder. The signable manifest is a JSON
> dictionary containing the layers, run configuration, and
> history. The entire signable manifest will signed, any changes
> including whitespace will require a new signature. To aid in
> readability the signable manifest should be well-formatted JSON.
>
> Example: (totally subject to change)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "fsLayers": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>  ],
>  "history": [
>    "{\"id\":\"a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721\",\"parent\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"created\":\"2014-06-05T00:05:35.990887725Z\"...",
>    "{\"id\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"parent\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"created\":\"2014-06-05T00:05:35.692528634Z\"...",
>    "{\"id\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"parent\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"created\":\"2014-06-05T00:05:35.589531476Z\"...",
>    "{\"id\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"comment\":\"Imported from -\",\"created\":\"2013-06-13T14:03:50.821769-07:00\"..."
>  ],
>  "schemaVersion": 1
> }
>
> Signed manifest refers to a manifest which includes the signature as
> well as the signable manifest. The signed manifest could be
> represented as either a JSON Web Signature (JSON serialization, see
> link), in which the payload is the base64 encoded signed manifest,
> or an altered version of the signed manifest JSON to include the
> signature as the last element of the JSON dictionary and a record of
> the alternations to the original signed manifest included in the
> signature. EIther format is fully verifiable and tamper-proof.
>
> http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
>
> Example: (human readable format)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "blobSums": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>   ],
>   "history": ["v1 compatible string encoded json for each layer"],
>   "schemaVersion": 1,
>   "signatures": [
>     {
>       "header": {
>         "jwk": {
>           "crv": "P-256",
>           "kid": "LYRA:YAG2:QQKS:376F:QQXY:3UNK:SXH7:K6ES:Y5AU:XUN5:ZLVY:KBYL",
>           "kty": "EC",
>           "x": "Cu_UyxwLgHzE9rvlYSmvVdqYCXY42E9eNhBb0xNv0SQ",
>           "y": "zUsjWJkeKQ5tv7S-hl1Tg71cd-CqnrtiiLxSi6N_yc8"
>         },
>         "alg": "ES256"
>       },
>       "signature": "m3bgdBXZYRQ4ssAbrgj8Kjl7GNgrKQvmCSY-00yzQosKi-8UBrIRrn3Iu5alj82B6u_jNrkGCjEx3TxrfT1rig",
>       "protected": "eyJmb3JtYXRMZW5ndGgiOjYwNjMsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxNC0wOS0xMVQxNzoxNDozMFoifQ"
>     }
>   ]
> }

I didn't delve too deeply into the JWS spec [3], just enough to get:

> The following members are defined for use in top-level JSON objects
> used for the JWS JSON Serialization:
>
> payload
>    The "payload" member MUST be present and contain the value
>    BASE64URL(JWS Payload).
> signatures
>    The "signatures" member value MUST be an array of JSON objects.
>    Each object represents a signature or MAC over the JWS Payload and
>    the JWS Protected Header.
>
> The following members are defined for use in the JSON objects that
> are elements of the "signatures" array:
>
> protected
>    The "protected" member MUST be present and contain the value
>    BASE64URL(UTF8(JWS Protected Header)) when the JWS Protected
>    Header value is non-empty; otherwise, it MUST be absent.  These
>    Header Parameter values are integrity protected.
> header
>    The "header" member MUST be present and contain the value JWS
>    Unprotected Header when the JWS Unprotected Header value is non-
>    empty; otherwise, it MUST be absent.  This value is represented as
>    an unencoded JSON object, rather than as a string.  These Header
>    Parameter values are not integrity protected.
> signature
>    The "signature" member MUST be present and contain the value
>    BASE64URL(JWS Signature).
>
> At least one of the "protected" and "header" members MUST be present
> for each signature/MAC computation so that an "alg" Header Parameter
> value is conveyed.

There are some inconsistencies in the Docker issues that I've
arbitrated:

* [1] has 'signature', but [2] has 'signatures'.  I expect this was
  just a typo in [1], since having an array of signatures is mentioned
  multiple times in [2], and nobody suggests having only a single
  signature.

* [2] has 'fsLayers' holding 'blobSum's in the signable manifest, but
  'blobSums' holding 'blobSum's in the signed manifest.  I expect they
  meant 'fsLayers' in both cases, because 'blobSums' is too generic,
  and [1] mentions 'fsLayers'.

I've left the other manifest fields ('architecture', 'history',
'schemaVersion', ...) unspecified, since that's all independent of the
registry API.

[1]: docker#9015
[2]: docker#8093
[3]: http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
f083eb1
@wking wking added a commit to wking/docker that referenced this issue Dec 11, 2014
@wking wking registry.json: Add manifest schema, with layerMetadata and signature
From [1]:

> For reference, the relevant manifest fields for the registry are the
> following:
>
> field      description
> name       The name of the image.
> tag        The tag for this version of the image.
> fsLayers   A list of layer descriptors (including tarsum)
> signature  A JWS used to verify the manifest content
>
> For more information about the manifest format, please see
> docker/docker#8093.

And from [2]:

> Image Manifest
>
> The image manifest file will contain all the information which is
> needed to pull, install, validate and run an image. It will contain
> a list of layers by a content addressable id, history, run time
> configuration, and signatures.  This manifest is generated by the
> daemon. Initially this generation will happen when an image is
> published, and ultimately happen anytime an image is built or
> committed. Each manifest is required to be signed by the client
> creating the manifest on push or build with additional signatures
> which can be added post build to verify the quality of the manifest
> or validity of the builder. The history will contain fully backward
> compatible metadata to allow old style layer and metadata to be
> recreated from the manifest.
>
> Signable manifest (or payload) refers to portions of the manifest
> which will be signed by builder. The signable manifest is a JSON
> dictionary containing the layers, run configuration, and
> history. The entire signable manifest will signed, any changes
> including whitespace will require a new signature. To aid in
> readability the signable manifest should be well-formatted JSON.
>
> Example: (totally subject to change)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "fsLayers": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>  ],
>  "history": [
>    "{\"id\":\"a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721\",\"parent\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"created\":\"2014-06-05T00:05:35.990887725Z\"...",
>    "{\"id\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"parent\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"created\":\"2014-06-05T00:05:35.692528634Z\"...",
>    "{\"id\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"parent\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"created\":\"2014-06-05T00:05:35.589531476Z\"...",
>    "{\"id\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"comment\":\"Imported from -\",\"created\":\"2013-06-13T14:03:50.821769-07:00\"..."
>  ],
>  "schemaVersion": 1
> }
>
> Signed manifest refers to a manifest which includes the signature as
> well as the signable manifest. The signed manifest could be
> represented as either a JSON Web Signature (JSON serialization, see
> link), in which the payload is the base64 encoded signed manifest,
> or an altered version of the signed manifest JSON to include the
> signature as the last element of the JSON dictionary and a record of
> the alternations to the original signed manifest included in the
> signature. EIther format is fully verifiable and tamper-proof.
>
> http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
>
> Example: (human readable format)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "blobSums": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>   ],
>   "history": ["v1 compatible string encoded json for each layer"],
>   "schemaVersion": 1,
>   "signatures": [
>     {
>       "header": {
>         "jwk": {
>           "crv": "P-256",
>           "kid": "LYRA:YAG2:QQKS:376F:QQXY:3UNK:SXH7:K6ES:Y5AU:XUN5:ZLVY:KBYL",
>           "kty": "EC",
>           "x": "Cu_UyxwLgHzE9rvlYSmvVdqYCXY42E9eNhBb0xNv0SQ",
>           "y": "zUsjWJkeKQ5tv7S-hl1Tg71cd-CqnrtiiLxSi6N_yc8"
>         },
>         "alg": "ES256"
>       },
>       "signature": "m3bgdBXZYRQ4ssAbrgj8Kjl7GNgrKQvmCSY-00yzQosKi-8UBrIRrn3Iu5alj82B6u_jNrkGCjEx3TxrfT1rig",
>       "protected": "eyJmb3JtYXRMZW5ndGgiOjYwNjMsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxNC0wOS0xMVQxNzoxNDozMFoifQ"
>     }
>   ]
> }

I didn't delve too deeply into the JWS spec [3], just enough to get:

> The following members are defined for use in top-level JSON objects
> used for the JWS JSON Serialization:
>
> payload
>    The "payload" member MUST be present and contain the value
>    BASE64URL(JWS Payload).
> signatures
>    The "signatures" member value MUST be an array of JSON objects.
>    Each object represents a signature or MAC over the JWS Payload and
>    the JWS Protected Header.
>
> The following members are defined for use in the JSON objects that
> are elements of the "signatures" array:
>
> protected
>    The "protected" member MUST be present and contain the value
>    BASE64URL(UTF8(JWS Protected Header)) when the JWS Protected
>    Header value is non-empty; otherwise, it MUST be absent.  These
>    Header Parameter values are integrity protected.
> header
>    The "header" member MUST be present and contain the value JWS
>    Unprotected Header when the JWS Unprotected Header value is non-
>    empty; otherwise, it MUST be absent.  This value is represented as
>    an unencoded JSON object, rather than as a string.  These Header
>    Parameter values are not integrity protected.
> signature
>    The "signature" member MUST be present and contain the value
>    BASE64URL(JWS Signature).
>
> At least one of the "protected" and "header" members MUST be present
> for each signature/MAC computation so that an "alg" Header Parameter
> value is conveyed.

There are some inconsistencies in the Docker issues that I've
arbitrated:

* [1] has 'signature', but [2] has 'signatures'.  I expect this was
  just a typo in [1], since having an array of signatures is mentioned
  multiple times in [2], and nobody suggests having only a single
  signature.  (Sephen confirmed this [4]).

* [2] has 'fsLayers' holding 'blobSum's in the signable manifest, but
  'blobSums' holding 'blobSum's in the signed manifest.  I expect they
  meant 'fsLayers' in both cases, because 'blobSums' is too generic,
  and [1] mentions 'fsLayers'.

I've left the other manifest fields ('architecture', 'history',
'schemaVersion', ...) unspecified, since that's all independent of the
registry API.

[1]: docker#9015
[2]: docker#8093
[3]: http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
[4]: f083eb1#commitcomment-8922939
e7e84a6
@wking wking added a commit to wking/docker that referenced this issue Dec 11, 2014
@wking wking registry.json: Add manifest schema, with layerMetadata and signature
From [1]:

> For reference, the relevant manifest fields for the registry are the
> following:
>
> field      description
> name       The name of the image.
> tag        The tag for this version of the image.
> fsLayers   A list of layer descriptors (including tarsum)
> signature  A JWS used to verify the manifest content
>
> For more information about the manifest format, please see
> docker/docker#8093.

And from [2]:

> Image Manifest
>
> The image manifest file will contain all the information which is
> needed to pull, install, validate and run an image. It will contain
> a list of layers by a content addressable id, history, run time
> configuration, and signatures.  This manifest is generated by the
> daemon. Initially this generation will happen when an image is
> published, and ultimately happen anytime an image is built or
> committed. Each manifest is required to be signed by the client
> creating the manifest on push or build with additional signatures
> which can be added post build to verify the quality of the manifest
> or validity of the builder. The history will contain fully backward
> compatible metadata to allow old style layer and metadata to be
> recreated from the manifest.
>
> Signable manifest (or payload) refers to portions of the manifest
> which will be signed by builder. The signable manifest is a JSON
> dictionary containing the layers, run configuration, and
> history. The entire signable manifest will signed, any changes
> including whitespace will require a new signature. To aid in
> readability the signable manifest should be well-formatted JSON.
>
> Example: (totally subject to change)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "fsLayers": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>  ],
>  "history": [
>    "{\"id\":\"a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721\",\"parent\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"created\":\"2014-06-05T00:05:35.990887725Z\"...",
>    "{\"id\":\"120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16\",\"parent\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"created\":\"2014-06-05T00:05:35.692528634Z\"...",
>    "{\"id\":\"42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229\",\"parent\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"created\":\"2014-06-05T00:05:35.589531476Z\"...",
>    "{\"id\":\"511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158\",\"comment\":\"Imported from -\",\"created\":\"2013-06-13T14:03:50.821769-07:00\"..."
>  ],
>  "schemaVersion": 1
> }
>
> Signed manifest refers to a manifest which includes the signature as
> well as the signable manifest. The signed manifest could be
> represented as either a JSON Web Signature (JSON serialization, see
> link), in which the payload is the base64 encoded signed manifest,
> or an altered version of the signed manifest JSON to include the
> signature as the last element of the JSON dictionary and a record of
> the alternations to the original signed manifest included in the
> signature. EIther format is fully verifiable and tamper-proof.
>
> http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
>
> Example: (human readable format)
>
> {
>   "name": "dmcgowan/test-image",
>   "tag": "latest",
>   "architecture": "amd64",
>   "blobSums": [
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:cea0d2071b01b0a79aa4a05ea56ab6fdf3fafa03369d9f4eea8d46ea33c43e5f",
>     },
>     {
>       "blobSum": "tarsum+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
>     },
>     {
>       "blobSum": "tarsum+sha256:2a7812e636235448785062100bb9103096aa6655a8f6bb9ac9b13fe8290f66df"
>     }
>   ],
>   "history": ["v1 compatible string encoded json for each layer"],
>   "schemaVersion": 1,
>   "signatures": [
>     {
>       "header": {
>         "jwk": {
>           "crv": "P-256",
>           "kid": "LYRA:YAG2:QQKS:376F:QQXY:3UNK:SXH7:K6ES:Y5AU:XUN5:ZLVY:KBYL",
>           "kty": "EC",
>           "x": "Cu_UyxwLgHzE9rvlYSmvVdqYCXY42E9eNhBb0xNv0SQ",
>           "y": "zUsjWJkeKQ5tv7S-hl1Tg71cd-CqnrtiiLxSi6N_yc8"
>         },
>         "alg": "ES256"
>       },
>       "signature": "m3bgdBXZYRQ4ssAbrgj8Kjl7GNgrKQvmCSY-00yzQosKi-8UBrIRrn3Iu5alj82B6u_jNrkGCjEx3TxrfT1rig",
>       "protected": "eyJmb3JtYXRMZW5ndGgiOjYwNjMsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxNC0wOS0xMVQxNzoxNDozMFoifQ"
>     }
>   ]
> }

I didn't delve too deeply into the JWS spec [3], just enough to get:

> The following members are defined for use in top-level JSON objects
> used for the JWS JSON Serialization:
>
> payload
>    The "payload" member MUST be present and contain the value
>    BASE64URL(JWS Payload).
> signatures
>    The "signatures" member value MUST be an array of JSON objects.
>    Each object represents a signature or MAC over the JWS Payload and
>    the JWS Protected Header.
>
> The following members are defined for use in the JSON objects that
> are elements of the "signatures" array:
>
> protected
>    The "protected" member MUST be present and contain the value
>    BASE64URL(UTF8(JWS Protected Header)) when the JWS Protected
>    Header value is non-empty; otherwise, it MUST be absent.  These
>    Header Parameter values are integrity protected.
> header
>    The "header" member MUST be present and contain the value JWS
>    Unprotected Header when the JWS Unprotected Header value is non-
>    empty; otherwise, it MUST be absent.  This value is represented as
>    an unencoded JSON object, rather than as a string.  These Header
>    Parameter values are not integrity protected.
> signature
>    The "signature" member MUST be present and contain the value
>    BASE64URL(JWS Signature).
>
> At least one of the "protected" and "header" members MUST be present
> for each signature/MAC computation so that an "alg" Header Parameter
> value is conveyed.

There are some inconsistencies in the Docker issues that I've
arbitrated:

* [1] has 'signature', but [2] has 'signatures'.  I expect this was
  just a typo in [1], since having an array of signatures is mentioned
  multiple times in [2], and nobody suggests having only a single
  signature.  (Stephen confirmed this [4]).

* [2] has 'fsLayers' holding 'blobSum's in the signable manifest, but
  'blobSums' holding 'blobSum's in the signed manifest.  I expect they
  meant 'fsLayers' in both cases, because 'blobSums' is too generic,
  and [1] mentions 'fsLayers'.

I've left the other manifest fields ('architecture', 'history',
'schemaVersion', ...) unspecified, since that's all independent of the
registry API.

[1]: docker#9015
[2]: docker#8093
[3]: http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2
[4]: f083eb1#commitcomment-8922939
668c764
@krnowak krnowak added a commit to endocode/docker that referenced this issue Feb 11, 2015
@krnowak @alban krnowak + alban aci: initial support of the App Container Specification
This patch is the initial proposal to add support for the App Container
Specification [1].

This adds support for the App Container Discovery protocol:
  # docker pull --format aci coreos.com/etcd:v2.0.0

The ACI can also be pulled from the local filesystem or by http:
  # docker pull --format aci file:/tmp/aciexamples-1-linux-amd64.aci
  # docker pull --format aci
  # http://example.com/aciexamples-1-linux-amd64.aci

The ACI is then stored in the Docker graph:
  # ls -l
  # /var/lib/docker/graph/12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27/
  total 1992
  -rw-r--r-- 1 root root 2034176 Feb 10 16:33 layer.tar
  -rw-r--r-- 1 root root     334 Feb 10 16:33 manifest
  #

And the images can be listed:
  # docker images
  REPOSITORY                             TAG                 IMAGE ID
CREATED               VIRTUAL SIZE
  google/golang                          latest
09cf064bdd4f        8 weeks ago           516.9 MB
  ACI: coreos.com/etcd
66da439421e1        45.143744 years ago   -1 B
  ACI: github.com/endocode/aciexamples
12331d9888ec        45.143744 years ago   -1 B
  #

An ACI container can be started with one of those methods:
  # docker run --format aci github.com/endocode/aciexamples
  # docker run --format aci
  # 12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27
  # docker run --format aci 12331d9888ec

Not supported yet:
- ACI with dependencies
- Isolators
- Tags

[1] App Container Specification
https://github.com/appc/spec/blob/master/SPEC.md

Other links of interest:

[] Proposal for a Docker image format v2
docker#8093

[] Docker Image Specification v1.0.0
https://github.com/docker/docker/blob/master/image/spec/v1.md
29ba98a
@krnowak krnowak added a commit to endocode/docker that referenced this issue Feb 11, 2015
@krnowak @alban krnowak + alban aci: initial support of the App Container Specification
This patch is the initial proposal to add support for the App Container
Specification [1].

This adds support for the App Container Discovery protocol:
  # docker pull --format aci coreos.com/etcd:v2.0.0

The ACI can also be pulled from the local filesystem or by http:
  # docker pull --format aci file:/tmp/aciexamples-1-linux-amd64.aci
  # docker pull --format aci
  # http://example.com/aciexamples-1-linux-amd64.aci

The ACI is then stored in the Docker graph:
  # ls -l
  # /var/lib/docker/graph/12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27/
  total 1992
  -rw-r--r-- 1 root root 2034176 Feb 10 16:33 layer.tar
  -rw-r--r-- 1 root root     334 Feb 10 16:33 manifest
  #

And the images can be listed:
  # docker images
  REPOSITORY                             TAG                 IMAGE ID
CREATED               VIRTUAL SIZE
  google/golang                          latest
09cf064bdd4f        8 weeks ago           516.9 MB
  ACI: coreos.com/etcd
66da439421e1        45.143744 years ago   -1 B
  ACI: github.com/endocode/aciexamples
12331d9888ec        45.143744 years ago   -1 B
  #

An ACI container can be started with one of those methods:
  # docker run --format aci github.com/endocode/aciexamples
  # docker run --format aci
  # 12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27
  # docker run --format aci 12331d9888ec

It is also possible to use the commands create/start:
  # docker create --format aci github.com/endocode/aciexamples
  b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc
  # docker start -i -a b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc

Not supported yet:
- ACI with dependencies
- Isolators
- Tags

[1] App Container Specification
https://github.com/appc/spec/blob/master/SPEC.md

Other links of interest:

[] Proposal for a Docker image format v2
docker#8093

[] Docker Image Specification v1.0.0
https://github.com/docker/docker/blob/master/image/spec/v1.md
1316c4f
@alban alban added a commit to endocode/docker that referenced this issue Feb 11, 2015
@alban alban aci: initial support of the App Container Specification
This patch is the initial proposal to add support for the App Container
Specification [1].

This adds support for the App Container Discovery protocol:
  # docker pull --format aci coreos.com/etcd:v2.0.0

The ACI can also be pulled from the local filesystem or by http:
  # docker pull --format aci file:/tmp/aciexamples-1-linux-amd64.aci
  # docker pull --format aci
  # http://example.com/aciexamples-1-linux-amd64.aci

The ACI is then stored in the Docker graph:
  # ls -l
  # /var/lib/docker/graph/12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27/
  total 1992
  -rw-r--r-- 1 root root 2034176 Feb 10 16:33 layer.tar
  -rw-r--r-- 1 root root     334 Feb 10 16:33 manifest
  #

And the images can be listed:
  # docker images
  REPOSITORY                             TAG                 IMAGE ID
CREATED               VIRTUAL SIZE
  google/golang                          latest
09cf064bdd4f        8 weeks ago           516.9 MB
  ACI: coreos.com/etcd
66da439421e1        45.143744 years ago   -1 B
  ACI: github.com/endocode/aciexamples
12331d9888ec        45.143744 years ago   -1 B
  #

An ACI container can be started with one of those methods:
  # docker run --format aci github.com/endocode/aciexamples
  # docker run --format aci
  # 12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27
  # docker run --format aci 12331d9888ec

It is also possible to use the commands create/start:
  # docker create --format aci github.com/endocode/aciexamples
  b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc
  # docker start -i -a
  # b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc

Not supported yet:
- ACI with dependencies
- Isolators
- Tags

[1] App Container Specification
https://github.com/appc/spec/blob/master/SPEC.md

Other links of interest:

[] Proposal for a Docker image format v2
docker#8093

[] Docker Image Specification v1.0.0
https://github.com/docker/docker/blob/master/image/spec/v1.md
4490419
@alban alban added a commit to endocode/docker that referenced this issue Feb 11, 2015
@alban alban aci: initial support of the App Container Specification
This patch is the initial proposal to add support for the App Container
Specification [1].

This adds support for the App Container Discovery protocol:
  # docker pull --format aci coreos.com/etcd:v2.0.0

The ACI can also be pulled from the local filesystem or by http:
  # docker pull --format aci file:/tmp/aciexamples-1-linux-amd64.aci
  # docker pull --format aci http://example.com/aciexamples-1-linux-amd64.aci

The ACI is then stored in the Docker graph:
  # ls -l /var/lib/docker/graph/12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27/
  total 1992
  -rw-r--r-- 1 root root 2034176 Feb 10 16:33 layer.tar
  -rw-r--r-- 1 root root     334 Feb 10 16:33 manifest
  #

And the images can be listed:
  # docker images
  REPOSITORY                             TAG                 IMAGE ID            CREATED               VIRTUAL SIZE
  google/golang                          latest              09cf064bdd4f        8 weeks ago           516.9 MB
  ACI: coreos.com/etcd                                       66da439421e1        45.143744 years ago   -1 B
  ACI: github.com/endocode/aciexamples                       12331d9888ec        45.143744 years ago   -1 B
  #

An ACI container can be started with one of those methods:
  # docker run --format aci github.com/endocode/aciexamples
  # docker run --format aci 12331d9888ecd95b15c2fdae7050e50e3d3c446df83224abb86d433a2568cc27
  # docker run --format aci 12331d9888ec

It is also possible to use the commands create/start:
  # docker create --format aci github.com/endocode/aciexamples
  b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc
  # docker start -i -a b2eb6289ced6506316b31daac101e3f3143f11ff96d046aca3338c9b9de934dc

Not supported yet:
- ACI with dependencies
- Isolators
- Tags

[1] App Container Specification
https://github.com/appc/spec/blob/master/SPEC.md

Other links of interest:

[] Proposal for a Docker image format v2
docker#8093

[] Docker Image Specification v1.0.0
https://github.com/docker/docker/blob/master/image/spec/v1.md
0febbd5
@stevvooe stevvooe referenced this issue in docker/distribution Feb 18, 2015
Closed

doc/spec: outline manifest v2, schema version 1 #183

@jessfraz jessfraz added feature and removed feature labels Feb 26, 2015
@vbatts
Contributor
vbatts commented Jun 8, 2015

Since this issue was initially opened there have been huge strides made in a
content addressable Docker image format.

This provisional image manifest allows for a more abstracted implementation and
will act as a compatibility layer for v1 images. Though as we refine multi-arch
images, federated image layers, content addressable references and fully
qualified docker images, the work has already begun for further refining this v2
image manifest
.

I am closing the step 1 issue as complete. Further work will continue in the
distribution workgroup and eventually the security/trust workgroup.

@vbatts vbatts closed this Jun 8, 2015
@xuedihualu

Hi:
how can i delete images from registry2,when i curl -v -X GET registry.com:5000/v2/centos/6.6

  • About to connect() to registry.com port 5000 (#0)
  • Trying 10.168.89.24...
  • Connected to registry.com (10.168.89.24) port 5000 (#0)

    GET /v2/centos/6.6 HTTP/1.1
    User-Agent: curl/7.29.0
    Host: registry.com:5000
    Accept: /

    < HTTP/1.1 404 Not Found
    < Content-Type: text/plain; charset=utf-8
    < Docker-Distribution-Api-Version: registry/2.0
    < Date: Tue, 13 Oct 2015 06:04:20 GMT
    < Content-Length: 19
    <
    404 page not found
@thaJeztah
Member

@xiekeyang i noticed you asked the same question in docker/distribution#1091 (comment). The GitHub issue trackers are not a support forum, Could you ask your question on;

  1. the forums: https://forums.docker.com/
  2. the docker-user mailing list https://groups.google.com/forum/#!forum/docker-user
  3. the #docker channel on Freenode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment