docs: proposal for CNAB Architecture

[bacongobbler]

We should determine the basic underlying architecture/layout of a CNAB manifest and document it. The architecture should be able to handle all the existing example use cases:

#11
#12
#13
#14
#15
#16
#21

relatedly, it should also present a proposal for how a CNAB manifest is addressed in storage via #5 and #6.

[bacongobbler]

Abstract

The Container-Native Application Bundle is a container that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

A CNAB is used as the first root container that your machine runs, similar to how the kernel first boots an initramfs to mount filesystems in user space. It is used for bootstrapping containers which has your application's runtime, system tools, libraries and settings, along with any other extra configuration required to run your application. The CNAB carries the modules needed for booting your containers.

The idea is to have a "user space" container to configure and boot the bundle appropriately. the bundle plays as a mediator between your application containers, any configuration for external tooling and the execution runtime, giving users a temporary container which bears the user space utilities needed for running your entire application.

Proposal Summary

Container-Native Application Bundles are self-describing manifests containing a list of containers, content-addressable layers for the "init" container, run configuration, and signatures to identify and verify the bundle meets the expectations of the installer.

CNAB Manifest

The CNAB manifest file will contain all the information which is needed to pull, install, validate and run a bundle. It will contain a list of images for the initcontainer and dependent containers using a content addressable ID, and signatures. This manifest is generated by duffle build. Initially, this generation will happen anytime a bundle is built. Each manifest may be signed by the client creating the manifest on duffle build with additional signatures, which can be added post-build to verify the quality of the manifest or validity of the builder.

The payload of the manifest file is a JSON dictionary containing the digest of the initcontainer and image digests of the dependent containers. The entire payload will be signed, and any changes including whitespace will require a new signature. The signable manifest should be well-formatted JSON. The signature could be represented as either a JSON Web Signature, in which the payload is the base64 encoded payload. This ensures the format is fully verifiable and tamper-proof.

http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31#section-7.2

For the POC, the manifest can be generated by the docker manifest create command, then hand-written to follow the spec:

Example:

{
    "schemaVersion": 1,
    "mediaType": "application/cnab.v1alpha+json",
    "initcontainer": {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 424,
            "digest": "sha256:f67dcc5fc786f04f0743abfe0ee5dae9bd8caf8efa6c8144f7f2a43889dc513b",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
    },
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 424,
            "digest": "sha256:b64ca0b60356a30971f098c92200b1271257f100a55b351e6bbe985638352f3a",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 425,
            "digest": "sha256:df436846483aff62bad830b730a0d3b77731bcf98ba5e470a8bbb8e9e346e4e8",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

In this example, the signature would be generated using the initcontainer and manifests sections of the above manifest, encoded as base64.

In other words, the base64 value of the above example would be

{
    "initcontainer": {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 424,
            "digest": "sha256:f67dcc5fc786f04f0743abfe0ee5dae9bd8caf8efa6c8144f7f2a43889dc513b",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
    },
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 424,
            "digest": "sha256:b64ca0b60356a30971f098c92200b1271257f100a55b351e6bbe985638352f3a",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 425,
            "digest": "sha256:df436846483aff62bad830b730a0d3b77731bcf98ba5e470a8bbb8e9e346e4e8",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

Which is then translated to base64:

ewogICAgImluaXRjb250YWluZXIiOiB7CiAgICAgICAgIm1lZGlhVHlwZSI6ICJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwKICAgICAgICAgICAgInNpemUiOiA0MjQsCiAgICAgICAgICAgICJkaWdlc3QiOiAic2hhMjU2OmY2N2RjYzVmYzc4NmYwNGYwNzQzYWJmZTBlZTVkYWU5YmQ4Y2FmOGVmYTZjODE0NGY3ZjJhNDM4ODlkYzUxM2IiLAogICAgICAgICAgICAicGxhdGZvcm0iOiB7CiAgICAgICAgICAgICAgICAiYXJjaGl0ZWN0dXJlIjogImFtZDY0IiwKICAgICAgICAgICAgICAgICJvcyI6ICJsaW51eCIKICAgICAgICAgICAgfQogICAgfSwKICAgICJtYW5pZmVzdHMiOiBbCiAgICAgICAgewogICAgICAgICAgICAibWVkaWFUeXBlIjogImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuZGlzdHJpYnV0aW9uLm1hbmlmZXN0LnYyK2pzb24iLAogICAgICAgICAgICAic2l6ZSI6IDQyNCwKICAgICAgICAgICAgImRpZ2VzdCI6ICJzaGEyNTY6YjY0Y2EwYjYwMzU2YTMwOTcxZjA5OGM5MjIwMGIxMjcxMjU3ZjEwMGE1NWIzNTFlNmJiZTk4NTYzODM1MmYzYSIsCiAgICAgICAgICAgICJwbGF0Zm9ybSI6IHsKICAgICAgICAgICAgICAgICJhcmNoaXRlY3R1cmUiOiAiYW1kNjQiLAogICAgICAgICAgICAgICAgIm9zIjogImxpbnV4IgogICAgICAgICAgICB9CiAgICAgICAgfSwKICAgICAgICB7CiAgICAgICAgICAgICJtZWRpYVR5cGUiOiAiYXBwbGljYXRpb24vdm5kLmRvY2tlci5kaXN0cmlidXRpb24ubWFuaWZlc3QudjIranNvbiIsCiAgICAgICAgICAgICJzaXplIjogNDI1LAogICAgICAgICAgICAiZGlnZXN0IjogInNoYTI1NjpkZjQzNjg0NjQ4M2FmZjYyYmFkODMwYjczMGEwZDNiNzc3MzFiY2Y5OGJhNWU0NzBhOGJiYjhlOWUzNDZlNGU4IiwKICAgICAgICAgICAgInBsYXRmb3JtIjogewogICAgICAgICAgICAgICAgImFyY2hpdGVjdHVyZSI6ICJhbWQ2NCIsCiAgICAgICAgICAgICAgICAib3MiOiAibGludXgiCiAgICAgICAgICAgIH0KICAgICAgICB9CiAgICBdCn0=

We'd then use this base64-encoded section of the manifest to generate a signature.

Content Addressable Images/Layers

Each digest in the bundle will be referenced by a checksum created from its contents. This checksum will be used on push and pull to verify contents have not been tampered and disallow the layer referred to in the manifest to be changed after it has been signed.

Manage manifest by tag

GET/PUT/DELETE /v2/images/<imgname>/<tagname>

List tags

GET /v2/images/<imgname>/tags

Download an image layer by content id

• Performs ACL verification
• Redirects to a temporary signed URL

GET /v2/images/<imgname>/<sumtype>/<sum>

Upload a bundle

PUT /v2/images/<imgname>/<sumtype>/<sum>

Upload an image layer

PUT /v2/images/<imgname>/<sumtype>

Compatibility

For compatibility with existing versions of docker registries, the manifests referenced in the bundle will address the existing Image v2 API. This compatibility will allow CNAB to re-use existing Docker containers available in their current format and layout.

[technosophos]

I'm currently moving over the part of the slide show that specifies this, then I will take a closer look at this.

[bacongobbler]

We've started to implement this and go a different route than by uploading the manifest to a docker registry. Closing!