New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conversion: add document about image -> runtime configuration #492

Merged
merged 1 commit into from May 31, 2017

Conversation

@cyphar
Member

cyphar commented Dec 12, 2016

Add documentation about how to convert from an OCI image configuration
to an OCI runtime configuration. In particular, describe precisely what
fields need to be filled given a particular configuration.

The fields have been grouped into several categories, because some of
the image fields are not as well-thought-out as others (such as the
resource limitation fields, which should really be decided by the user
not by the image creator). In addition, some fields (such as Volumes)
cannot be understood by a generic configuration converter.

Fixes: #479
Signed-off-by: Aleksa Sarai asarai@suse.de

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from fe74738 to ea753c8 Dec 12, 2016

@cyphar cyphar changed the title from conversion: add documentation about image config -> runtime config to conversion: add document about image -> runtime configuration Dec 12, 2016

@wking

Do we want to specify defaults for runtime-spec fields not set in the image? E.g. in a converted Linux config.json, should linux.namespaces be set?

Also, do we have version requirements for the generated config.json? Or is the converter free to target any tagged runtime-spec release?

@@ -0,0 +1,89 @@
# Conversion to OCI Runtime Configuration
When extracting an OCI Image into an [OCI Runtime bundle][oci-runtime-bundle], two components of the orthogonal components of the extraction are relevant:

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

“components of the orthogonal components of the extraction” → “orthogonal components”?

| `Config.Cmd` | `process.args` | 2 |
| `Config.Labels` | `annotations` | 3 |
1. The converter MAY *prepend* additional environment variables to `process.env`, but it MUST NOT *append* values to the environment.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

From POSIX:

There is no meaning associated with the order of strings in the environment. If more than one string in an environment of a process has the same name, the consequences are undefined.

So I think we should replace the append/prepend rule with one about not duplicating names.

| Image Field | Runtime Field | Notes |
| ------------------- | --------------- | ----- |
| `Config.User` | `process.user.*` | |

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

Unless we have notes entries, I'd rather drop the column. You have notes on this conversion below, so maybe add [notes](#config-user) or similar to the notes column here?

### `Config.User`
When converting the value of `Config.User` a converter SHOULD resolve the user information using a method appropriate for the container's context.
For Linux based systems this would involve parsing `/etc/passwd` from the extracted container's root filesystem to determine the values of `process.user.uid` and `process.user.gid`.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

The config docs allow both numeric and named user/group values, and only mention /etc/passwd in the context of default groups. So I think we need to weaken this line to say “MAY involve parsing /etc/passwd and /etc/group” to allow for:

  • Configs using numberic uid/gids (where there is no need for a lookup)
  • Agents using other NSS backends (e.g. LDAP), since the config spec doesn't require a particular NSS backend.
When converting the value of `Config.User` a converter SHOULD resolve the user information using a method appropriate for the container's context.
For Linux based systems this would involve parsing `/etc/passwd` from the extracted container's root filesystem to determine the values of `process.user.uid` and `process.user.gid`.
In addition, a converter SHOULD set the value of `process.user.additionalGids` to a value corresponding to the user described by `Config.User`.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

Probably “corresponding to the user” → “corresponding to the group”.

This comment has been minimized.

@cyphar

cyphar Dec 12, 2016

Member

No, because on Linux additionalGids is defined by group association of a user. Maybe "corresponding to the value in Config.User" if you want to make it more generic.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

Ah, right. Maybe there is wording that makes “this is an NSS lookup in the container context” more obvious, so folks don't trip into the same hole I just did.

## Optional Fields
Certain image configuration fields are not considered to

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

This sentence is incomplete.

| Image Field | Runtime Field | Notes |
| --------------------- | ------------------ | ----- |
| `Config.Memory` | platform dependant | 1 |

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

“dependant” → “dependent”

| `Config.ExposedPorts` | none | 2 |
| `Config.Volumes` | `mounts` | 3 |
1. The meaning of these fields is platform dependant, and runtimes MAY ignore these fields.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

“runtimes” → “converters”

1. The meaning of these fields is platform dependant, and runtimes MAY ignore these fields.
On Linux these image fields correspond to the runtime fields `linux.resources.memory.limit`, `linux.resources.memory.swap`, and `linux.resources.cpu.shares` respectively.
2. The runtime configuration does not have a corresponding field for this image field.
Converters SHOULD use this as a hint to allow a container to provide a network service from the corresponding container-side port.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

Do we want to SHOULD an annotations key for this information?

This comment has been minimized.

@cyphar

cyphar Dec 12, 2016

Member

Maybe, but since we haven't defined such an annotation in the org.opencontainers. namespace I'm a bit iffy about it.

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

… but since we haven't defined such an annotation in the org.opencontainers. namespace…

Can't we define one here? With something like:

Converters SHOULD set org.opencontainers.imageSpec.exposedPorts to [stringified value].

As @philips floated in #87.

2. The runtime configuration does not have a corresponding field for this image field.
Converters SHOULD use this as a hint to allow a container to provide a network service from the corresponding container-side port.
3. The converter MUST set the `destination` of the mountpoint to the value specified in `Config.Volumes`.
The other fields are platform and context dependant, though the converter SHOULD NOT allow volumes to be include in newly created [layers](layer.md).

This comment has been minimized.

@wking

wking Dec 12, 2016

Contributor

“include” → “included”.

And “SHOULD NOT allow volumes to be included in the newly created layers” sounds like it may belong here, and not in the conversion spec.

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch 3 times, most recently from 70e6a89 to fd65679 Dec 13, 2016

3. `annotations` in the [manifest list](manifest-list.md) of an image.
A converter SHOULD extract all three of these sets of annotations to `annotations` in the runtime configuration.
However, since there may be conflicts between different sets of annotations (same key but different value), the order of precedence SHOULD be in the following (ascending) order:

This comment has been minimized.

@stevvooe

stevvooe Dec 13, 2016

Contributor

Why this ordering?

To tell you the truth, this is why have been against adding annotations everywhere, especially when there isn't easy ways to distinguish runtime annotations/labels from those inherited from the source image.

To tell you the truth, this should probably be the opposite order. Manifests and Manifest-List tend not to survive to the runtime portion and are more oriented towards distribution.

This comment has been minimized.

@cyphar

cyphar Dec 13, 2016

Member

The ordering was based on what I thought made sense (the different blob layers are like onions, with each descriptor dereference going down a layer). But I wasn't sure why there were multi-level annotations in the first place (I assumed it was historical).

The other reason is that it means that if you share the same config between different manifests it's possible to override the config's annotations without needing to modify it -- if the ordering was in the reverse order then you wouldn't be able to do anything similar (you'd have to store two copies of the config).

This comment has been minimized.

@wking

wking Dec 13, 2016

Contributor

To tell you the truth, this should probably be the opposite order. Manifests and Manifest-List tend not to survive to the runtime portion and are more oriented towards distribution.

These are in order of ascending precedence, and while that's a somewhat unusual way to list precedence, it does mean that Config.Labels has the highest precedence. I think that's what you're asking for too.

This comment has been minimized.

@wking

wking Dec 13, 2016

Contributor

The other reason is that it means that if you share the same config between different manifests it's possible to override the config's annotations without needing to modify it…

Heh, so I read this completely wrong. Whatever we settle on, I suggest a clarifying example to avoid future confusion like mine.

This comment has been minimized.

@cyphar

cyphar Dec 13, 2016

Member

I actually meant descending order when writing this. But we should figure out what order makes sense first. I'd also understand if we don't include the annotations from manifests at all (which would be an even better idea because it would mean that the conversion is entirely done with v1.Image and doesn't require knowing what the manifest was).

That would make the distinction more clear between distribution and extraction/runtime.

This comment has been minimized.

@wking

wking Dec 13, 2016

Contributor

I'd also understand if we don't include the annotations from manifests at all (which would be an even better idea because it would mean that the conversion is entirely done with v1.Image and doesn't require knowing what the manifest was).

This sounds reasonable to me.

This comment has been minimized.

@cyphar

cyphar Dec 14, 2016

Member

@stevvooe @philips What do you think? Should we not require that runtimes include Manifest and ManifestList annotations in the runtime bundle?

wking referenced this pull request in cyphar/image-spec Dec 14, 2016

manifest: define com.opencontainers.org.imageSpec.exposedPorts
This is necessary in order for the image specification to be able to
hint to runtimes what ports the container wishes to expose.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from fd65679 to 79e35e6 Dec 14, 2016

@cyphar

This comment has been minimized.

Member

cyphar commented Dec 14, 2016

@stevvooe Can you take another look? I've switched around the wording and changed my annotation conversion description so that only Config.Labels MUST be extracted (and it takes precedence over any of the implicit conversions).

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from 79e35e6 to 3d8d99b Dec 14, 2016

@stevvooe

This comment has been minimized.

Contributor

stevvooe commented Dec 16, 2016

@cyphar Could update this to reflect the removal of Memory, et. al.?

@cyphar

This comment has been minimized.

Member

cyphar commented Dec 16, 2016

Yup, it's on my list of things to do. If I get the chance I'll do it this weekend or on Monday if I'm too busy.

@cyphar

This comment has been minimized.

Member

cyphar commented Dec 18, 2016

@stevvooe Updated.

The former component of extraction is defined [elsewhere](layers.md) and is orthogonal to configuration of a runtime bundle.
[oci-runtime-bundle]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/bundle.md
[oci-runtime-config]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md

This comment has been minimized.

@wking

wking Dec 18, 2016

Contributor

runtime-spec has cut an rc3.

| Image Field | Runtime Field | Notes |
| --------------------- | ------------------ | ----- |
| `Config.ExposedPorts` | none | 1 |

This comment has been minimized.

@wking

wking Dec 18, 2016

Contributor

In the verbatim section, you use annotations as the runtime field in cases like this.

| `Config.Volumes` | `mounts` | 2 |
1. The runtime configuration does not have a corresponding field for this image field.
However, converters SHOULD set the [`org.opencontainers.imageSpec.exposedPorts` annotation](#config.exposedports).

This comment has been minimized.

@wking

wking Dec 18, 2016

Contributor

Maybe "... annotation to the JSON serialization of the ExpopsedPorts value? And copy footnote 1 about precedence from the verbatim table. Then we could drop the ExposedPorrs section below without losing specificity.

This comment has been minimized.

@wking

wking Dec 18, 2016

Contributor

Scratch this, because the JSON for ExposedPorts is wonky. Your current handling is fine, although coping down your precedence footnote is probably still a good idea for consistency.

If there is a conflict (same key but different value) between an implicit annotation and an explicitly specified annotation in `Config.Labels`, the value specified in `Config.Labels` MUST take precedence.
A converter MAY add annotations which have keys not specified in the image.
A converter MUST NOT modify the values of annotations specified in an image.

This comment has been minimized.

@wking

wking Dec 18, 2016

Contributor

"an image" -> "the image" to match the preceding sentence and be more accurate.

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from 304c185 to 3ee6f4f Dec 18, 2016

@jonboulle

at a first pass this looks good!

## Optional Fields
Certain image configuration fields are not applicable to all conversion usecases, and thus are optional for configuration converter to implement.

This comment has been minimized.

@jonboulle

jonboulle Dec 19, 2016

Contributor

converters

@cyphar

This comment has been minimized.

Member

cyphar commented Dec 28, 2016

Can you all PTAL? Is there any outlying issues I need to fix up?

/ping @opencontainers/image-spec-maintainers

@wking

I think this still needs wording around the runtime-config properties which aren't related to the image config. Are they forbidden? Unspecified? Implementation-defined? For any fields not specified here, I prefer implementation-defined.

| `os` | `platform.os` | |
| `author` | `annotations` | 1,2 |
| `created` | `annotations` | 1,3 |
| `os` | `platform.os` | |

This comment has been minimized.

@wking

wking Dec 28, 2016

Contributor

Duplicate entry for os.

### `Config.ExposedPorts`
The OCI runtime configuration does not provide a way of expressing the concept of "container exposed ports".
However, converters SHOULD provide a hint to the runtime what the set of container exposed ports are by using the **org.opencontainers.imageSpec.exposedPorts** annotation, unless doing so will [cause a conflict](#annotations).

This comment has been minimized.

@wking

wking Dec 28, 2016

Contributor

"provide a hint to the runtime what the set of container exposed ports are by using" -> "set"?

If the values of [`user` or `group`](config.md#properties) in `Config.User` are numeric (`uid` or `gid`) then the values MUST be copied verbatim to `process.user.uid` and `process.user.gid` respectively.
If the values of [`user` or `group`](config.md#properties) in `Config.User` are not numeric (`user` or `group`) then a converter SHOULD resolve the user information using a method appropriate for the container's context.
For Linux and Solaris based systems, this MAY involve resolution through NSS or parsing `/etc/passwd` from the extracted container's root filesystem to determine the values of `process.user.uid` and `process.user.gid`.

This comment has been minimized.

@jonboulle

jonboulle Jan 3, 2017

Contributor

Can we just disallow NSS? Is this happening anywhere today?

This comment has been minimized.

@cyphar

cyphar Jan 14, 2017

Member

I don't see a compelling reason to disallow NSS. The point of this section is to say that the mapping of Config.User -> {uid, gid} is implementation defined, with some hints about how it should be done.

## Optional Fields
Certain image configuration fields are not applicable to all conversion usecases, and thus are optional for configuration converters to implement.

This comment has been minimized.

@jonboulle

jonboulle Jan 3, 2017

Contributor

usecases -> use cases

This section defines how to convert an `application/vnd.oci.image.config.v1+json` blob to an [OCI runtime configuration blob][oci-runtime-config] (the latter component of extraction).
The former component of extraction is defined [elsewhere](layers.md) and is orthogonal to configuration of a runtime bundle.
The values of runtime configuration properties not specified by this document are implementation defined.

This comment has been minimized.

@jonboulle

jonboulle Jan 3, 2017

Contributor

implementation defined -> implementation-defined

If the values of [`user` or `group`](config.md#properties) in `Config.User` are not numeric (`user` or `group`) then a converter SHOULD resolve the user information using a method appropriate for the container's context.
For Linux and Solaris based systems, this MAY involve resolution through NSS or parsing `/etc/passwd` from the extracted container's root filesystem to determine the values of `process.user.uid` and `process.user.gid`.
In addition, a converter SHOULD set the value of `process.user.additionalGids` to a value corresponding to the user described by `Config.User`.

This comment has been minimized.

@stevvooe

stevvooe Jan 6, 2017

Contributor

This doesn't sound right. Config.User should have a colon separated user and primary group. If there is no colon, the primary group is the user id. I don't think there is any support for supplementary (somehow, these where NIH'd to "additional") in the image format.

I am not sure why runtime-spec diverged from this. Having a separate field doesn't really provide much and makes it clear what is primary version. For example, in swarmkit, we have User and Groups, where user can be user or user:group and Groups is always supplementary.

I am not sure about the utility of supplementary groups in the image format.

This comment has been minimized.

@wking

wking Jan 6, 2017

Contributor

I don't think there is any support for supplementary (somehow, these where NIH'd to "additional") in the image format.

The config doesn't support it directly, which is why the following line suggests a way to extract this from the root filesystem or other source (“…MAY involve resolution through NSS or parsing /etc/group…”). Resolving this via the root filesystem is the same type of thing that this conversion doc suggest for non-numeric users and groups, although @jonboulle recently questioned the usefulness of generic NSS vs. requiring /etc/passwd / /etc/group.

This comment has been minimized.

@cyphar

cyphar Jan 13, 2017

Member

@stevvooe To clarify, "corresponding to the user" refers to the concept of a user in the image as opposed to the user string. The utility is so that groups like video, dialout and audio all still work inside containers.

I actually completely agree with what you said in your first paragraph. The point is that the resolution of supplementary groups (either through NSS or /etc/group) is recommended.

| `Config.Labels` | `annotations` | 6 |
1. If a user has explicitly specified this annotation with `Config.Labels`, then the value specified in this field takes lower [precedence](#annotations) and the converter MUST instead use the value from `Config.Labels`.
2. This value of this field MUST be set as the value of `org.opencontainers.authors` in `annotations`.

This comment has been minimized.

@stevvooe

stevvooe Jan 6, 2017

Contributor

Let's separate the active fields from the annotation fields.

Also, we need to be careful about the impact of picking up image labels into runtime annotations. I know I am having trouble convincing everyone this isn't a good idea, but it is a security hole waiting to happen.

For Linux and Solaris based systems, this MAY involve resolution through NSS or parsing `/etc/passwd` from the extracted container's root filesystem to determine the values of `process.user.uid` and `process.user.gid`.
In addition, a converter SHOULD set the value of `process.user.additionalGids` to a value corresponding to the user described by `Config.User`.
For Linux and Solaris based systems, this MAY involve resolution through NSS or parsing `/etc/group` and determining the group memberships of the user specified in `process.user.uid`.

This comment has been minimized.

@stevvooe

stevvooe Jan 6, 2017

Contributor

Unices?

This comment has been minimized.

@cyphar

cyphar Jan 13, 2017

Member

I'll change it to Unix-like

1. The runtime configuration does not have a corresponding field for this image field.
However, converters SHOULD set the [`org.opencontainers.imageSpec.exposedPorts` annotation](#config.exposedports).
2. If a converter implements conversion for this field, it MUST set the `destination` of the mountpoint to the value specified in `Config.Volumes`.

This comment has been minimized.

@stevvooe

stevvooe Jan 6, 2017

Contributor

The Config.Volumes field doesn't necessarily require a mount. That is really an implementation detail.

Config.Volumes really just marks a section of the file system as not part of the primary container filesystem.

This comment has been minimized.

@cyphar

cyphar Jan 13, 2017

Member

Okay, but is there a way of phrasing it in terms of the runtime specification to guarantee this? Maybe there is another way of pasting together a filesystem, but I'm having trouble thinking of one at the moment.

This comment has been minimized.

@stevvooe

stevvooe Jan 13, 2017

Contributor

I don't have a good suggestion here. The volume really acts as almost a "mask" of the filesystem. The implementation of the mask can vary.

2. This value of this field MUST be set as the value of `org.opencontainers.authors` in `annotations`.
3. This value of this field MUST be set as the value of `org.opencontainers.created` in `annotations`.
4. The converter MAY add additional entries to `process.env` but it MUST NOT add entries that have variable names present in `Config.Env`.
5. If both `Config.Entrypoint` and `Config.Cmd` are specified, the converter MUST append the value of `Config.Cmd` to the value of `Config.Entrypoint` and set `process.args` to that combined value.

This comment has been minimized.

@stevvooe

stevvooe Jan 6, 2017

Contributor

MUST is too strong here when taking user input for entrypoint and args.

This comment has been minimized.

@cyphar

cyphar Jan 13, 2017

Member

My main issue with entrypoint and cmd is that it's not really clear anywhere (outside of the Docker documentation) what their individual purposes are. The semantics also are quite hazy if you read config.md (and I think there's some hard-to-parse sentences there too).

This comment has been minimized.

@stevvooe

stevvooe Jan 13, 2017

Contributor

@cyphar Entrypoint and Cmd are indeed a bit of a mess. We've shored this up slightly in the docker documentation.

If you can find a way to word it such that implementations may substitute either Entrypoint or Cmd with user input, that would shore up the problem with this statement.

This comment has been minimized.

@wking

wking Jan 13, 2017

Contributor

If you can find a way to word it such that implementations may substitute either Entrypoint or Cmd with user input, that would shore up the problem with this statement.

Without an API spec to pin down “user input”, I think this is going to be hard to do. Can the user-input additions be a follow-up step? So:

image config →{this translation}→ runtime config →{user input}→ runtime config

Then whether the translator handles this translation correctly can be validated without worrying about user input, and whether the user-input injection happens correctly is up to whoever specified the user-input API.

This comment has been minimized.

@stevvooe

stevvooe Jan 13, 2017

Contributor

@wking Not really.

This comment has been minimized.

@wking

wking Jan 13, 2017

Contributor

If you need to know where the break between the entrypoint and command is, you can turn it around and have:

image config →{user input}→ image config →{this translation}→ runtime config

Then @cyphar's current append wording matches what is listed here for the case where both are arrays of strings. Or is image spec supposed to support the shell form versions too?

This comment has been minimized.

@cyphar

cyphar Jan 14, 2017

Member

@wking

Or is image spec supposed to support the shell form versions too?

Absolutely not. IMO those were a bad decision which Docker has had to stick with because of backwards compatibility issues.

This comment has been minimized.

@cyphar

cyphar Jan 14, 2017

Member

@stevvooe I'll give it a shot in terms of wording.

vrothberg pushed a commit to vrothberg/umoci that referenced this pull request May 16, 2017

Valentin Rothberg
generate: match conversion spec
Extend the conversion mechanism to match the specification as described
in: opencontainers/image-spec#492

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>

vrothberg pushed a commit to vrothberg/umoci that referenced this pull request May 16, 2017

Valentin Rothberg
generate: match conversion spec
Extend the conversion mechanism to match the specification as described
in: opencontainers/image-spec#492

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
@vbatts

This comment has been minimized.

Member

vbatts commented May 17, 2017

@cyphar could you update the couple of nits? 😸

@vbatts

This comment has been minimized.

Member

vbatts commented May 19, 2017

@cyphar bump. please ❤️

@stevvooe stevvooe modified the milestones: v1.0.0, v1.0.0-rc6 May 19, 2017

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from c352aef to 651d494 May 20, 2017

@cyphar

This comment has been minimized.

Member

cyphar commented May 20, 2017

@vbatts I was having trouble tracking what issues have not already been addressed. Here is my understanding of the current issues:

  • Config.User (@stevvooe) appears to have been addressed. The concern was the difficulty in requiring converters to implement getpwnam(3) but my response was that implementing user:group support is only a recommendation (and the method of resolving them is also only a recommendation) and only numeric uid:gid forms MUST be implemented verbatim.

  • The worries about Config.Env and other user-changable options (@crosbymichael + @stevvooe) should be resolved by the new wording which was sent after the last comments on those issues. There hasn't been much discussion in this thread recently so I don't know if anyone actually objects to @philips's proposal (and my extensions to it).

The one final point which we discussed in the call and also mentioned by @crosbymichael is

[...] I'm wondering how you will identify what a converter is and what an end user is?

The distinction in this document is not between end users and converters. It's between implementation-defined defaults and externally provided inputs. In the basic case there is no distinction, but for the spec the difference is quite important.

An implementation-defined default is something that we can test by providing a bunch of images that contain various bits of metadata and our testing will only check the outputs that were actually specified in the image (the image is the source of truth). Any extra fields or whatever don't matter. But if the converter changes fields that we specified in the image, that's where non-compliance kicks in.

When it comes to externally defined input, the way I'm thinking of it is that when an externally provided input is specified it is applied to the source image as though the source image always had the value specified. For a more concrete example, imagine that umoci had some flags to modify the configuration. The following two examples would be identical.

% # unpack the image, with an externally provided value for Config.User
% umoci unpack --image theimage:latest --override=config.user="someone:else" bundle
% # modify the image and change Config.User
% umoci config --config.user="someone:else" --image theimage:latest
% umoci unpack --image theimage:latest bundle

But the important point is that a converter wouldn't be able to change Config.User without being told to by the user.

And please note that this entire document is describing the "default generated runtime configuration". The point is that if a converter decides to start overriding things it's okay as long as they specify that they aren't outputting the default generated runtime configuration (and maybe we should require that implementations provide a way to use the "default generated runtime configuration" -- I'm not sure).

However, converters SHOULD set the [`org.opencontainers.image.exposedPorts` annotation](#config.exposedports).
2. If a converter implements conversion for this field using mountpoints, it SHOULD set the `destination` of the mountpoint to the value specified in `Config.Volumes`.
The other `mounts` fields are platform and context dependent, and thus are implementation-defined.
Note that the implementation of `Config.Volumes` need not use mountpoints, as it is effectively a mask of the filesystem.

This comment has been minimized.

@runcom

runcom May 22, 2017

Member

@cyphar could you help me fully understand point 2 for Config.Volumes; the first line reads:

If a converter implements conversion for this field using mountpoints, it SHOULD set the destination of the mountpoint to the value specified in Config.Volumes.

what does that mean? are there any other way to implement conversion for this field other than mountpoints? if yes, which ones?
How's the expected flow to implement the field with mountpoints for runtimes? create a host dir for that volume and bind mount it in the container?

Note that the implementation of Config.Volumes need not use mountpoints, as it is effectively a mask of the filesystem.

as a non-native speaker this seems in contrast with the first sentence. It now reads "don't use mountpoints!"

This comment has been minimized.

@cyphar

cyphar May 22, 2017

Member

are there any other way to implement conversion for this field other than mountpoints?

Yes, you could just copy the contents of the volume into the container. Hopefully (see #496) the purpose of volumes is to restrict how implementations should handle diff layers (allow external data that is not snapshotted).

In the case of umoci I could envisage this all being done through manifests and black-holing certain directories so umoci will simply ignore them.

How's the expected flow to implement the field with mountpoints for runtimes?

I only mention the destination. In particular what this is meant to mean is that the value of Config.Volumes should be a mountpoint (though there are reasons to not do that). The source of the mountpoint, the type of filesystem and so on can be whatever you want (it could be a tmpfs for example or NFS).

as a non-native speaker this seems in contrast with the first sentence. It now reads "don't use mountpoints!"

The first sentence says you SHOULD, but the second sentence says you don't have to. To be fair, maybe I shouldn't repeat that point but I felt worried people would make the same assumption you did.

This comment has been minimized.

@runcom

runcom May 22, 2017

Member

Yes, you could just copy the contents of the volume into the container.

alright, I guess what's still not clear to me is the definition of a volume which this doc is missing. What's a volume then? some directory/mountpoint on the host or somewhere else?
from that sentence, as you said, it can be anything.

However, converters SHOULD set the [`org.opencontainers.image.exposedPorts` annotation](#config.exposedports).
2. If a converter implements conversion for this field using mountpoints, it SHOULD set the `destination` of the mountpoint to the value specified in `Config.Volumes`.
The other `mounts` fields are platform and context dependent, and thus are implementation-defined.
Note that the implementation of `Config.Volumes` need not use mountpoints, as it is effectively a mask of the filesystem.

This comment has been minimized.

@stevvooe

stevvooe May 22, 2017

Contributor

Should this also stipulate that data from the image may be copied into the volume?

This comment has been minimized.

@cyphar

cyphar May 23, 2017

Member

No, because that's forbidden by the definition of Config.Volume (https://github.com/opencontainers/image-spec/blame/master/config.md#L152):

If a file or folder exists within the image with the same path as a data volume, that file or folder will be replaced by the data volume and never be merged.

This comment has been minimized.

@cyphar

cyphar May 23, 2017

Member

Sorry, the entire directory is masked, not just the files that are in the data volume. Maybe I should make this sentence clearer but given the Config.Volume definition I can't imagine a reasonable reading of this text would conclude that merging is okay.

This comment has been minimized.

@stevvooe

stevvooe May 23, 2017

Contributor

This is behavior that docker allows today: effectively, the contents of a volume can be seeded with the contents of the image.

This comment has been minimized.

@cyphar

cyphar May 23, 2017

Member

Okay, but that's not what the spec currently allows (I don't agree with that design either, but let's discuss that somewhere else). Would you mind if we handle that in a separate PR (it's got nothing to do with the config generation and more to do with extraction surely).

This comment has been minimized.

@stevvooe

stevvooe May 25, 2017

Contributor

@cyphar I'll file an issue and not hold up this PR further on this matter.

@vbatts

This comment has been minimized.

Member

vbatts commented May 25, 2017

discussed on a call between @stevvooe and I just now. @stevvooe to make more commentary from his perspective.

Externally provided inputs are considered to be a modification of the `application/vnd.oci.image.config.v1+json` used as a source, and such modifications have no restrictions.
For example, externally provided inputs MAY cause an environment variable to be added, removed or changed.
However an implementation-defined default MUST NOT result in an environment variable being removed or changed.

This comment has been minimized.

@stevvooe

stevvooe May 25, 2017

Contributor

I think if we relax this to a SHOULD NOT, we can go forward here.

This comment has been minimized.

@cyphar

cyphar May 26, 2017

Member

Presumably you also want me to change this line too:

  1. The converter MAY add additional entries to process.env but it MUST NOT add entries that have variable names present in Config.Env.

?

This comment has been minimized.

@stevvooe
@cyphar

This comment has been minimized.

Member

cyphar commented May 30, 2017

Alright, fixed @stevvooe @vbatts. PTAL.

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from 9c6ca2d to 0e507a5 May 30, 2017

@stevvooe

This comment has been minimized.

Contributor

stevvooe commented May 30, 2017

LGTM

Approved with PullApprove

conversion: add document about image -> runtime configuration
Add documentation about how to convert from an OCI image configuration
to an OCI runtime configuration. In particular, describe precisely what
fields need to be filled given a particular configuration.

The fields have been grouped into several categories, because some of
the image fields are not as well-thought-out as others (such as the
resource limitation fields, which should really be decided by the user
not by the image creator). In addition, some fields (such as Volumes)
cannot be understood by a generic configuration converter.

In addition, the annotation org.opencontainers.imageSpec.exposedPorts
has been defined in order to allow for hinting to runtimes what ports
are exposed by a container. The same for
org.containers.imageSpec.stopSignal.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

@cyphar cyphar force-pushed the cyphar:479-define-conversion-to-runtime-spec branch from 0e507a5 to 544e5ff May 30, 2017

@cyphar

This comment has been minimized.

Member

cyphar commented May 30, 2017

Sorry, I just squashed it. Can you LGTM again?

@stevvooe

This comment has been minimized.

Contributor

stevvooe commented May 30, 2017

LGTM

Approved with PullApprove

@vbatts

This comment has been minimized.

Member

vbatts commented May 31, 2017

@vbatts vbatts merged commit db0dfbd into opencontainers:master May 31, 2017

2 checks passed

code-review/pullapprove Approved by stevvooe, vbatts
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

vbatts added a commit to vbatts/oci-image-spec that referenced this pull request May 31, 2017

conversion: follow-up touch-ups
Reference: opencontainers#492 (comment)

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

@cyphar cyphar deleted the cyphar:479-define-conversion-to-runtime-spec branch May 31, 2017

@vbatts vbatts referenced this pull request Jul 5, 2017

Merged

bump to v1.0.0-rc7 #708

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment