Technical-proposal needed for compliance and certification

[vbatts]

Ref #452 (comment)

The changes for validation need to have a bit of guidance as to the nature of the compliance validation.
This would ideally be a product from the cert and compliance working group to ensure validation of OCI image artifacts satisfies the requirements.

[vbatts]

cc @opencontainers/certification-maintainers

[wking]

On Wed, Feb 15, 2017 at 02:40:57PM -0800, Vincent Batts wrote: This would ideally be a product from the cert and compliance working group to ensure validation of OCI image artifacts satisfies the requirements.

In opencontainers/image-tools#66, I floated “strict” (only contains OCI-defined content) and “base” (only contains content which the image-spec REQUIREs unpackers to support) levels for compliance testing. Both of those are more restrictive than a “legal” level (content does not violate any MUST-level image-spec requirements). Another axis which cert/compliance would be interested in is MUST-level vs. SHOULD-level validation. So I think we should make it easy for callers to validate with any combination of legal/strict/base and legal/should. Beyond the cert/compliance working group, I also floated an “unpackable” level (can image-tools unpack the content). That would not impact the cert/compliance working group, but it would be unfortunate to address only legal/strict/base (or whatever) in image-spec and leave no way for image-tools to support legal/strict/unpackable/base. It would also be nice if callers could register support for additional media types (e.g. “I can also unpack application/vnd.example.layer.v3.json”). And it should be easy for Go callers to figure out about validator concerns (e.g. “what were the SHOULD-level violations, if any?”) which is hard if the validation code is printing warnings to stdout [1]. I've taken a few stabs at implementating a framework for this with no success (e.g. in #403), so I'll leave figuring that out to runtime-tools maintainers. The above stuff is my take on the constraints for that framework. This issue may just be about collecting constraints anyway. [1]: #491 (comment)

[RobDolinMS]

@vbatts Would the proposal at: opencontainers/oci-conformance#26 fulfill this issue?

[vbatts]

@RobDolinMS perhaps so

[vbatts]

The conformance issue was closed because it went stale. I still think a conformance/validation piece would be nice here.