Implement CF conformance testing program for certification

[gberche-orange]

The cloudfoundry certitication process has long lacked from transparency and the process to evaluate/adopt/reject community inputs is lacking inclusivity

https://lists.cloudfoundry.org/g/cf-dev/topic/6371996#7543
https://lists.cloudfoundry.org/g/cf-dev/message/5996?p=,,,20,0,0,0::relevance,,%222017+PaaS+Certification+Requirements%22,20,2,0,6333680

The current certification process is also hard to find, and apparently not referenced in public website at https://www.cloudfoundry.org/certified-platforms/

[chipchilders]

Personally, I'm looking forward to a TOC defining explicit API conformance. Tagging @troytop as he might have some thoughts.

[gberche-orange]

+1 for TOC defining api conformance (such as https://github.com/cloudfoundry/cf-acceptance-tests/ )

The existing certification process was however wider in scope, e.g. aiming at avoiding forks and mandating use of core component binaries, see https://docs.google.com/document/d/1QRtzYcumcNmEiGTmzMjcxnWffcPsu7AyHlReFx5eY7M/edit?ts=5a0277af#heading=h.v1uqb5pkq2ul

Certified offerings are required to use the exact software packaged in specific releases of each of the: Cloud Foundry Elastic Runtime and Developer CLI tool. Details for each component are noted below.

When certifying and during a verification audit, the program participant must provide the Foundation the exact release numbers for each required component.

[troytop]

@gberche-orange I'm currently working on a document proposing certification via conformance testing (using CATs to start). I'm going to share this with cf-dev soon, but if you want a look at the work in progress: https://docs.google.com/document/d/1tI2uY7AXmL7ED-FlJLxpuAVBmRr_I11XMm1u68J-6NE/edit?usp=sharing

[chipchilders]

The existing certification process was however wider in scope, e.g. aiming at avoiding forks and mandating use of core component binaries, see https://docs.google.com/document/d/1QRtzYcumcNmEiGTmzMjcxnWffcPsu7AyHlReFx5eY7M/edit?ts=5a0277af#heading=h.v1uqb5pkq2ul

Certified offerings are required to use the exact software packaged in specific releases of each of the: Cloud Foundry Elastic Runtime and Developer CLI tool. Details for each component are noted below.
When certifying and during a verification audit, the program participant must provide the Foundation the exact release numbers for each required component.

+1 to the idea that the current model is broader in scope. My suggestion for the "rebooted" technical governance process would be to toss out the old cert goals and refocus on what is best for the project going forward, starting with API conformance first. The whole program should be up for discussion at this point, while we may find value in the past implementation.

[chipchilders]

Changed title to accurately reflect this as a TODO for the TOC once we have one.