Add 2021-05-03-introducing-prometheus-conformance-program.md#1941
Add 2021-05-03-introducing-prometheus-conformance-program.md#1941
Conversation
content/blog/2021-05-03-introducing-prometheus-conformance-program.md
Outdated
Show resolved
Hide resolved
Signed-off-by: Richard Hartmann <richih@richih.org>
RichiH
force-pushed
the
richih/prometheus-conformance
branch
from
dd1d6bb to
a3a7c96
Compare
content/blog/2021-05-03-introducing-prometheus-conformance-program.md
Outdated
Show resolved
Hide resolved
content/blog/2021-05-03-introducing-prometheus-conformance-program.md
Outdated
Show resolved
Hide resolved
content/blog/2021-05-03-introducing-prometheus-conformance-program.md
Outdated
Show resolved
Hide resolved
|
|
||
| For every component, there will be a mark "foo YYYY-MM compliant", e.g. "OpenMetrics 2021-05 compliant", "PromQL 2021-05 compliant", and "Prometheus Remote Write 2021-05 compliant". Any project or vendor can submit their compliance documentation. Upon reaching 100%, the mark will be granted. | ||
|
|
||
| For any complete software, there will be a mark "Prometheus x.y compatible", e.g. "Prometheus 2.26 compatible". Relevant component compliance scores are multiplied. Upon reaching 100%, the mark will be granted. |
There was a problem hiding this comment.
"Upon reaching 100%, the mark will be granted."
Note that query tweaks in my PromQL tests (https://promlabs.com/promql-compliance-tests) have been treated separately from the % score so far, and the current 100% score wouldn't always be a true 100% when we factor those query tweaks in (which we have to, to be fully compliant). E.g. Grafana Cloud would be failing then due to their timestamp alignment, but this is probably also how it should be unless Prometheus Team wants to grant certain exemptions.
There was a problem hiding this comment.
Our thinking in OM is to follow the system of MUST, SHOULD, MAY and we chose the same language in the Remote Write spec. This maps 1:1 to if tests are fail, warning, or info level. If timestamp alignment MUST NOT happen, then Grafana Cloud does not pass until it stops aligning timestamps, agreed.
This is why I like normative language: It makes gaps and disagreements visible.
Signed-off-by: Richard Hartmann <richih@richih.org>
|
Thanks, LGTM! Merged as requestes on live PromCon 2021 ((: |
5 participants
No description provided.