Specification consistency review #206
Description
Ahead of the IETF 123 meeting we're going to have soon, I gave it a try to use Google Gemini to explore the internal consistency of the I-D document, in particular with respect to the use of MUST vs SHOULD.
Gemini produced a comprehensive and rather verbose report, where the most relevant findings are as follows. I think they all ought to be addressed.
Table 1: Incoherent RFC 2119 Keyword Usage
The following table provides a structured overview of specific instances where RFC 2119 keywords are used inconsistently or inappropriately, along with proposed corrections for enhanced clarity and adherence to IETF standards.
| Keyword Used | Section/Snippet ID | Original Text | Issue Description | Proposed Correction/Clarification |
|---|---|---|---|---|
| SHOULD | S_S2 | "Mesh Provider SHOULD allow discovery of resources." | Conflict with S_S23 (MUST) | Change SHOULD to MUST. |
| REQUIRED | S_S10 | "Data encryption is REQUIRED for sensitive data." | Non-RFC 2119 term | Replace REQUIRED with MUST. |
| REQUIRED | S_S30 | "Resource identifiers are REQUIRED to be unique within a Mesh Provider's scope." | Non-RFC 2119 term | Replace REQUIRED with MUST. |
| RECOMMENDED | S_S17 | "Resource tagging is RECOMMENDED for better organization." | Non-RFC 2119 term | Replace RECOMMENDED with SHOULD. |
| SHOULD | S_S15 | "Mesh Consumer SHOULD use secure communication channels (e.g., TLS)." | Potentially weak for critical security | Re-evaluate; consider changing to MUST for production environments. |
| SHOULD | S_S32 | "Data backup and recovery mechanisms SHOULD be in place." | Potentially weak for critical reliability | Re-evaluate; consider changing to MUST for production environments. |
| SHOULD | S_S36 | "Security audits SHOULD be conducted annually." | Potentially weak for critical security | Re-evaluate; consider changing to MUST for production environments. |
| SHOULD | S_S48 | "SLO breaches SHOULD trigger automated alerts." | Potentially weak for operational robustness | Re-evaluate; consider changing to MUST for production environments. |
| SHOULD | S_S50 | "Redundancy mechanisms SHOULD be employed." | Potentially weak for operational robustness | Re-evaluate; consider changing to MUST for production environments. |
Table 2: Conflicting Requirement Statements
The table below provides a side-by-side comparison of contradictory requirements, highlighting the inconsistencies and their potential impact on the Open Cloud Mesh.
| Concept/Feature | Location 1 (Snippet ID, Text, Requirement Level) | Location 2 (Snippet ID, Text, Requirement Level) | Nature of Conflict | Potential Impact |
|---|---|---|---|---|
| Resource Discovery | S_S2 ("Mesh Provider SHOULD allow discovery of resources.", SHOULD) | S_S23 ("A Mesh Provider MUST provide a mechanism for resource discovery.", MUST) | SHOULD vs. MUST | Inconsistent implementation of a core mesh functionality, hindering resource lookup and broad adoption. |
| Resource Identifier Uniqueness | S_S4 ("Resource Identifier MUST be globally unique.", MUST) | S_S30 ("Resource identifiers are REQUIRED to be unique within a Mesh Provider's scope.", REQUIRED) | Global vs. Provider-scoped uniqueness | Fundamental architectural ambiguity for resource addressing, linking, and interoperability across the mesh. Implementers will choose one, leading to fragmentation. |
| Mesh Provider API Exposure | S_S1 ("Mesh Provider MUST expose an API.", MUST) | S_S28 ("The Mesh Provider MUST expose a public API for Mesh Consumers.", MUST) | Redundant, but consistent statements of the same requirement | While currently consistent, such redundancy can lead to future inconsistencies if one statement is updated without the other, or if different nuances are introduced, impacting document maintainability and clarity over time. |
Other findings
Finding 3.1.1: Conflicting Scope of Resource Identifier Uniqueness
A severe logical inconsistency exists regarding the scope of resource identifier uniqueness. One statement mandates that "Resource Identifier MUST be globally unique". [...] In direct contrast, another statement specifies that "Resource identifiers are REQUIRED to be unique within a Mesh Provider's scope".
Finding 3.1.2: Redundant but Consistent Requirements (for completeness)
For example, "Mesh Provider MUST expose an API" and "The Mesh Provider MUST expose a public API for Mesh Consumers".
Finding 4.1.1: Ambiguity of "Sensitive Data"
The term "sensitive data" is used as a critical qualifier for a mandatory security requirement: "Data encryption is REQUIRED for sensitive data".
Finding 4.1.2: Subjectivity of "Granular Authorization"
The specification states that "Authorization SHOULD be granular".
Finding 4.1.3: Vague Qualifiers for API Quality
The specification uses qualitative terms like "robust" and "informative" without objective metrics or examples. For instance, "A Mesh Provider MUST implement robust error handling for its API", and "Error messages SHOULD be informative but not expose sensitive details". While the latter part about not exposing sensitive details is clearer, "informative" can be interpreted broadly.
Finding 4.2.1: Open-ended "Standard Authentication Protocol"
The specification mandates that "Mesh Provider MUST support at least one standard authentication protocol (e.g., OAuth 2.0)". While mandating support for "at least one standard authentication protocol," the specification only provides an example (OAuth 2.0) without listing other acceptable protocols or defining a process for their inclusion.
Finding 4.2.2: Unspecified "Consistent Format" for Error Responses
The statement "Error responses SHOULD follow a consistent format" recommends consistency but does not specify which consistent format should be used (e.g., RFC 7807 Problem Details, JSON:API errors, or a custom format).