Week of Apr 6

[duglin]

• add 'consistentformat' to Resource model
• make the default value for the validation flags 'false'
• make schema registry be true/strict wrt 3 format/compat flags (not strict)

Fixes #453

[duglin]

Just moved up - no changes

[jskeet]

So if validatecompat is true, but format is missing, should that raise an error?

[duglin]

If "format" is missing, it's basically saying "turn off format/compat validation for this version". So, no it doesn't raise an error.

I struggled with the location for text (model vs spec), but in the core spec w.r.t. the "compatvalidated" attribute it says this:

  - MUST NOT be present if the Resource model's `validatecompatibility`
    attribute is `false`, or the Version's `format` value is absent, or the
    Resource's `meta.compatibility` attribute is absent.

[jskeet]

So is the format just false because I felt like it or is there more structure, e.g. false (reason here)?

[duglin]

I wasn't sure how picky I should be about the shape of the text. I tried to stick with just "true" vs "false.*" (just first 5 chars matter). Is that ok? I was torn between human readable sentence and from machine parse-able (e.g. your false (reason here)), and decided to leave that as an impl choice.

[jskeet]

This answers my earlier question - I'm not particularly comfortable with it though. I'd personally prefer a more structured approach, keeping formatvalidated as a Boolean, and having a separate formatvalidationdetail attribute. The idea of any user code needing to check whether a value starts with false feels icky to me.

[duglin]

I share the concern, and we did talk about this a bit. For me, I think the overriding factor was that I didn't want to introduce "yet another" attribute when it pretty much goes "hand in hand" (ie required) with the "false". The other thought I had was that when it's a boolean it'll be too easy for people to only look at the boolean, ignore the text, and mentally think of "false" as the same thing as "invalid". Which it's not. "false" means "I didn't check". And seeing the other text right next to the "false" increases the chances of people reading it and realizing the server didn't even try.

Perhaps true/false is part of the problem. Maybe we should not try to squeeze a boolean into a string - but if we don't, I think it makes it even parse for a machine to know what's going on. At least now they can parse the first few chars to get a quick answer.

[jskeet]

If this were a project I were more deeply invested in, I'd definitely push back on this. I don't think there's significant cost to an extra attribute, and it would be easier to use. But given that I don't have skin in the game, I'll let y'all decide :)

[duglin]

oh the guilt trip!

[jskeet]

Same comment as above, really.

[duglin]

Summary:

/capabilities
{
  "formats": [ "<STRING>", * ], ?          # "avro", "protobuf", "jsonSchema" ],
  "compatibilities": {
    "<STRING>", [ "<STRING>", * ], *       # "avro": [ "backward", "forward", "full" ],
  }                                        # "protobuf": [ "backward" ]
}

/model  (on RESOURCE definition)
{
  ...
  "validateformat": <BOOLEAN>,             # Default is false
  "validatecompatibility": <BOOLEAN>,      # Default is false. If true then validateformat MUST be true
  "strictvalidation": <BOOLEAN>,           # Default is false. Allow unknown format/compat values, RESOURCEurl
  "consistentformat": <BOOLEAN>,           # Default is false. All versions MUST have the same format value
  ...
}


/...RESOURCE Instance
{
  ...
  "format": "<STRING>", ?                   # Absent=no validation. Unknown values are ignored
  "formatvalidated": "<STRING>", ?          # Absent=disabled. "true" or "false, because...". Read-only
  "compatibilityvalidated": "<STRING>", ?   # Absent=disabled. "true" or "false, because...". Read-only
  ...
  "meta": {
    ...
    "compatibility": "<STRING>", ?          # Absent=no validation. Unknown values are ignored
    ...
  }
}

[duglin]

Approved on the 4/16 call