Need tests for JSON Production and Consumption

[msporny]

6.2.1 JSON Production
The DID document and any DID document data structures expressed by the data model MUST be serialized to the JSON representation according to the following production rules:

Data Type	JSON Representation Type
map	A JSON Object, where each entry is serialized as a member of the JSON Object with the entry key as a JSON String member name and the entry value according to its type, as defined in this table.
list	A JSON Array, where each element of the list is serialized, in order, as a value of the array according to its type, as defined in this table.
ordered set	A JSON Array, where each element of the set is added, in order, as a value of the array according to its type, as defined in this table.
datetime	A JSON String serialized as an XML Datetime normalized to UTC 00:00:00 and without sub-second decimal precision. For example: 2020-12-20T19:17:47Z.
string	A JSON String.
integer	A JSON Number without a decimal or fractional component.
double	A JSON Number with a decimal and fractional component.
boolean	A JSON Boolean.
null	A JSON null literal.

All entries of a DID document MUST be included in the root JSON Object.

When serializing a DID document, a conforming producer MUST specify a media type of application/did+json to downstream applications such as described in § 7.1.2 DID Resolution Metadata.

6.2.2 JSON Consumption
The DID document and any DID document data structures expressed by a JSON representation MUST be deserialized into the data model according to the following consumption rules:

JSON Representation Type	Data Type
JSON Object	An map, where each member of the JSON Object is added as an entry to the map. Each entry key is set as the JSON Object member name. Each entry value is set by converting the JSON Object member value according to the JSON representation type as defined in this table. Since order is not specified by JSON Objects, no insertion order is guaranteed.
JSON Array where the data model entry value is a list or unknown	A list, where each value of the JSON Array is added to the list in order, converted based on the JSON representation type of the array value, as defined in this table.
JSON Array where the data model entry value is an ordered set	An ordered set, where each value of the JSON Array is added to the ordered set in order, converted based on the JSON representation type of the array value, as defined in this table.
JSON String where data model entry value is a datetime	A datetime.
JSON String, where the data model entry value type is string or unknown	A string.
JSON Number without a decimal or fractional component	An integer.
JSON Number with a decimal and fractional component, or when entry value is a double regardless of inclusion of fractional component	A double.
JSON Boolean	A boolean.
JSON null literal	A null value.

If media type information is available to a conforming consumer and the media type value is application/did+json, then the data structure being consumed is a DID document, and the root element MUST be a JSON Object where all members of the object are entries of the DID document.

A conforming consumer for a JSON representation that is consuming a DID document with a root element that is not a JSON Object MUST report an error.

[OR13]

@csuwildcat any chance Microsoft / you can help test the representation you fought so hard for us to include :) ?

[iherman]

The issue was discussed in a meeting on 2021-04-06

• no resolutions were taken

View the transcript

7.8. Need tests for CBOR/JSON Production and Consumption

See github issue did-test-suite#28, did-test-suite#26.

Markus Sabadello: those are about sections that don't have tests
… 28 is about cbor
… 26 is about json
… there are test for jsonld production and consumption
… and for generic sections about production and consumption that are not representation specific
… it looks like there are no tests yet for cbor and json production and consumption
… the cbor one is assigned to Orie and json is assigned to Daniel

Daniel Burnett: we were hoping for someone like Jonathan to contribute on the cbor

Jonathan Holt: doing my best to get back in the swing with DID contributions

Daniel Burnett: daniel can you contribute on the json? the json format in particular was one of the biggest requests from Microsoft on day one of the working group
… so we really want to make sure it's complete

Daniel Buchner: the way we've gone about it I don't know... it's worked perfectly fine working jsonld as json and saying you don't have to resolve contexts
… I can work on any tests that take that approach
… that's what we're doing in our implementation
… make sense?
… are there bugs or issues out there that want it to be different than that?

Markus Sabadello: i know some people want to treat jsonld and json as equivalent and to include @context in both and this has to do with long discussions we had
… my personal opinion is this is not what the did core spec describes right now
… it does not envision that json and jsonld are byte to byte equivalent
… if that's what WG members want to do we should change the did core spec
… right now on the test suite we need to take the normative statements
… the open issues on the test suite they list the normative statements that need to be tested
… so we need to work based on that

Justin Richer: +1 to what markus was saying
… one of the things that got brought up a necessary test condition is that a did doc that is described to be in plain json should be allowed to have the @context field and have it be a completely invalid value, and that should pass
… that should get parsed, the @context dumped into the bucket
… with whatever value it is
… no checking if it's an array or string or anything

Daniel Buchner: Yes Justin, yeeeessss

Daniel Buchner: Love everything you are saying right now

Justin Richer: so if you have the @context value with boolean false an d a plain json encoding by the way the did core spec is defined today that is allowable
… because the re-serialization on the producer side needs to be able to place the appropriate context value and not just pass through blindly whatever it was handed
… I will say I'm not personally familiar with the structure and function of the test suite code to suggest how to contribute this piece and don't have the bandwidth, I apologise for that
… but that is the type of corner case we need to have tests cases for
… if you're doing lazy validation that's not proving you're compliant

Daniel Buchner: the interesting thing.. it doesn't say anything about must exclude properties... isn't lazy validation accurate?
… if you had context with a bool and the spec doesn't say it has to be right isn't the way i'd write the test to not deal with that property?
… lazy validation is the valid way to do it?

Justin Richer: you need to go one step further because the spec says what to say with unknown properties
… in this case @context would be an unknown property
… you need to make sure things will accept things with an invalid @context
… that's important
… not enough to just say if i give you something an call it plain json but it's really jsonld that you'll be happy with that, that's not enough
… we need to test these kinds of things
… if you do that by ignoring it using a precise definition of ignore that's in the spec, that's fine
… that's not lazy validation
… that's explicitly handling unknown properties in a particular way

Daniel Buchner: instead of completely avoiding it or not putting any lines of code that deal with unknown properties, it'll iterate all the properties and have some sort of if that says if you're not in the set.. log it.. do something.. recognise it exists.. if it is in the set do what you're supposed to do?

Justin Richer: to be specific what it does is processes the serliazed property based on the type value of the serialized property and puts it into the properties map
… there will be an @context with boolean value of false inside the ADM

Daniel Buchner: ok, this makes sense

Daniel Buchner: thank you justin

Justin Richer: when it comes time to do a production from that ADM if you want to do it into jsonld your jsonld producer needs to be able to make sense of that bad @context value
… and either throw an error there or replace it with a good @context value on thew ay out which is more likely
… there are specific rules of how to handle these
… from a code standpoint it does look like the else at the end of your validators

Daniel Buchner: that's a good answer, thank you

Daniel Burnett: issue 26 about json... Orie had asked in feb whether you might be able to contribute in writing some tests
… largely this is in here because Microsoft fought hard for it
… we want to make sure everyone else is not doing all of the test work

Daniel Buchner: I'll try to do something about it

Daniel Burnett: I think that's it for the test issues

[msporny]

Reminder that complete tests, including review, auditing, and at least one implementation, are due for this item by the end of this month.

[msporny]

All JSON Production and Consumption tests have been implemented. Closing.