JSON-LD context specific questions about DID Core specification

[veikkoeeva]

My questions seem to be related to DID Core editorial review (2) and DID Core editorial review (1).

I wrote a small parser based on DID Core Specification. It is based on System.Text.Json, which is a very new forward-only .NET JSON parser (but one can store data to objects while processing so everything is possible). It does not have JSON-LD processing rules.

Which brings me to my questions

1. Do DID Documents restrict the @context need to appear as the first element in its respective JSON object? This could be useful while processing the document. I tried to check the specification if the order is always specified so and then also check if JSON-LD specification tells something about it, but didn't find anything conclusive.
2. Is it possible the documents can be serialized in otherJSON-LD format than the compacted form? It might be useful to explicitly note this case. If several cases are possible, maybe recommend one particular serialization.

I think it would be useful to provide more complete examples of DID documents that conform the core specification and then also https://www.w3.org/TR/did-spec-registries/ ones and also some examples that specify something more custom such as spamCost.

The reason for this separation is that it took a while for me to understand that publicKeyBase58 is a registered feature. While I think it's excellent to note this in the core document, it would be nice to explicitly note it and link to https://www.w3.org/TR/did-spec-registries/. The reason I think both this and the spamCost example are very good is that I took a bit more time to understand how to make a parser that can account for these matters.

It would be nice if there if there would be a good compilation of conforming documents one could use as test cases. Now I picked up something from the specification and one Veres One document while writing this. I may push the code to GH soon(ish) in case someone in the future finds it a useful starting point. I took a look at https://github.com/w3c/did-test-suite and may try to make this conforming (depending on how much JSON-LD specific processing is needed).

Thank you all for your hard work!

[veikkoeeva]

I pushed the repo to https://github.com/veikkoeeva/DotDecentralized in case someone wants to take a look. This is a draft in a temporary repo and I intend to move it around the end of month.

I have not looked if there are other .NET libraries in general nor if there are with this approach in particular. It looks like implementing the specification will not be awfully lot code if implementing JSON-LD processing rules are not needed. The cases are more about interpreting the specification and working out a system that is open enough for extensions and composing (while still being secure to use, maintain reasonable developer ergonomics, fast enough etc.).

It appears this draft can parse most samples I could find (not all but easily fixable). It does not implement checking all the rules on DIDs nor cryptographic operations but I intend to add these later after I've turned this into a more proper project.

I also came about with new questions (such as what is alsoKnownAs). Some are here and some I think can be discussed in Interop group, thanks for Juan C. for monitoring here and contacting. :)

[veikkoeeva]

I link a few more recent issues and comment from the perspective of using a JSON parser to implement the specification. I think you guys do a good job and the intention is not to demand work but more like write down some notes for myself and others who look from .NET implementation perspective.

As a general note reflecting the discussion in linked threads is that while I use a plain forward-only parser, it can use converters almost at any stage in the process. This allows considerable flexibility in parsing and using state within the parser as long as the serialized format is textual JSON (it doesn't even need to be that but things get difficult if not). I think many parsers in strongly typed languages have this kind of a functionality.

#432
This would be nice to know. As for this quick library to ingest DID documents the goal is just to parse the documents to a reasonable, strongly typed data structure and define as methods that work for this data structure. I see builder pattern used in one of the Java libraries referenced in one of the issues. The idea here is similar with converters and all checks and operations that work on the parsed data structure (they could also work on in-memory JSON). Any restrictions like saying

Context is always an array of IRIs, even if there is only one element. The first element is always https://www.w3.org/ns/did/v1.

would make parsing easier. Currently I wrote the parser so it checks if it's a single element string, an array of strings or an object or array of objects. A complication here is that if the shape of @context objects can be whatever, it will be necessary to implement this so a user registers types it expects and if it will be something else, it will be extension data, which is key value pairs in dictionary or a JSON fragment. This can be queried but it's not strongly typed.

As for internal, strongly typed structure, currently it is a collection of either strings or objects and there's logic in deserializing to check this. (*1) It is possible to add a convenience property to make single element arrays appear as a single attribute, but the library user would need to check beforehand if this is the case and it sort of removes the purpose.

Here it would help if the specification would be explicit (or did I miss it?) if it said that if it's only a single element, it is written as such, not as an array of one element. Now I decided single element arrays are serialized out as single elements.

It feels like that if there is @context or @vocab attributes present, they are used as parameters to select the type as for example in type in Service Endpoints. In this quick library where I have tried to suss out details this would look like having an extra parameter to lambda function to make a decision which type to serialize. If there is no type registered for the parameters, then just some general object can be created and it's left for the library user to do whatever is done. This can be using some other logic to query the structure. It implies the type was not expected, has not been implement or something of the sorts. It would be helpful to be explicit that if this is so, @context (or other needed information) is always the first element in the document and used in subsequent parts X, Y and Z.

#439
It looks to me this relates to #432. If I have understood correctly, I think that if a JSON parser can parse it and the specification tells what are the rules things are implementable. Specifically:

• If some rule is JSON-LD specific, like using @base, it would be nice if it would be spelled out what it means in context of DID specification. E.g. strip fragments, use it to prefix in places X, Y and Z. Additionally maybe linked to a JSON-LD specification for exact explanation. Or that if @base is not used and anything that begins with # and should be an IRI is relative to the document (or prefixed with @base or @vocab). If there is an assumption "it's JSON-LD, so all JSON-LD rules apply", it's problematic since then one needs to go to read the JSON-LD specification... And maybe quit and wait for someone to write one, like the one linked to this thread. :)
• If some rule is DID specific and examples of how it is intended to be used. This maybe means lifting some discussion off of GH threads.

Being a rather rookie with JSON-LD, I think the issue on a strongly typed language is that do I have classes for the types to which parse them, because in business logic parts it would nice to make use of the type system. Maybe not even full types but only to parts I care about and if there's "unknown data", I might want to just store it into a in-memory like structure that can be queried for some attributes.

Here it would help if:

1. It were clear in which DID sections one can refer to open types like from https://schema.org. Since in this case it can be indicated to the parser that if it will be an unknown type, do not throw but parse it to a JSON fragment like structure (as I did in my quick'n'dirty implementation).
2. Which parts are "closed" so this does not happen.
3. Explicit mention to registered extensions since these have some shape but the details may be unknown. I included some of the already in the quick'n'dirty parser with the intention there can be any kind of them and it will be up to the library provide concrete handlers or it can be also application specific (e.g. pass data to a function that can load appropriate libraries to handle the data as it does have some shape).

What feels problematic is that if it is implied or really needed to have a full JSON-LD processor is needed to all parts the DID documents when parsed. Since it might be possible to come along without one if the rules were possible to make explicit. I do not know if this is possible but maybe useful to be explicit about this.

Specifically sticky point is serialized format. It would be easier to explicit if and/or when a format like compacted (looks easiest to parse) is OK. In my case this is because, e.g. with type in this case I know to expect that type and then have a dictionary with lambda functions to choose from which class the runtime object should be parsed (it can be a base class with some open details if it's, say, an extension). It feels to me at least that now it was relatively easy to take an example, read a bit of the specification and implement the serialization to strong type (or a more general data structure if there is not a strongly typed format for it).

(*1) Currently they're Uri objects where possible, fragments are at least currently as strings since they aren't URIs and .NET would throw and they're not resolved, at least now, relative to any base.

[OR13]

I am trying to parse a question to answer out of this.

You don't need to use a JSON-LD parser to parse JSON... anything that passes JSON.parse is JSON.

[veikkoeeva]

@OR13 You may not need to parse a question out of that. Maybe it is better put so that it is notes about writing code to support the DID Core specification as written now and coming to GH issues to figure out more. It is certainly about thinking

that if I use abstract base class in verification relationships, can I assume some basic structure.

Or something along those lines in various parts. I maybe have misunderstood something. It might be I should ask something but I don't know it yet. This may be a wrong place to jot down such notes. I might have related questions when I get back into writing more in a few weeks.

[dlongley]

@OR13, @veikkoeeva, I think this is related to the WG's current discussion around adding guidance for processing JSON-LD as JSON, whether that be guidance added to the JSON-LD representation and/or guidance added to the JSON representation (or guidance for a single unified representation). I'm very much in favor of providing rules for the simplest way to process the presence of @context in both/either representation.

@OR13, some of what @veikkoeeva brings up relates to our discussion over @base -- and how it can simplify things for implementers if we require full URLs to be present when a DID Document is returned from resolveRepresentation.

@veikkoeeva -- So, I think the WG is presently considering more language that is related to this issue that may end up addressing your concerns.

[veikkoeeva]

@dlongley Yes, those points and some thoughts about implementing the standard in general.

For this particular case I would say it does not matter if it is just #key-1 as in the examples if it's clearly spelled out what is the base IRI in that case. Of it the base can changed using @base. If in the specification JSON-LD is assumed, a person like me either assumes that specification needs to be read and understood or just knowingly making an omission that for the time being not to support it and maybe assume some implementations will not be compatible and that needs to be thought about later (e.g. implement the feature, wait for a JSON-LD processors to mature in .NET world and use them etc.).

Actually I realized now that I could comment on JSON.parse by @OR13. If we assume parseable JSON, in .NET there isn't anything I would need to do to parse the document. Just parse it. But then I would be stuck at querying the JSON tree structure and it is not that friendly. So I created a class structure that represents the the DID specification in a way I think makes it easier to handle in code. For that I have to assume something about that the serialized JSON format. Also when serializing the object tree the target format matters.

[OR13]

@dlongley @veikkoeeva regarding relative URIs (fragment ids).... I deal with them all the time... and they are 100% more annoying that just forbidding them.... pretty much everywhere you need to make sure a verificationMethod is resolvable, you will be doing stuff like this:

https://github.com/transmute-industries/vc.js/blob/master/packages/vc.js/src/vc-ld/__tests__/vc-js.sanity.test.ts#L7
https://github.com/transmute-industries/vc.js/blob/master/packages/vc.js/src/__fixtures__/vendors/digitalbazaar.ts#L7

thanks for the requirement to support them from @csuwildcat and @dhh1128 ... :)

If we forbid relative refs, everything would be easier.... yes.... but I imagine there would be significant push back to doing that now.

[veikkoeeva]

@OR13 It's not that bad, maybe. :) In spirit of syncing minds with people reading this,

I was thinking to handle this particular case at https://github.com/veikkoeeva/DotDecentralized/blob/main/src/DotDecentralized.Core/DidDocument.cs#L352, hence it doesn't use Uri. The plan is to wrap this there and make it transparent to end user either way. This does cause a bit of gymnasics, so treating it like https://github.com/veikkoeeva/DotDecentralized/blob/main/src/DotDecentralized.Core/VerificationRelationshipConverter.cs#L75. E.g. this prevent that VerificationReferenceId or EmbeddedVerification does not to get serialized out.

There will be more, like a builder pattern, more checks to prevent illegal states etc. but have to start from somewhere.

[OR13]

I don't know what too do with this issue, since we have examples for both normal and fragment identifiers i suggest we close this.

[msporny]

No objections to closing issue from @veikkoeeva or WG members since pending close tag was added. Ultimately, the WG couldn't determine what changes could be made to the specification that would address @veikkoeeva's concerns. Closing.