Remove reliance on external URL by allowing primary context to be a JSON Object

[yancyribbens]

Working with examples in the test-suit has presented continuous problems, either resolving URIs that should be functional (down at time of writing), specifying the correct content type, and test-suit examples that aren't hosted.

From the spec:
The value of the @context property MUST be an ordered set where the first item is a URI with the value https://www.w3.org/2018/credentials/v1

If we allow inline context in the spec, the test-suite examples could also be modified to inline context, offering a simple way to present cached content to ease the burden on implementors during CR and later implementations.

[msporny]

If we allow inline context in the spec, the test-suite examples could also be modified to inline context, offering a simple way to present cached content to ease the burden on implementors during CR and later implementations.

That change would require JSON-LD processing for all processor classes (a normative change which we're trying to avoid right now), and it would result in less consensus on the spec (and most likely a larger number of formal objections when we try to transition)... because it will force everyone to use a JSON-LD processor.

Note that the specification currently states:

The data available at https://www.w3.org/2018/credentials/v1 is a static document that is never updated and SHOULD be downloaded and cached.

It also states:

JSON-based processors MUST process the @context property, ensuring the expected values exist in the expected order for the credential type being processed. It is advised that the expected order of values of the @context property should be defined by at least a human-readable extension specification and preferably by a machine-readable specification as well.

We also have these PRs pending that clarify this further in the spec:

#546
#548

As the spec is currently written, there is no requirement to dereference/fetch the URLs. It is also recommended that you cache the contents of the contexts you care about and load them from disk/memory.

I believe that doing so would address all of the issues you raised above. Given the language above, and that at least one of the PRs above being merged, it would require no further changes to the specification. Does that work for you, @yancyribbens?

[rhiaro]

That change would require JSON-LD processing for all processor classes

Not necessarily. If we specify the exact character string of the JSON object that should be used for the inline context, that's equivalent to specifying the URL so no additional processing of the context would be required, right?

This would give implementations which object to leaning on the w3.org URL for any reason a possibility to include the context in a way that JSON-LD processors would still understand, and JSON implementations could still string-match on as before.

And it makes it consistent with the fact that other (non VC core) contexts can be included inline as objects too.

On a related note - especially since we're telling people they don't have to fetch the context and should cache it - should we have a copy of the context in an appendix in the spec anyway? Then nobody ever has to fetch the context URL even once, if they don't want to. (Happy to make a PR for this if so.)

[yancyribbens]

Does that work for you, @yancyribbens?

My primary objective is to see if @context can be modified such that if you're consuming a vc for the first time and haven't already downloaded the context, you won't need to because the context is provided inline.

The data available at https://www.w3.org/2018/credentials/v1 is a static document that is never updated and SHOULD be downloaded and cached.

If the context is provided inline, then implementors have a stronger way of knowing the credentials/v1 will never change and is implicitly cached.

Something I don't know is if credentials/v1 is provided inline, will the @context be excessively large if all external references needed are also provided inline? For example, credentials/v1 relies on security/v2 etc which would also need to be provided inline to prevent the downloading of any context.

On a related note - especially since we're telling people they don't have to fetch the context and should cache it - should we have a copy of the context in an appendix in the spec anyway? Then nobody ever has to fetch the context URL even once, if they don't want to. (Happy to make a PR for this if so.)

+1 to adding a copy of the context in the appendix. We could also include a hash of the context in the spec to assert the context is in a certain byte order and hasn't been changed.

I understand not wanting to introduce any normative changes, and I'm not proposing this as a blocker necessarily, however, if a second CR is needed this is still a change that would add value.

[dlongley]

@yancyribbens,

My primary objective is to see if @context can be modified such that if you're consuming a vc for the first time and haven't already downloaded the context, you won't need to because the context is provided inline.

Just make an in-memory map in your consuming software that maps the string to the full context. Once the spec reaches REC it won't change (but we can expect some tweaks during CR based on implementation feedback). There's no reason for you to ever retrieve it (not even once). Just hard-code all contexts from the spec and you're done.

[rhiaro]

For example, credentials/v1 relies on security/v2 etc

Where "etc" is security/v1 which relies on Dublin Core, which relies on ..?

[rhiaro]

Just hard-code all contexts from the spec and you're done

@dlongley I think part of the problem here is that the contexts are not in the spec..

[yancyribbens]

@dlongley I agree that may be the best way, however, I thought we could reduce the cognitive burden of requiring caching at the software level by instead providing the context inline or at least, in the spec.

[dlongley]

+1 for eventually having an appendix with the contexts in the spec, keeping in mind Issue 1 we noted in the spec:

The specific location and content in the JSON-LD Context URLs used throughout this document may change during the W3C Candidate Recommendation phase based on implementer feedback. The test suite will reflect any changes to the URLs or content located at the URLs.

[rhiaro]

@dlongley including all the nested contexts, down to Dublin Core and whatever that points to?

[dlongley]

There is no use of a dublin core context. There are presently only three contexts: security/v1, security/v2, and credentials/v1. The security/v1 context only references the dublin core vocabulary in term mappings, it does not include another context for it.

[TallTed]

Do I understand this thread to be saying not only that there are now only three contexts, but that is all there ever will be ("Just hard-code all contexts from the spec and you're done")?

I think that clearly stating this at the points where @context enters the discussion -- especially but not only in regards to JWT and JSON and JSON-LD -- would make those points much easier for many to digest and accept.

That is, requiring JWT users to choose between 3 specific URIs (and the immutable content to which they dereference, which is also visible in an appendix to the spec) for @context is a substantially lower burden than being required to mint and populate one's own @context URI -- which indeed would require understanding the nuances of JSON-LD.

[msporny]

@TallTed wrote:

Do I understand this thread to be saying not only that there are now only three contexts, but that is all there ever will be ("Just hard-code all contexts from the spec and you're done")?

No, that is not correct. There will always be the base context (which is an amalgamation of 3 different contexts credentials/v1, security/v1, and security/v2) plus one or more application / market vertical-specific contexts.

[msporny]

@yancyribbens wrote:

@dlongley I agree that may be the best way, however, I thought we could reduce the cognitive burden of requiring caching at the software level by instead providing the context inline or at least, in the spec.

We should not provide the context inline (as an object associated with @context) for at least two reasons:

1. Doing so would bloat every verifiable credential and verifiable presentation by 1.8 KBs. Consider that Example 1 in the specification is around 900 bytes in size, developers would rightfully complain that there is 200% overhead by inlining the @context. This is further exacerbated by megacontexts, such as schema.org, which weighs in at 139KB.
2. JSON implementers would need to either 1) use a JSON-LD processor, or 2) do a deep compare of the object w/ one that they have in memory to determine if the context is the one they think it is.

Based on these two reasons, -1 to inlining the context as an object.

I thought we could reduce the cognitive burden of requiring caching at the software level by instead providing the context ... in the spec.

We could do this, but I think what I'd rather do is:

1. Flatten all of the contexts into the one served at https://www.w3.org/2018/credentials/v1.
2. Provide a SHA-256 hash of the URL above in the spec as we go into PR.

If we do that, developers can curl https://www.w3.org/2018/credentials/v1 > credentials-v1.jsonld and then check it w/ sha256sum.

Would that address your concern, @yancyribbens?

[stonematt]

From VCWG call on April 30, 2019:
RESOLUTION: Issue #549 should be addressed by flattening all contexts that the specification depends on into https://www.w3.org/2018/credentials/v1, adding the JSON-LD Context into an appendix in the spec along with its SHA-256 hash, all of which would be a non-substantive modification. Issue #549 should be closed after these actions are taken.

[yancyribbens]

@stonematt did you mean "non-normative" instead of "non-substantive"?

[msporny]

@stonematt did you mean "non-normative" instead of "non-substantive"?

No, we meant "non-substantive". :)

We are changing normative things in the spec (and around the spec, like flattening the context) in ways that some might thing will affect implementations, but will in fact not do that. So, some would argue that we are making changes to normative sections of the specification, but in ways that will not change what implementers have to do. In these cases, we refer to those changes as "non-substantive". Hope that helps explain the particular word choice in the RESOLUTION.

[yancyribbens]

PR has been created here #587

[burnburn]

@yancyribbens I think this is complete. Can you confirm? If so, please just go ahead and close.