Linking instance documents and context documents

[lanthaler]

To have a smoth upgrade path from simple (existing) JSON APIs/feeds to JSON-LD we should consider adding a HTTP Link header (RFC5988) to link from the plain JSON document to the JSON-LD context document. I would suggest a link in the following form:

Link: <http://www.example.com/context.jsonld>; rel="describedby"; type="application/ld+json";

or (I'm not sure if MIME type parameters are allowed)

Link: <http://www.example.com/context.jsonld>; rel="describedby"; type="application/ld+json;format=context";

To link from a context document to an instance document we could add a "data" or "instance" MIME type parameter. E.g.

Content-Type: application/ld+json; data=http://www.example.com/context.json

There were also some discussion on the mailing list to add a format=context parameter to the MIME type. So it might look as follows:

Content-Type: application/ld+json; format=context; data=http://www.example.com/context.json

[msporny]

Still don't know how I feel about this. The upside is that this would be a very clean way for people to publish the context along-side their regular JSON data. The downside is that using HTTP headers have been something that many developers don't know how to use correctly, or do not have access to in their development environments. If we were to implement this, we could depend on large publishers using the feature, but smaller web design shops may not know how to work with HTTP headers.

We may just want to tell people that putting a Link header is RECOMMENDED but not require it in any way. In other words, it is a hint to JSON-LD applications, but not required. REST Web Service developers may also specify what the default context is in their API documentation as a completely out-of-band way of specifying how to process the JSON-LD document.

[lanthaler]

I think we should specify it and say that the (link to the) context has to be EITHER place in the document OR in the link header OR both. We should then make it clear that just having the link header keeps the JSON data clean but might be problematic at it is not accessible in some environments, i.e., JavaScript in the browser.

Just because some people can't use it shouldn't stop us to allow people using it that need it to e.g. evolve existing systems.

-----Original Message-----
From: Manu Sporny [mailto:reply+i-1477704-
4238172095ef71dbe62b376df9f4cab1a5b542ef@reply.github.com]
Sent: Sunday, September 11, 2011 10:49 PM
To: Markus Lanthaler
Subject: Re: [json-ld.org] Linking instance documents and context
documents (#16)

Still don't know how I feel about this. The upside is that this would
be a very clean way for people to publish the context along-side their
regular JSON data. The downside is that using HTTP headers have been
something that many developers don't know how to use correctly, or do
not have access to in their development environments. If we were to
implement this, we could depend on large publishers using the feature,
but smaller web design shops may not know how to work with HTTP
headers.

We may just want to tell people that putting a Link header is
RECOMMENDED but not require it in any way. In other words, it is a
hint to JSON-LD applications, but not required. REST Web Service
developers may also specify what the default context is in their API
documentation as a completely out-of-band way of specifying how to
process the JSON-LD document.

Reply to this email directly or view it on GitHub:
#16 (comment)

[lanthaler]

Having spend some more thoughts on this I think a good compromise would be to specify the link header for JSON-LD _and for plain old JSON_. We could then define the header to be optional for JSON-LD while keeping the @context directive mandatory.

For plain old JSON documents @context is not supported and so the link header is the only way to reference a context document to transform the data into linked data. This would also provide a viable upgrade path for existing systems as changing the MIME type would probably break some clients.

[lanthaler]

RESOLVED: The relation name used in a Link header to associate a JSON-LD context with a JSON document will be "describedby".

RESOLVED: A JSON-LD document MUST have all context information required for processing within the body of the document.

[gkellogg]

Current thought seems that "profile" is a better link relation term. Here's the conversation from Twitter:

Jeni Tennison: @ldodds Been thinking about overlap between JSON-LD's context and @dret's profile link relation proposal http://tools.ietf.org/html/draft-wilde-profile-link-02.

Leigh Dodds: @JeniT yes. if served as app/json then could reference a profile URI to indicate its JSON-LD as well as the context via described by...but..

Leigh Dodds: @JeniT JSON-LD could use a profile for syntactic variations too. Also wondering if profile could be used instead of describedby for context?

@gkellogg: @JeniT @ldodds @dret we've considered using the profile link relation in favor to described by for JSON-LD. Feedback useful!

Leigh Dodds: @gkellogg @JeniT @dret with specific reference to: json-ld.org/spec/latest/js… I think the profile link relation makes more sense than described by
... @gkellogg In general best to use the json-ld media type, but where app/json is required a profile feels more accurate, IMO /cc @JeniT @json-ld

@gkellogg: @ldodds @JeniT @dret thanks, I'll add the use of profile LR for JSON-LD as an issue. We were just waiting for some direction.

[gkellogg]

@msporny @lanthaler @dlongley I don't think this is too controversial. Unless anyone objects, I'll just change the link relation from "describedby" to "profile".

I seem to recall that @msporny also had had an email conversation with Mark Nottingham, but it wasn't conclusive.

[dret]

i don't think i fully understand the context of this decision, but i am of course glad that "profile" already sees some uptake. if you think there's anything that needs to be clarified, or if you have any questions, please let me know. also, i am not sure what the current timeline is for JSON-LD, but if it's due soon, then adding it as an example would be great. but i don't want to do this until the spec is stable enough.

[ldodds]

Hi @gkellogg thanks for responding to my twitter comments. I want to expand on those a little here to outline my thinking and also raise an issue that @dret may have some useful input on.

JSON-LD currently has its own media type. Which has enough additional semantics that it should be preferred over use of application/json.

However one of the goals is to also allow service providers to "retrofit" the additional semantics onto existing JSON services. The correct way and recommended way to do that, IMO, is to alter those services so that they support negotiation for application/ld+json as well as application/json.

This would allow a client to request the media type it desires. Use of the Vary header can also indicate to existing clients that alternate formats are (now) available.

But the specification notes also notes that there's a desire to allow APIs to retain use of application/json but indicate that additional semantics are available. E.g. the document could also be interpreted as JSON-LD.

Currently this is achieved by referencing the JSON LD context from a describedby link. The describedby link is also used elsewhere in the specification to reference external contexts required to properly process an application/ld+json document.

The flagging of additional semantics from an application/json document feels more like a profile to me. So this is what prompted my suggestion to switch from describedby.

However the issues that I want to flag to @dret are:

• This approach would mean that there isn't a single static identifier to indicate the profile: the context URI will vary across services. However the type of the link relation (application/json+ld) should be enough for clients to detect the profile.
• A client will almost certainly need to do a GET on the profile URI, to fetch the relevant context. This breaks the SHOULD NOT recommendation in the draft document.

Would be useful to get @dret's feedback on those issues before making changes.

An alternative approach would be to separate out identification of the profile from the linking to the context. Arguably these are different things. This would mean retaining describedby for linking to context (in any scenario) but, in the case where we're serving application/json that can be processed as application/json+ld, we also include a profile link, e.g:

Link: <http://www.example.com/context.jsonld>; rel="describedby"; type="application/ld+json";
Link: <http://json-ld.org/spec>; rel="profile"

This is obviously more complex than using a single header, but may fit better with the intent of the profile link type.

[lanthaler]

I think two things are conflated here and @dret might might be a bit confused because I was discussing something different but related with him recently :-) Let me try to explain.

JSON-LD should be served as application/ld+json whenever possible. Furthermore we state in the spec

JSON-LD documents served with the application/ld+json media type must have all context information, including references to external contexts, within the body of the document.

So, if we use "the describedby link ... elsewhere in the specification to reference external contexts required to properly process an application/ld+json document" as @ldodds states, that's clearly an error (I couldn't find it though).

JSON-LD's processing model is quite a bit different from traditional JSON, so I think a separate media type to signal that is the way to go. Nevertheless, JSON-LD often looks like plain-old JSON (and that was a design goal btw). Therefore I proposed to use the describedbylink relation to link a context to a JSON document so that JSON-LD capable clients can make smarter use of such a document. Please note, that it is still a fully valid JSON document and can be used as such.

The profile link relation as proposed by Erik is used to add a semantic layer on top of (existing) media types such as saying that an Atom feed represents a podcast e.g. - at least that's how I understand it. It is not intended to change the processing model of media type. JSON-LD is a format to express linked data graphs, JSON expresses trees. Furthermore, as Leigh already outlined, the profilelink relation should be constant, i.e., it should signal that that JSON document can be interpreted as JSON-LD but it shouldn't have a different value for every single document as we use the describedby relation at the moment. I think we should keep the describedby link relation for referencing contexts from plain old JSON documents, the context really describes the document, so I think it's a reasonable choice.

On the other hand, I think adding a profile media type parameter to JSON-LD would make sense as well. That's something I was discussing with Erik recently. If you look at Web APIs, it could for example be used to signal the semantics without having to parse and process the whole document first or to request the data in a specific form. For example a client could express whether it would like to get the data in the form of FOAF or schema.org. Another use case could be versioning, service evolution etc. I could also envision that in the future "service standards" emerge that are purely modeled at the data layer on top of JSON-LD, e.g. So the header of a JSON-LD response could look something like Content-Type: application/ld+json; profile=http://socialnetworking.com/federationapi

Summarized, I think it would be better to keep the describedbylink relation and look at the potential of a profile media type parameter.

[gkellogg]

I can certainly see the advantage of keeping describedby and adding a separate profile link relation. Describedby is only necessary when the content type is something other than application/ld+json. In fact, it's use when the content type is application/ld+json might be confusing, as the document MUST also contain a profile, and if the two forms were used, both MUST be loaded and I think that the document itself is ment to be self-describing.

Adding a profile argument to content type is something else entirely, and should probably be the subject of a different issue. However, the idea that such a parameter could change the actual content of the document, say from using schema.ort to FOAF, is not something I'm ready to accept.

[lanthaler]

I can certainly see the advantage of keeping describedby and adding a separate profile link relation. Describedby is only necessary when the content type is something other than application/ld+json

And this is how it is specified at the moment.. a way to link a context to a JSON document via an HTTP link header.

In fact, it's use when the content type is application/ld+json might be confusing, as the document MUST also contain a profile, and if the two forms were used, both MUST be loaded and I think that the document itself is ment to be self-describing.

I posted the current wording of the spec already above, but here it is again. I think that explains the usage of describedbyquite clearly, i.e., they are not needed for application/ld+json and can be safely ignored:

JSON-LD documents served with the application/ld+json media type must have all context information,
including references to external contexts, within the body of the document.

Adding a profile argument to content type is something else entirely, and should probably be the subject of a different issue.

Agree. I was discussing that with Erik and that's the only reason I mentioned it in this context. I thought he might be a bit confused because you contacted him about something related, yet completely different.

However, the idea that such a parameter could change the actual content of the document, say from using schema.ort to FOAF, is not something I'm ready to accept.

I wasn't able to get my head around it yet.. that's why I didn't bring it up yet. I don't think we should address this now but might be something we could do in the future.

Do you still believe the link relation should be changed? Otherwise I would go ahead and close this issue.

[dret]

reading through @lanthaler's comment i have to say that i agree with him that for the scenario he is describing, "profile" would not be a good choice. "profile" is intended to be used when clients can happily ignore it, but if they find one they know, it allows them to extract a richer representation that they would do without it, but without making any changes to the original parts. it seems to me that "casting" JSON into JSON-LD is a different kind of beast, because it gives you an entirely new view of the representation. it's similar to the XML and RDF/XML situation, it seems to me, where a client radically changes its world view going from one interpretation to the other.
this may be a bit strange as a comparison, but isn't what you're trying to do a little bit like a JSON-oriented GRDDL, and what you're trying to achieve is that both clients of the original as well as clients of the transformed results can be made equally happy? or am i completely misunderstanding what's going on?

[gkellogg]

I agree with @lanthaler, let's stick with described by for referencing the context. Adding something like Link: <http://json-ld.org/spec>; rel="profile" might also be encouraged as a way of describing that a document served with a content type other than application/ld+json adheres to the JSON-LD spec, though. Note that a document might be fed as application/json, and still contain an @context, so the describedby link might not really be needed. In this case, have a 'profile' LR could be useful.

[lanthaler]

That could be a potential use case for profile but I think it's usage would be quite limited and we address legacy JSON documents already well enough with the describedby header IMO.

I'll close this issue now. Feel free to re-open it if you feel something wasn't addressed yet.