Explain how "in-between" resources for the "include" parameter should be handled #497
Comments
|
I believe the {
"type": "articles",
"id": "1",
"title": "Rails is Omakase",
"links": {
"author": {
"self": "/articles/1/links/author",
"related": "/articles/1/author",
"linkage": { "type": "people", "id": "9" }
},
"comments.author": {
"self": "/articles/1/comments/links/author",
"related": "/articles/1/comments/author",
"linkage": [{
"type": "people",
"id": "11"
}, {
"type": "people",
"id": "12"
}]
}
}
}We do need an example of this in the spec, for sure. I'm trying to set up a separate section for advanced examples. This one would fit right in. |
|
Right, but wouldn't that require more changes to the format? For example, if I look at the definition for
The word "association" appears one other time in the entire format description - in an unrelated note. I'm new to JSON API and although I like the format, I find it hard to pinpoint definitions such as these. |
|
@gmta, you're right. Definitions and terminology have not been strong points of JSON API, but lately there's been huge progress towards fixing all that. Let's keep the issue open until this case has been addressed, too. As for pinpointing definitions, I'm planning a PR that overhauls the formatting of all primary definitions, but I can't submit it right now with so many PRs pending. |
|
So, I started working on this, and it turns out it's not so simple. I think this question should be milestoned for 1.0 (unless there's already a definitive solution I'm not aware of). What if we have Should the It seems a bit complicated if the presence of One could also expect the outcome to be governed by the To solve this without considering Example:
{
"data": [{
"type": "articles",
"id": "1",
"title": "JSON API paints my bikeshed!",
"links": {
"author": {
"linkage": { "type": "people", "id": "9" }
},
"comments": {
"linkage": [
{ "type": "comments", "id": "5" },
{ "type": "comments", "id": "12" }
]
}
}
}],
"included": [{
"type": "people",
"id": "9",
"first-name": "Dan",
"last-name": "Gebhardt",
}, {
"type": "people",
"id": "11",
"first-name": "Bin",
"last-name": "Toro",
}, {
"type": "people",
"id": "12",
"first-name": "Leeroy",
"last-name": "Jenkins",
}, {
"type": "comments",
"id": "5",
"links": {
"author": {
"linkage": { "type": "people", "id": "11" }
}
}
}, {
"type": "comments",
"id": "12",
"links": {
"author": {
"linkage": { "type": "people", "id": "12" }
}
}
}]
}The important thing here is that the included comments above do not contain the comment body or any other info apart from the
While this system would get rid of the dotted-notation "deep relationships" within links objects, it is somewhat incompatible with default includes. Since |
|
See http://github.com/endpoints/example for an actual working implementation of this. The following is not fully compatible with RC3, but it shows how I think this should work: GET /authors/1 {
"data": {
"id": "1",
"name": "J. R. R. Tolkien",
"date_of_birth": "1892-01-03",
"date_of_death": "1973-09-02",
"type": "authors",
"links": {
"books": "/authors/1/books",
"books.chapters": "/authors/1/books.chapters",
"self": "/authors/1"
}
}
}GET /authors/1?include=books {
"data": {
"id": "1",
"name": "J. R. R. Tolkien",
"date_of_birth": "1892-01-03",
"date_of_death": "1973-09-02",
"type": "authors",
"links": {
"books.chapters": "/authors/1/books.chapters",
"books": {
"type": "books",
"id": [
"1",
"2",
"3",
"11"
]
},
"self": "/authors/1"
}
},
"included": [
{
"id": "1",
"date_published": "1954-07-29",
"title": "The Fellowship of the Ring",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "1",
"resource": "/series/1"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/1/chapters",
"firstChapter": "/books/1/firstChapter",
"stores": "/books/1/stores",
"self": "/books/1"
}
},
// ...
{
"id": "11",
"date_published": "1937-09-21",
"title": "The Hobbit",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "null"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/11/chapters",
"firstChapter": "/books/11/firstChapter",
"stores": "/books/11/stores",
"self": "/books/11"
}
}
]
}GET /authors/1?include=books.chapters {
"data": {
"id": "1",
"name": "J. R. R. Tolkien",
"date_of_birth": "1892-01-03",
"date_of_death": "1973-09-02",
"type": "authors",
"links": {
"books": "/authors/1/books",
"books.chapters": {
"type": "chapters",
"id": [
"1",
// ...
"289"
]
},
"self": "/authors/1"
}
},
"included": [
{
"id": "1",
"title": "A Long-expected Party",
"ordering": 1,
"type": "chapters",
"links": {
"book": {
"type": "books",
"id": "1",
"resource": "/books/1"
},
"self": "/chapters/1"
}
},
// ...
{
"id": "289",
"title": "The Last Stage",
"ordering": 19,
"type": "chapters",
"links": {
"book": {
"type": "books",
"id": "11",
"resource": "/books/11"
},
"self": "/chapters/289"
}
}
]
}GET /authors/books?include=books,books.chapters {
"data": {
"id": "1",
"name": "J. R. R. Tolkien",
"date_of_birth": "1892-01-03",
"date_of_death": "1973-09-02",
"type": "authors",
"links": {
"books": {
"type": "books",
"id": [
"1",
"2",
"3",
"11"
]
},
"books.chapters": {
"type": "chapters",
"id": [
"1",
// ...
"289"
]
},
"self": "/authors/1"
}
},
"included": [
{
"id": "1",
"date_published": "1954-07-29",
"title": "The Fellowship of the Ring",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "1",
"resource": "/series/1"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/1/chapters",
"firstChapter": "/books/1/firstChapter",
"stores": "/books/1/stores",
"self": "/books/1"
}
},
{
"id": "2",
"date_published": "1954-11-11",
"title": "The Two Towers",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "1",
"resource": "/series/1"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/2/chapters",
"firstChapter": "/books/2/firstChapter",
"stores": "/books/2/stores",
"self": "/books/2"
}
},
{
"id": "3",
"date_published": "1955-10-20",
"title": "Return of the King",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "1",
"resource": "/series/1"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/3/chapters",
"firstChapter": "/books/3/firstChapter",
"stores": "/books/3/stores",
"self": "/books/3"
}
},
{
"id": "11",
"date_published": "1937-09-21",
"title": "The Hobbit",
"type": "books",
"links": {
"series": {
"type": "series",
"id": "null"
},
"author": {
"type": "authors",
"id": "1",
"resource": "/authors/1"
},
"chapters": "/books/11/chapters",
"firstChapter": "/books/11/firstChapter",
"stores": "/books/11/stores",
"self": "/books/11"
}
},
{
"id": "1",
"title": "A Long-expected Party",
"ordering": 1,
"type": "chapters",
"links": {
"book": {
"type": "books",
"id": "1",
"resource": "/books/1"
},
"self": "/chapters/1"
}
},
// ...
{
"id": "289",
"title": "The Last Stage",
"ordering": 19,
"type": "chapters",
"links": {
"book": {
"type": "books",
"id": "11",
"resource": "/books/11"
},
"self": "/chapters/289"
}
}
]
} |
|
Whoops, our implementation doesn't include inter-linking within the included records, but it will soon (e.g. each book in included should list all chapter ids under the linkage when both are requested).
It doesn't matter if the associations appear separately or in both places--a client can resolve the relationships in either case. I suppose it would be good to provide a SHOULD or MUST picking one, though. |
Sure. I was trying to see if there's a better way to accomplish uniform representations in the various cases. If we're sticking with dot-separated association names in
Once there's an answer to this, I can put together a PR that explains it. Also, we need a place for an example, which is going to be too marginal to include in the base spec. @tkellen, what do you think about repurposing the Examples page to hold some advanced usage examples? |
|
I think the first option is the one we want. I don't think:
...is actually bad. I also doubt that the payload size would appreciably decrease after gzipping in the second option. I'm 👍 on having expanded examples. I've yet to discuss this with @dgeb, @steveklabnik or @wycats, but I want to modify all the examples to use http://github.com/endpoints/fantasy-database soon (and probably move that repo under the json-api org), and to host a reference implementation of it (probably using endpoints). I also plan to provide a test suite that anyone can implement against using that reference database. I think that will go a long way to alleviate concerns about lack of examples. |
|
So, my understanding of this has been different than @bintoro's and @tkellen's. I assumed it would work as follows:
So, for example, a request to
This means that the client can't link the authors to their comments, but this may not matter. E.g. the client may just want to populate a "Recent Commenters" widget and not care who commented on what. Then, if the client does want to be able to join the authors to their comments, it simply must do |
|
@ethanresnick I have deployed applications using nested relations. It's very useful to reach into them without including the intermediary records. Also, the spec already mandates dot-notated links keys:
|
|
@tkellen I actually read that part of the spec as suggesting that dot-separated keys should not be used in Re it being useful to reach into nested relations without including the intermediate records: I can see that. It seems like the main advantage would be simpler processing on the client side, and possibly a smaller payload (though the payload size can usually be mitigated with the |
|
I would like to vote for a combination of points suggested above, and also discuss how the conflict/ambiguity of TL;DR I think it is best not to impose any implicit requirements on inclusion of related resources and instead rely on explicit
I would like to propose the following option: server should only return what has been explicitly requested either via query parameters An extended description:
Motivation:
I would like to illustrate the 3rd point with an example. Imagine a generic server-side JSON API library to fetch data and build responses, which follows the approach described above. Let the default for
Imagine an API implementation which wishes to use this library. That API has a custom convention: for every relationship, either both link object and corresponding related resource (in IMHO, adherence to explict values of
@ethanresnick, not sure I understand. Since attributes and relationships share namespace within a resource, could such a conflict even exist? For the example of I would like to join others in requesting to milestone this issue for 1.0. |
@hhware Sounds like we're on the same page about how this should work overall. To your particular question about my remark on complex attributes: I didn't mean to suggest that using dot-separated |
|
As a general note: if we wanted to make it easy for clients to use the raw payload to randomly access Then The fact that we didn't use this representation implies to me that we expect that clients will loop over the Therefore, when it comes to allowing dot-separated That still leaves the question of payload size—i.e., not allowing dot-separated keys means the client has to include each intermediate resource if it wants to be able to connect the primary resources to the lowest-level included ones—but again:
Meanwhile, the advantage of not allowing dot-separated keys for now is that it makes the meaning of the keys in I definitely haven't thought about this enough/considered all the consequences of either route, but my strong gut reaction is that dot-separated keys are really not a road we want to go down for the base spec. |
|
This might be a bit off topic but recently new technologies like Relay (incl. GraphQL), Falcor (Netflix), Datomic, ... are popping up which all of them allows to define queries to retrieve a complex data-tree within a single response containing all the data required e.g. by a UI component. For me this sounds very similar to what can be achieved via the Something that's not really clear for me is how to handle pagination / limitation of included resources in JSON API. In GraphQL you'd be able to fetch only a range of items from any "in-between" resource (see example Or would you say such complex data queries are not necessary or out of scope from JSON API and you'd be better off issuing several GET requests to fetch the resources separately and then build the result via code? |
|
@nevson I think that is outside of the scope of this specification. That is what the |
|
@nevson, IMHO, necessity of several GETs is not implied. I think it is within the scope of the spec:
The spec just does not define the exact way of requesting it. |
|
I'm not sure what to do about what I'm about to write, but I'm leaving it here for posterity. I would argue that in any case where you cannot link the data in From @ethanresnick's example:
...you don't actually care about the events, you're just "hacking" JSON API to get the orgs, yeah? |
|
@tkellen I want to re-read this issue and think about it more closely but, just as a tentative response to your comment: yeah, I was "hacking" for the orgs in my example. And you might be right that, in general, the use cases in which the client can't link I'm not sure what the implications of this are though... |
@tkellen, I do not think it is always the case. Another consideration along the lines of what @ethanresnick is talking about: suppose one wants to display a set of resources along with some statistical information about them, which is based on attributes/number/types/etc of their related resources. So both primary IMHO, there is no reason for the spec not to be flexible in this case and require (potentially tons of) in-betweens to be shipped to the client along with useful information... |
|
As I see it, the core tradeoff here is: A) A clear normative requirement that linkage data be included to connect primary and included resources. vs. B) The flexibility to allow clever ad hoc queries in which clients can intuit connections between primary and deeply nested resources, without the need to include intermediate resources. The value of A is undeniable. It ensures that servers provide the linkage necessary to connect primary and included resources. It ensures that the included resources are in fact related to primary resources. It means that compound documents will always be fully "connected". The value of B is more tenuous. It provides some potential for minimizing request count and payload size. However, it is truly not a generalizable benefit. It only "works" in certain cases for certain requests in particular applications. I had a long discussion with @tkellen and @lgebhardt today about this tradeoff. We were unable to arrive at a best-of-all-worlds solution that provided both A and B. We discussed options such as complex relationship paths (e.g. Therefore, we are leaning heavily toward providing benefit A for everyone with a simple and clear requirement. The alternative - to remove that normative requirement for linkage data - would mean that compound documents could be "broken". Some implementations would choose to not provide linkage data, even for parent-child relationships, and that seems like a much worse consequence than losing the potential benefit B for certain select cases with certain select implementations. This issue, which seems rather simple on the surface, has turned out to be surprisingly tricky. |
|
@daliwali, @hhware Thanks for your feedback :)
How could an exemplary implementation look like for my example mentioned in #497 (comment) ? Honestly this is to me a bit unclear.
@dgeb Sorry for asking but would that mean that a request for |
|
@nevson As an API implementor, you'd internally create an alias, like |
The format documentation states the following:
Now if the relation
comments.authorreferences to resources of typepeople, how do I know that the resources in theincludedkey reference thecomments.authorrelationship? In other words, how would the response look like if I were to execute the following request:Given that there exists a link called
authorthat references a resource of typepeople. How would I differentiate between the two relations when I do not have any information about thecommentresources that live in between thearticleandpeopleresources?I could not find anything explaining how I should handle such a request and I think that this paragraph could include an example.
The text was updated successfully, but these errors were encountered: