Proposed Merger of URL and ID Styles

[lgebhardt]

@dgeb and I are working on tools to generate client and server implementations of JSON API and have been tripped up by incompatibilities in the URL and ID styles.

We'd like to propose a merger of these two styles so that they can co-exist in the same spec without conflict. This proposal slightly modifies the way links and related IDs are handled.

Current Spec

Currently the same keys can be used for either IDs or URLs for a document's links with no indication of what type of data the key contains:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "9",
      "comments": [ "5", "12", "17", "20" ]
    }
  }]
}

or

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "http://example.com/people/1",
      "comments": "http://example.com/comments/5,12,17,20"
    }
  }]
}

Problems with the Current Spec

Using the same keys to store either IDs or URLs creates confusion for producers and consumers of the spec (as shown by the number of issues here related to IDs and links). Given that not all IDs are going to be integer numbers (UUIDs in particular, but there are other key systems in use, some of which may have a passing resemblance to urls), it is non trivial to programmatically determine whether an ID or a URL was provided by an API in a general way.

Using the same keys to store either IDs or URLs in the current manner also precludes the option of providing both IDs and URLs. If related documents are specified by URLs in the primary document and included as linked documents in a compound document, then there is no path to get from a primary document to its included linked documents. For example, a client would not be able to easily determine which comments go with which post in the following example:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "http://example.com/people/9",
      "comments": "http://example.com/comments/5,12,17"
    }
  }, {
    "id": "2",
    "title": "The Parley Letter",
    "links": {
      "author": "http://example.com/people/9",
      "comments": "http://example.com/comments/20"
    } 
  }],
  "linked": {
    "people": [{
      "id": "9",
      "name": "@d2h"
    }],
    "comments": [{
      "id": "5",
      "body": "Mmmmmakase"
    }, {
      "id": "12",
      "body": "I prefer unagi"
    }, {
      "id": "17",
      "body": "What's Omakase?"
    }, {
      "id": "20",
      "body": "Parley is a discussion, especially one between enemies"
    }]
  }
}

Furthermore, if a document's link can only be an ID or URL, there is no way to specify a TYPE without requiring use of the top level links.

Proposed Changes

I propose we remove the ambiguity around the links key in documents by requiring that an attribute in a document's links be one of the following:

• ID - if a string or number
• IDs - if an array of strings or numbers
• link object - an object that can contain one or more of the attributes id, href and type

Default to IDs

In this example, an API consumer can assume that authors and comments are IDs:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "9",
      "comments": [ "5", "12", "17", "20" ]
    }
  }]
}

However, this is no longer a valid representation of URLs:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "http://example.com/people/1",
      "comments": "http://example.com/comments/5,12,17,20"
    }
  }]
}

Support URLs and TYPEs with Link Objects

To handle URLs we will allow each attribute of links to contain a "link object", which can contain any or all of the following attributes: href, id and type. This is discussed in #164.

Simple URL

To return URLs we use a link object and the href attribute. For example:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": { 
        "href":"http://example.com/people/9"
      },
      "comments": {
        "href":"http://example.com/comments/5,12,17,20"
      }
    }
  }]
}

Support for Types

We can add a TYPE like this:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": {
        "href": "http://example.com/people/9",
        "type": "people"
      }
      "comments": {
        "href": "http://example.com/comments/5,12,17,20",
        "type": "comments"
      } 
  }]
}

Also support IDs

We can also add IDs for fully defined links, like this:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": {
        "href": "http://example.com/people/9",
        "id": 9,
        "type": "people"
      }
      "comments": {
        "href": "http://example.com/comments/5,12,17,20",
        "ids": [5,12,17,20],
        "type": "comments"
      } 
  }]
}

IDs with Types

Finally, we can use the TYPE with only IDs if necessary:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": {
        "id": 9,
        "type": "people"
      }
      "comments": {
        "ids": [5,12,17,20],
        "type": "comments"
      } 
  }]
}

There is now no ambiguity between what's an ID and what's a URL.

Less Wordy IDs and URLs together - Templates to the Rescue

Since links can also be defined with URL Templates we still have the opportunity for (relatively) concise responses.

This allows IDs to be converted to URLs using the URL Template Shorthands, such as:

{
  "links": {
    "posts.author": {
      "href": "http://example.com/people/{posts.author}",
      "type": "people"
    },
    "posts.comments": {
      "href": "http://example.com/comments/{posts.comments}",
      "type": "comments"
    }
  },
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "9",
      "comments": [ "5", "12", "17", "20" ]
    }
  }]
}

Because the top level links key is not repeated we can keep the response more concise, while still providing both IDs and URLs to the client. It's then up to the client to decide how best to access these.

Benefits

• Removes ambiguity between URLs and IDs when linking related documents.
• Allows for both URLs and IDs to be specified for related documents.
• Avoids using special suffixes (_id or _href) to determine whether a link is an ID or a URL.
• Allows specifying TYPEs without the use of URL Templates.
• Allows for convergence of the ID and URL styles of JSON API. The ID style becomes a special case of the full style.

Please let me know what you think. If approved, @dgeb and I will be glad to put together a PR and close out all related issues.

[gr0uch]

+1. But what about allowing arbitrary keys on a links object? There may be more meta-data than just the type.

[lgebhardt]

@daliwali I think it would make sense to allow any meta-data fields that are allowed in the top level links. I'm not sure if other keys are allowed there.

What meta-data do you have in mind?

[steveklabnik]

We'd like to propose a merger of these two styles so that they can co-exist in the same spec without conflict.

I very much want this as well.

I really like this proposal. I believe that this would fix #159 as well, and that has some answers as to 'which metadata'.

👍 here. @wycats ?

[gr0uch]

@lgebhardt actually disregard that. I had thought it might be possible to expose extra meta-data per link, like if I have a join table with extra columns describing the relation. Then I realized that it wouldn't work given that the format doesn't allow for more than just the IDs per relation.

[wycats]

👍 after I realized that template URLs reduce the verbosity (as intended).

[steveklabnik]

Woot! @lgebhardt @dgeb , would you like to submit a patch that fixes this, or should I work on it? I'd like you to get the credit, but I know you may be busy.

[lgebhardt]

@steveklabnik Thanks. I assume you must be busy too. I'm currently taking a whack at a patch. I hope to have a PR soon.

[steveklabnik]

Great! I'll leave you to it then.

[MajorBreakfast]

One style to rule them all :)
Just one question: For what is the non-inline templating good for?
Wouldn't this be shorter:

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": {
        "href": "http://example.com/people/{posts.author}",
        "type": "people",
        "id": "9"
      },
      "comments": {
        "href": "http://example.com/comments/{posts.comments}",
        "ids": [ "5", "12", "17", "20" ],
        "type": "comments"
      }
    }
  }]
}

What exactly do these non-inline templates enable us to do?
Reasoning in my sample above is like this: Does href contain a { -> use href as a template
(The curly brace isn't allowed in urls. So it's not a limitation)

Just some thoughts. Tell me what you think.

[lgebhardt]

@MajorBreakfast In the case you present there may or may not be a space savings (I haven't counted the characters). However if you are returning a long array of Posts you will see some space savings since the links object can be shortened down to a single id, or an array of ids. You can eliminate the duplicate keys href, type and id from each document by replacing them with just the id values.

{
  "links": {
    "posts.author": "http://example.com/people/{posts.author}",
    "type": "person"    
  },
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "12"
    }
  }, {
    "id": "2",
    "title": "The Parley Letter",
    "links": {
      "author": "12"
    }
  }, {
    "id": "3",
    "title": "Dependency Injection is Not a Virtue",
    "links": {
      "author": "12"
    }
  }]
}

instead of

{
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": {
        "id": "12",
      "href": "http://example.com/people/12",
        "type": "person"    
      }
    }
  }, {
    "id": "2",
    "title": "The Parley Letter",
    "links": {
      "author": {
        "id": "12",
      "href": "http://example.com/people/12",
        "type": "person"    
      }
    }
  }, {
    "id": "3",
    "title": "Dependency Injection is Not a Virtue",
    "links": {
      "author": {
        "id": "12",
      "href": "http://example.com/people/12",
        "type": "person"    
      }
    }
  }]
}

In addition to some space savings, I think it also makes the document easier to read.

Hope that helps

[MajorBreakfast]

That makes sense. That's a lot of repetition
Okay then what about the other way around:
Does it make sense to have the object notation "links": { "author": {...}}" at all?
Just: Number/String means toOne ("author":1), Array means toMany ("comments":[1,2])
It would really be nice to not have 100 special cases :) Less is more

[lgebhardt]

If we were to eliminate the links object and just promote author to a document attribute it would be programmatically harder to determine which attributes are links. Then there's the inconsistency you would have if you wanted to specify additional links metadata.

[MajorBreakfast]

To be clear, I mean:

{
  "links": {
    "posts.author": {
      "href": "http://example.com/people/{posts.author}",
      "type": "people"
    },
    "posts.comments": {
      "href": "http://example.com/comments/{posts.comments}",
      "type": "comments"
    }
  },
  "posts": [{
    "id": "1",
    "title": "Rails is Omakase",
    "links": {
      "author": "9", <- Must be a Number, String or Array (not an Object)
      "comments": [ "5", "12", "17", "20" ] <- Must be a Number, String or Array (not an Object)
    }
  }]
}

Number,String -> toOne
Array -> toMany

[lgebhardt]

@MajorBreakfast I'm addressing this in my PR. Here's what I have written about that so far:

Use of a document level links object is generally discouraged because it is
wordier than using a URL Template. However there may be situations where each
document will have a URL that isn't supported by the rigid structure of a
template. If compound documents are being returned it will also be necessary
to include either id or ids for the related documents.

[MajorBreakfast]

You could add "... wordier than using URL Templates if you return many documents." because that helped me understand it. :) However I now see why we need both.

[mikelinington]

I'd like to raise a tangenital point while #185 is gaining traction. Once this goes live, we'll have de facto support for to-one polymorphism, but not to-many.

With regard to several other issues about polymorphic types, this solution seems to solve the problem for to-one relationships – specifically, I could have a different links.author.type for every document. I realize this is not the intent of the change, but I know for sure that I'll be exploiting it that way, and it seems perfectly valid wrt the spec to do so. It essentially makes the rule "one type per link relation per document," with the ability to use link templates for brevity if they'll be the same.

For to-many relationships, though, we're stuck with: one type per collection per link relation per document. It seems arbitrarily restrictive given the freedom we're now being given in to-one relations.

One way we might do this would be, instead of an ids key, allow the link relation to be an array of link objects. To build off of one of your examples:

{
  "posts": [
    {
      "id": "1",
      "title": "One Type Purr Author",
      "links": {
        "authors": [
          {
            "href": "http://example.com/people/9",
            "type": "people"
          },
          {
            "href": "http://example.com/cats/1",
            "type": "cats"
          }
        ],
        "comments": {
          "href": "http://example.com/comments/5,12,17,20",
          "type": "comments"
        }
      }
    }
  ]
}

The most obnoxious part about this is that you have to check whether it's an array or an object, but we already have this annoyance with the current (pre-#183) link relations, which expresses to-one or to-many by checking if it's an array or a single id.

I'd be open for a better way to do it, too, but wanted to submit a possible solution instead of just quibbling.

[lgebhardt]

@mikelinington It's simple and consistent with the other changes we're proposing. I like it.

[MajorBreakfast]

Is there a way to make it more brief using templates?

Query explicitly using the ids:

{
  "links": {
     "posts.authors[type=people]": { "href": "http://example.com/people/{posts.authors}" },
     "posts.authors[type=cats]": { "href": "http://example.com/cats?authorOfPost={posts}" }
   },
  "posts": [{
    "id": "1",
    "title": "One Type Purr Author",
    "links": {
      "authors": [
        { "id": "1", "type": "people" },
        { "id": "1", "type": "cats" }
      ]
    }
  }]
}

and this would also work because we know the types and thus where to look in the returned compound response:

{
  "links": {
    "posts.authors": { "href": "http://example.com/posts/{posts}/authors" }
  }
}

[dgeb]

Closing now that 8d3adde has been merged and @mikelinington's suggestion has been captured in #187.

@MajorBreakfast would you mind repeating your comment about templates in #187? I don't want to lose your suggestion and that issue is more focused than the discussion here.