Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON:API v1.1 New Directions #1435

Open
dgeb opened this issue Oct 8, 2019 · 10 comments
Open

JSON:API v1.1 New Directions #1435

dgeb opened this issue Oct 8, 2019 · 10 comments

Comments

@dgeb
Copy link
Member

@dgeb dgeb commented Oct 8, 2019

A couple months ago, @gabesullice and I requested feedback regarding the adoption of v1.1-RC1 of the spec, which was first published in December of 2018. You can read the request and responses in this issue, which also includes my analysis.

According to our own rules, we saw insufficient adoption of RC1 to move forward with finalization of v1.1, and we think we understand why. The main feature introduced in v1.1 (as of RC1) was "profiles", which are based on IETF RFC 6906. By design, profiles are limited in scope and can't be used to extend the spec to include new document elements and processing rules. Many developers seem to have chosen to sit out implementing v1.1 and wait for v1.2 to solve more pressing problems that can only be solved through extensions of the spec. And others have attempted to stretch the limits of profiles beyond their defined scope, since profiles represent the only "official" tool offered to customize the spec. Both scenarios are understandable, but not desirable.

Since then, Gabe and I have had our heads down trying to plot a new course for v1.1, and we're now ready to make a proposal to the community.

The gist of our proposal is that we would like to simplify and descope profiles as they're currently defined in v1.1-RC1 AND simultaneously introduce a complementary means to customize the spec: "extensions". While profiles can only specify meaning for members already reserved for users, extensions can only specify meaning for members reserved for the spec itself. Because any members added by extensions will be namespaced, extensions do not block any future expansion of the spec. Rather, most extensions should be viewed as a way to explore just such a future expansion.

The details of our proposal are discussed below, along with a few other significant changes planned for v1.1.

Simplify Profiles

The first part of our proposal is to scale back the complexity of profiles as they're defined in v1.1-RC1. The following simplifications are either planned or underway:

  • Change profiles to be purely opt-in by the server to align with RFC 6906. A client may request one or more profiles (via the Accept header), but the server is under no obligation to honor that request in the response. The server can choose which profiles to apply, and should reflect those profiles in the returned Content-Type. The concept of a profile query param will be removed, since it also violated the spirit of RFC 6906. This change has been PR'd.

  • Remove the ability for profiles to assign meaning to reserved query params such as filter and page. RFC 6906 states that "A profile MUST NOT change the semantics of the resource representation when processed without profile knowledge, so that clients both with and without knowledge of a profiled resource can safely use the same representation.". Since these params are intended to provide processing instructions, they are inappropriate for profiles.

  • Move profile authoring guidelines out of the base spec. We can make recommendations for best practices in a non-normative way.

  • Remove guarantees about profile registration and publication on jsonapi.org.

Introduce Extensions

Just as profiles can be negotiated via the profile media type parameter, extensions will be negotiable via the ext media type parameter. Unlike profiles, a server will be required to understand and apply every extension requested via Content-Type and Accept headers, or else respond with an error. In this way, clients can be guaranteed that their requests will be processed according to the base spec plus any extensions they've applied.

Extensions are strictly additive to the base spec, and can never impose rules about members already reserved for users. Any members introduced via extensions MUST be namespaced with a : prefix (e.g. local:id). Namespaces can only be used by extensions. Besides the fact that members are namespaced, most extensions could be merged with the base spec.

Extensions will become the means to define custom processing directions for query params such as filter and page.

Sharing Official and Community Extensions and Profiles

Extensions and profiles are both specified by URI, so we (the spec editors) have no means or desire to control which ones are used by the community.

However, we do feel a responsibility to provide a means to share extensions and profiles on the official site (jsonapi.org) just as we share implementations of the spec.

We plan to be cautious in particular about which extensions are seen as "official". We may choose not to support any community extensions at first on jsonapi.org as we wade into this new territory. Essentially, we have to be cautious about making the spec confusing through an endless array of conflicting extensions.

Since profiles are applied purely to "user space", we don't see the need to be so cautious about publishing community profiles.

First Official Extensions

Two of the longest awaited additions to the spec, Local identifiers and Operations, are proposed as the basis for the spec's first "official extensions". Content from the original PRs has been simplified, refactored, and reopened as new PRs that are compatible with the new extension proposal.

The "Atomic Operations" extension (#1437) will use the namespace atomic to emphasize the atomicity guarantee that motivates the need for operations. Operations will be requested via an atomic:operations member, and results returned as atomic:results.

The "Local Identitities" extension (#1436) will use the namespace local. Local identity members (local:id) can be used to identify resources that have not yet been assigned an id by the server. This fills a gap in the base spec in which resources without IDs assigned can't yet reference themselves in relationships. This extension will also be critical to enable resources in different operations to reference each other without IDs.

Note that neither of these extensions will be part of the base spec nor do they need to be versioned along with it. Care will be taken to allow side-by-side evolution. It's possible that one or both of these extensions will land in the base spec eventually (without namespaces of course).

Links Upgrade

The other change that we'd like to support for v1.1 has been PR'd and refined by Gabe for a while: Upgrade links to RFC8288-modelled web links. This PR should help improve hypermedia support by allowing params to be optionally added to links to convey more information about them.

Summary

All aspects of this proposal are either currently in development or already PR'd. We now want to reach out to the community for feedback and support in helping us reboot v1.1 of the spec ASAP. Please let us know your thoughts!

@rwjblue

This comment has been minimized.

Copy link

@rwjblue rwjblue commented Oct 8, 2019

Any members introduced via extensions MUST be namespaced with a : prefix (e.g. local:id).

This seems reasonable to me, but just to clarify:

  • this means the spec itself is prevented from ever using members with a :, correct?
  • this doesn't specifically infer where the members with a : are allowed, is it the intention to allow them anywhere (e.g. within all types of top level object types)? Wouldn't there still need to be some restrictions (to avoid collisions with userland attribute names for example)?
@rwjblue

This comment has been minimized.

Copy link

@rwjblue rwjblue commented Oct 8, 2019

Please let us know your thoughts!

I'm excited!

@dgeb

This comment has been minimized.

Copy link
Member Author

@dgeb dgeb commented Oct 8, 2019

@rwjblue thanks for reviewing :)

this means the spec itself is prevented from ever using members with a :, correct?

That's correct. The spec will reserve : for namespacing extensions.

this doesn't specifically infer where the members with a : are allowed, is it the intention to allow them anywhere (e.g. within all types of top level object types)? Wouldn't there still need to be some restrictions (to avoid collisions with userland attribute names for example)?

That's correct. Extensions should be viewed as potential additions to the spec itself, so they shouldn't take liberties that the spec couldn't. I feel strongly that extensions shouldn't be able to add members to objects that are reserved for users, just as the spec itself could never add a new member to attributes or meta.

@jugaadi

This comment has been minimized.

Copy link

@jugaadi jugaadi commented Oct 8, 2019

A recommendation on how JSON:API can leverage HTTP/2 features like server push would be hugely beneficial for the community. This can be absorbed in the specification in the near future.

cc: @gabesullice

@sandstrom

This comment has been minimized.

Copy link
Contributor

@sandstrom sandstrom commented Oct 9, 2019

Seems like a good way forward regarding extensibility!

Thanks for your work on this @dgeb!

@vasilakisfil

This comment has been minimized.

Copy link

@vasilakisfil vasilakisfil commented Oct 9, 2019

Excellent stuff. I really like the turn around regarding profiles, because I felt there were being misused (and probably Dret as well).

In an ideal world, we should opt for a decoupled IETF RFC that covers the functionality of the extensions, in relation to RFC 6838 (Media Types). I had done some research (introspected.rest) around Media Types in the past and I had seen the same issues: it's important to be able to extend the spec with small well-known (or even non-well known, although it should be avoided) reusable modules (=extensions), in a way to compose the needed functionality for our Hypermedia Interface. This has many advantages, one of them being evolvability. But current Web Infrastructure don't really allow us to do that. There is not even a proper way to negotiate those extensions, starting from the client side (unless you do reactive negotiation). And that's a big problem, that here JSON:API needs to solve by itself coming up with some custom solutions which I don't think it's a good for the community (introspected.rest did the same).

To sum up, my point is that I completely like/need/want/support extensions, and I am looking forward to that, I just think we have to push in general for a common extension mechanism in HTTP through IETF, regardless what's going to happen in JSON:API.

Because in the end, some of those extensions could be designed in a more generic way, outside of JSON:API (although that would be orders of magnitude more challenging).

@remmeier

This comment has been minimized.

Copy link
Contributor

@remmeier remmeier commented Oct 9, 2019

thank you for that large & great effort! as soon as people think the specs are more or less stable I could offer one of the first implementations in the context of crnk.io.

@gabesullice

This comment has been minimized.

Copy link
Contributor

@gabesullice gabesullice commented Oct 9, 2019

@jugaadi, entirely agreed! I'm excited for experimentation/modernization like you describe.

@jugaadi

This comment has been minimized.

Copy link

@jugaadi jugaadi commented Nov 6, 2019

When does a proposal(spec related changes, extensions, etc) enter FCP(Final Comment Period) and how long does it last?

@Panman82

This comment has been minimized.

Copy link

@Panman82 Panman82 commented Jan 6, 2020

So one concern I have with opening up an extension point is that there might be some things better suited in the spec itself. I've seen other ecosystem open up an extension point like this and everything gets pushed out into an extension and it becomes the wild-wild-west. Additionally, it's easier to create an implementation when things are fairly strict. To that, is it expected that implementations build support for extensions/profiles or are they to open up extension points themselves for others to build support for JSON:API extensions/profiles??

Also, when can we expect a new 1.1 draft? 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.