Associate URIs with transitions

[jonathanrobie]

Requested by Dilip Krishnan @dilipkrish.

We should allow an optional URI for a transition name so it can be used to associate semantics with transitions.

[RaySinnema]

How is this URI different from a link relation?

[jonathanrobie]

One link relation can implement multiple transitions, so I don't think they are the same thing. For instance in the RESTBucks RADL:

<link-relation name="edit">
    <specification href="http://tools.ietf.org/html/rfc5023"/>
    <transitions>
        <transition ref="Change"/>
        <transition ref="Cancel"/>
        <transition ref="Take serving"/>
    </transitions>
</link-relation>

The client may want to "Take serving", as defined in some ontology somewhere where the definition is associated with a URI. The implementation may use a particular link relation for this that is not in the ontology. A library may use information from RADL to mediate between what the user wants and the link relations and conventions used to implement.

[RaySinnema]

Ah, my bad. That should have been

<link-relation name="https://schema.org/DeleteAction">
  <specification href="https://schema.org/DeleteAction"/>
  <transitions>
    <transition ref="Cancel"/>
  </transitions>
</link-relation>

<link-relation name="https://schema.org/ReplaceAction">"/>
  <specification href="https://schema.org/ReplaceAction"/>
  <transitions>
    <transition ref="Change"/>
  </transitions>
</link-relation>

<link-relation name="http://schema.org/ReceiveAction">
  <specification href="http://schema.org/ReceiveAction"/>
  <transitions>
    <transition ref="Take serving"/>
  </transitions>
</link-relation>

Why would an implementation "use a particular link relation for this that is not in the ontology"? If you're using semantics at all, you'll want to confer the correct semantics, right? Why confuse things with a link relation that doesn't cover those semantics? What's the added value of such a link relation?

[jonathanrobie]

Let me try to explain what I think @dilipkrish wanted - I hope he will weigh in. He is trying to do something like a "smart Traverson", where the semantics he is really interested in are probably equivalent to the transition names. For instance, he would like to order an item and pay for it, without specifying the steps in between. I think this requires more specific semantics than link relations or schema.org actions.

I initially suggested that he simply provide RADL at runtime and use the transition names. I think that's the right semantic level, but it ties his application to one set of transition names in a RADL description. He wanted something more universal than that.

In practice, I'm not sure that there is anything more universal than that. Registries like IANA or schema.org tend to have very general semantics, and history shows that it is rather difficult to reach consensus on more detailed application semantics. On the other hand, if an group were able to reach such agreement, such a URI would be very helpful for them.

I'll ping @dilipkrish and try to get him to clarify and weigh in.

[dilipkrish]

@ApiSecRay I'm not intimately familiar with RADL as a spec so you will pardon my ignorance... Having said that, briefly going over the spec and some conversations with @jonathanrobie here is what I would like to be able to do.

I'd like to express (infer) the ordering workflow from RADL.

I'm a little confused by the semantics of transitions and link relations in RADL. What I'm missing is the tie in between a transition and a link relationship. Should they be the same thing? Link relations certainly seems like what we want here (since it has the semantics), but it seems like its glued together only at runtime.

What I think really differentiates RADL from something like swagger is to be able to express application semantics, and it seems like we're this close to doing that.

Let me describe why I say that; the client only thinks of a given goal in terms of transitions.

Start -> Arrive -> Read-Menu -> Place Order -> Pay -> Take Receipt

So based on this example as a client developing against this radl spec it doesn't seem easy for me to describe to a developer what I need to do to buy coffee.

From what I gather we have all the ingredients to do that but they are bound together by the server at runtime. Since transitions are not first class citizens (states are), its hard for a developer to infer what needs to happen without much effort. On the same token, I don't think that the server side implementers will know which link relationships to present to the client when its in a certain state, since the RADL spec doesn't describe it concretely.

I'm not sure what the right solution is. Hope that makes sense!

[RaySinnema]

@dilipkrish Maybe it makes sense to have a higher-bandwidth conversation? You can join our RADL HipChat room at https://www.hipchat.com/invite/364248/d568b128497e7e3b143239a74f2058f8.

[RaySinnema]

The state diagram (states and transitions) describes the abstract interaction with the system from the client's point of view. In that sense, they capture the requirements in an abstract, non-technical way. State diagrams can be captured in RADL in the analysis phase, long before we make implementation choices like what link relations to use.

At runtime, the client follows transitions from the start state to some desired end state to achieve its goal. To make that interaction concrete, we couple transitions to link relations that are actually found in the representations. Link relations are thus the concrete implementation of the abstract transitions. By using URIs like http://schema.org/ReceiveAction as link relations, we can document the semantics.

The RESTBucks example you gave can be translated into a series of link relations to follow by looking up which link relation implements each transition:

-> GET billboard URI
  -> http://schema.org/menu
    -> http://schema.org/OrderAction
      -> http://schema.org/PayAction
        -> http://schema.org/Order

Does that make sense?

[jonathanrobie]

I'm a little confused by the semantics of transitions and link relations in RADL. What I'm missing is the tie in between a transition and a link relationship. Should they be the same thing?

I think that's the key question.

At the abstract level, a state has properties and transitions. In a concrete instance, a representation has properties and links in some known format, and the links have link relations.

In APIs I've seen so far, the names of link relations are often different from the names of the corresponding transitions, and the same link relation is often used for many transitions. Properties, on the other hand, generally have the same name. Link relations registered on IANA are a bit of a hodgepodge, at different semantic levels. Ray's recent responses suggest using schema.org link relations instead, and they clearly give more specific semantics - are they specific enough for your needs? RADL does keep track of transitions associated with each link relation, and if these link relations are specific enough for your needs, that may be enough.

I imagine many APIs will have a mixture of link relations, including very general ones like "collection", "item", "next" that are not very specific, and many APIs will use primarily IANA link relations or home grown link relations.

It certainly would make sense to provide RADL at runtime for applications that need to know about transitions if you need that, if you control the server. If the client and server both understand the names of the transitions, they can be used to communicate, and you can use URIs for transition names if you prefer.

I don't think link relations and transitions are the same thing, no more than a customer name and a business name are the same thing. Transitions provide more context. But in current REST APIs, link relations are in the API and transitions are not, so anything you do here is experimental. Would it make more sense to use URIs as transition names in a POC and gather experience with this idea first?

[jonathanrobie]

Another thought: if a URI is the name of a link relation, wouldn't it make sense for a URI to be the name of a transition in a system where you want agreement on the semantics of a transition? Why would you want both a name and a URI?

[dilipkrish]

At the abstract level, a state has properties and transitions. In a concrete instance, a representation has properties and links in some known format, and the links have link relations.

Thats exactly what I suspected. Since link relationships are abstractions by themselves, I'm not really convinced we need 2 levels of abstraction here. I wonder if we can fold them into one abstraction, instead of having 2 levels of abstractions.

@ApiSecRay that totally makes sense, but in my opinion as an engineer writing to that spec, I would like it to be easy to infer how to accomplish a use case (buy coffee and pay for it) without going through 2 levels of indirection, actually 3 in this case (if you include going to schema.org to know what PayAction means).

IMO, Thats the whole purpose of RADL to make it easy to write clients and services against.

[RaySinnema]

@dilipkrish Engineers don't have to read the RADL directly: we generate documentation from it. RADL is just a format to enable tooling; the outputs of the various tools are what makes it useful.

After you've run ./gradlew from the RADL directory, the RESTBucks documentation is in java/core/build/docs/clientDoc/restbucks/index.html. It will show you an overview of the states and transitions. Click on a transition, and it will tell you what link relation and HTTP method to use to effect the transition. Please let us know if we should improve the documentation to make it clearer.

[RaySinnema]

@jonathanrobie One defines semantics so machines can understand. Those machines are clients that process HTTP messages. Therefore, URIs defining semantics are useful on the HTTP message level. That is where we find link relations, not state transitions.

States transitions have a different audience: the developer who builds the client. Developers are humans and can understand the transition names just fine. So I don't think we need URIs for transitions.

[dilipkrish]

@ApiSecRay Totally understand where you're coming from.

However, I just feel that the spec needn't be overly complex or abstract. I would argue that we don't really need the transition -> link relation indirection. It could be modeled as an extensible attribute of the transition and wouldn't render the spec any less.

For e.g. Lets take this example

    <state name="Deciding" property-group="menu">
      <transitions>
        <transition name="Place order" to="Ordered">
          <documentation>
            The customer composes an order from the items on the menu and places the order with the cashier.
          </documentation>
          <input property-group="order"/>
        </transition>
      </transitions>
    </state>

    <link-relation name="http://schema.org/OrderAction">
      <specification href="http://schema.org/OrderAction"/>
      <transitions>
        <transition ref="Place order"/>
      </transitions>
    </link-relation>

could be re-written as like this without loss of fidelity.

    <state name="Deciding" property-group="menu">
      <transitions>
        <transition name="Place order" to="Ordered" 
              profile:rel="OrderAction" xmlns:profile="http://schema.org">
          <documentation>
            The customer composes an order from the items on the menu and places the order with the cashier.
          </documentation>
          <input property-group="order"/>
        </transition>
      </transitions>
    </state>

While it satisfies the requirement of the feature request without any modifications. As a spec consumer who is reading it without any tooling (html output) it could be less abstract and more concrete (and consequently less verbose). This kind of indirection just harks back to the WS-* WSDL days. Just my two cents.

[RaySinnema]

@dilipkrish I understand your point. We did it this way to prevent duplication. A lot of APIs use the same link relation for more than one transition. Maybe selfis a special case, but things like edit, collection, and item are prevalent as well.

Plus we wanted to be able to document the link relation in addition to the transition.

Finally, we wanted the structure of RADL to guide API designers using processes like this, where capturing requirements in a state diagram is a distinct first step.

[jonathanrobie]

@jonathanrobie One defines semantics so machines can understand. Those machines are clients that process HTTP messages. Therefore, URIs defining semantics are useful on the HTTP message level. That is where we find link relations, not state transitions.

States transitions have a different audience: the developer who builds the client. Developers are humans and can understand the transition names just fine. So I don't think we need URIs for transitions.

I think the whole question is whether we want to make it easier for machines to read the state diagram directly, and whether transitions should be identified as something distinct from link relations.

For instance, I may only be interested in something that happens at step 5 in the process, and I want to know how to get to that step. The link relations don't tell me that, I need the state diagram for that.

[jonathanrobie]

@dilipkrish Transition names and link relation names have different semantics. In a collection, I might use the 'collection' and 'item' link relations, but the actions that I perform might be 'add', 'delete', update'. In my designs, link relation names are generally nouns, transition names are generally verbs. You can often perform more than one action on a given thing.

And transition names are often more specific than link relation names, perhaps identifying an action with respect to a specific kind of thing found in a collection.

[jonathanrobie]

In #46 (comment), @dilipkrish points out that we could use schema.org action URIs as transition names. This is true, and it works precisely because the link relation name is an action.

A lot of REST designers have preached that link relation names should not be verbs. A link relation name like https://schema.org/ReplaceAction is a noun, but only because Action is added at the end to change the verb into a noun. We probably could eliminate a level of indirection by stipulating that a distinct link relation be used for every action that a transition represents, but that would also require people to use a different approach to REST design than most people are currently teaching. And I'm not yet convinced that this would encourage better REST design than RADL currently does. If this is what you want in a particular API, you can use the schema.org name as both the transition name and the link relation name, and it will look like duplication in the resulting RADL, but it works fine.

In general, a transition and a link relation represent different things. We can make them represent the same thing, but only by imposing constraints on the REST APIs that RADL represents. The constraints that would be required have the benefit of encouraging more specific semantics, but they seem to have the cost of requiring link relation names to be actions, or taking actions out of the modeling process.

And I suspect that it would also mean that RADL would no longer be able to easily represent APIs that most people consider purist REST unless we removed the concept of transitions from RADL. To me, that would be a loss. I like to be able to capture the overall design in states, transitions, and properties before I worry about the HTTP implementation or media types, and I think the corresponding documentation really helps clients know what they need to do to program with an API.

[dilipkrish]

@jonathanrobie like you said the other day at RestFest, and I paraphrase "People like to think concretely", I would like RADL to affords one to think concretely about APIs.

In the spec itself, there is indirection and a mix of abstract concepts like states and transitions and concrete definitions for resources which maps to these abstract concepts. In my experience It is easy for someone to write a service document specification by hand when things are either completely concrete or completely abstract (like ALPS).

Since the various abstract concepts in a RADL definition need to be defined and later "bound" together, The moment we have more than 7 concepts we're going to need a tool to write a radl document by hand even if there is a published schema for the document. If its going to need a tool, then duplication of link relations etc. is moot.

Also, I don't think link relations are necessarily nouns for e.g.edit is an example; RFC5023 that describes it. Link relations are an abstract pointer to a resource or action that has a semantic meaning associated with it.

Sorry I've been a lil inundated after restfest and haven't had much time but I'd love to collaborate some more. I'm certainly not an expert in this, so I'd also solicit some feedback from folks, like Mike Amundsen, Darrel Miller, Mike Kelly perhaps.

/c @ApiSecRay

[jonathanrobie]

I definitely want RADL to help people think concretely as they design. Someone enters your service, what kind of document do you hand them? A menu perhaps? What can you do with a menu? Order something? If you order something, where is that recorded? On an order, which is handed back to the client so it can maintain its own state? What can you do with an order? Pay for it and get a receipt?

The transitions represent the things you can do with each kind of document, the properties represent the information found on such a document. All of this can be determined before thinking about how it is implemented in HTTP / Media Types / Link Relations. This is more at the level of Domain Driven Design.

I'm going to take the milestone off of this issue. There's a lot to think about here, I'm not sure if anything needs to change, and I'm pretty sure that it will require a lot more discussion before we make a change here.

[dilipkrish]

Agree 💯% Just to re-iterate, what I said in a comment ☝️

... it satisfies the requirement of the feature request without any modifications.

I think my feedback is only in terms of UX of the spec to describe services.

[jonathanrobie]

@dilipkrish OK to close this issue, given your last comment?

[dilipkrish]

👍 thanks for entertaining the discussion!