-
Notifications
You must be signed in to change notification settings - Fork 1
Designing RESTful Services
Starting in this module we're going to shift the focus from talking about the definition and theory of REST to actually designing a sample RESTful system. This module will focus on designing a RESTful service while the next module will focus on designing a RESTful client. Also while we won't be showing much code in either module, the sample that I'll be describing is publicly available. You can find a link to the project in the references section. This module is broken out into basically two parts. First we'll introduce a framework for thinking about RESTful design and see how great examples of REST are all around us in the real world. We'll then introduce a sample bug tracking application and spend the remainder of the module walking through the process of designing the service.
One of the biggest barriers to approaching RESTful design, for me at least, was that regardless of how well I understood the theory, no matter how many times I read Fielding's dissertation, I still approached the design problem with all of the mental baggage of architecting object oriented systems. Even worse, I tended to look at the problem space through the lens of the tools and the technologies that I was already familiar with. Because I had a background in buildings MBC applications that used a relational data base as the backing store, guess how I tried to think about RESTful design? Now I describe this knowledge as baggage and that's not really a great description. Having a solid background in technology is a really good thing. However, we have to be cognizant of the fact that it also gives us a lens or a metaphor for how we see the problem space. And subsequently how we think about design. The danger then of approaching REST with a strong background in some of the Web technologies I mentioned, is that they bring with them some existing metaphors. Like for example, classes and methods. And these naturally lean toward an RPC style of writing services. If we want to build great RESTful services then, we need a new metaphor. Now fortunately we don't have to look far to find a workable metaphor. In fact, we simply need to look outside of our closed in technology universe or perhaps back in time a bit and examine how we would construct systems. And by system I mean any organizational structure if we didn't have computers at all. How would you construct a system if all you had to work with was paper and something to organize those piles of paper? Like for example a desk tray. This isn't a very far fetched idea, in fact I think that if we went around and looked at many smaller businesses we would find that this approach is still favored for many types of work flows. As a matter of fact we really don't even need to look outside of the software development world to see a very relevant example of REST. Almost every development team these days manages some or all of their work flow using a very non-technical solution. A set of sticky notes and a board divided into columns representing the state of work items. Let's take a deeper look at this example and see how it can help us better think about RESTful design.
When we consider the process of managing a bug tracking work flow, using the example of our team bug board, we don't start breaking things down into classes and methods do we? Of course not. We start by looking at the physical elements that enable the work flow. And in this case there are really just two main elements. The sticky note and the bug board. Let's start by looking at the sticky note. In order for the team to work efficiently everyone needs to understand and agree to the format of the sticky note. As it will serve as one of the key ways that the team communicates internally as well as its status externally. So let's start with a title and a description. This seems pretty straight forward. It will help everyone who looks at our bug board know what our bug is as well as get some level of context. Next let's put the assigned user's initials in the bottom left hand corner of the note. Also because we're an agile team and we want to promote self assignment, bugs won't be assigned to a team member until they are going to be actively worked on. So that gives us a basic idea of what our sticky note should look like. Let's take a look at the other major element of our work flow, the bug board itself. In our simple example we're going to divide the board into three basic columns. Backlog, working and done. When a new bug is created it will automatically go into the backlog. When a team member decides to pick up a bug to work on we'll move the sticky note into the working column and naturally when a bug is fixed we'll move the note into the done column. Pretty straight forward, right? Now the work flow that we've defined likely matches many of the work flows that you're used to seeing. However, from a REST prospective there's one fundamental problem with it. Namely, we're relying on a shared implicit understanding that the team knows how to move bugs across the board. For example, how does the team know that new bugs should be placed into the backlog column and not directly into the working column? To address this potential problem let's add some iconography to make the next steps more explicit. We'll take advantage of the bottom right corner of the sticky note and add a few icons that represent our bug states. We'll also add those same icons next to the words on our bug board. As we complete one state we simply cross it off on our sticky note and then look at the next icon to see what column to place the note into next. This gives us the ability to do something pretty cool. We can now completely change the workflow without having to disseminate implicit knowledge throughout the team. Now this is important because depending on the size of the team, communicating this kind of knowledge reliably could be all but impossible. In this case we've added a new bug state called QA. And it sits between working and done on the bug board. After all, it's good to test your bugs, right? In addition to simply adding a new column on the board, we've also added a new icon to the sticky note. This means that team members can follow the exact same workflow that they had previously, cross off the completed state, look to the next icon and put the note into the column that has the matching icon. This is actually a pretty big deal as it's where a lot of RPC systems fall over from a maintenance prospective. To make this kind of change in an RPC system you would have to version and redeploy all of the clients so that they know to call a different method. Or in fact, even know that that method exists in the first place. In this model our client workflow hasn't changed at all though the overall system workflow has. Let me reiterate that. In a RESTful design I can change my system workflow without changing my client workflow. We'll see this in action later in this module as we design the RESTful bug tracking solution. Now, our solution is looking pretty good, however it has some issues in the physical world. If I want all of my clients to automatically start moving notes from working to QA, I would need to modify all of the notes that have already been created and placed on the board with the new icon. This is where the world of software can actually help a great deal. Since your client always interacts with resources that are generated by a server, then the server can dynamically make these kinds of insertions without the client needing to know or care. Hooray for software.
Okay. So we now have a physical world analog for our bug tracking service. Which answers that initial question, how would you design a workflow if you had no computers? Now let's apply some of the design principles of our physical world design to our software system design. Like any good software project we'll start out with a few high level requirements. However, this is where our process may change a little bit from what you might otherwise do. Because our contract in REST is the uniform interface, we want to start mapping our requirements to that contract as soon as possible. Therefore we will identify the possible state transitions for the system, identify the resources that fall out of that and then design our media type based on the possible state transitions as well as the state elements from our business domain. Because a RESTful design gives the flexibility to dynamically modify things without breaking components and of course because this is an example and not a real application, our requirements are pretty simple. First we want to discover bugs in the various workflow states. We'll start with our three initial states of backlog, working and done. We'll also see how to dynamically change this as we did in our physical world example but more on that later. Next we want to be able to add a new bug. As I described earlier, we want to ensure that new bugs are added in the backlog state. Finally we want the ability to move bugs through the various states. From backlog to working and from working to done. Notice that I used some RPC ish language here. I did this deliberately, not to promote RPC, this is after all, a course on REST but to give you a way to see how an RPC way of thinking about this domain can translate into a RESTful design. Because we have a simple domain, we have a pretty simple state transition model. From a starting state we can move through a series of state transitions all of which put the system into a state that is a collection of bugs. There are a few important points to note here. First, we've captured two things in this state model. We've captured the application states such as entry and bugs collection. As you'll see in a few minutes, this becomes important for helping to design your media type. Second, we've captured the different states that our bug can go through. We'll use this information in the next step where we identify resources. Another point is that a client for our service needs to know about only one URL and that is the entry URL. From this resources representation the user, by way of the client, should be able to discover all of the other capabilities that the service has to offer. Finally, for the sake of diagram simplicity, we're not showing navigation and error transitions. Navigation transitions are those transitions available in every representation that enable a client to get a representation of all bugs in the various bug states, backlog, working and done. And thereby addressing our first requirement of discovering bugs in various states. Similarly transitioning to an error state is possible from every other state.
Based on our previous state model, let's tease out some resources. Because I want to follow the design of my physical bug board, I'm going to create individual collection resources to identify the various states that a bug can go through. This is by no means a requirement for my design to be considered RESTful. I could have arrived at a design where progressing the work flow was accomplished by manipulating a data element on a bug representation and then updating the state of an individual bug resource. However, I chose to create different collection resources because it better mimics our physical world analog and I also think it helps to illustrate the point that resources are maps to entities that change over time and it keeps us from inadvertently falling into the belief that REST equals crud over the network. So we've mapped the workflow onto conceptual resources. The next step is to map those resources onto something a little more concrete. Since we're building this system on top of http let's map our resources and state transitions. We'll start appropriately with the entry point resource. The purpose of this resource is really just to serve as a jumping off point for the other resources with its representations. Think of this resource as being equivalent to the person at the reception desk who says why yes, the bug board is in room 3224. This resource is only responsible for handing out representations and therefore supports only the http get method. The backlog resource or the backlog column on our physical board represents bugs that have not yet been activated. To support our first requirement, of being able to discover all bugs in the backlog, we want to support the http get method. Additionally in order to support the requirement that a bug can be moved to the backlog, we also support posting a bug representation to this collection resource. The working resource represents bugs that are being actively worked on. The behavior of state transitions here is the same as it is for the backlog resource. Additionally the server should validate that a bug transitioning into this state is actually assigned to someone. Finally the done resource represents bugs that are, well, done. Its transitions are designed in the same way as those of backlog and working. Now you may be wondering why I chose post rather than put to move bugs between states. The reasoning has to do with the meaning that is ascribed to each method by http. According to the http specification, the URI in a post request identifies the resource that will handle the enclosed entity. In contrast, the URI in a put request identifies the entity enclosed with the request. The user agent knows what URI is intended and the server must not attempt to apply the request to some other resource. In my service I'm sending a representation to the collection resource and then modifying the state of a bug entity with the relevant status, backlog, working or done. Therefore it's important that I use post.
So at this point we've identified application states and the transitions between them. And we've mapped those onto the uniform interface elements of resources and self describing messages. Now let's spend some time focusing on the design of how we will represent our resources. When you're planning your representation there are quite a few options that you have to choose from. In his book on Hypermedia API design, Mike Amundsen breaks these options down to four questions. The first question is around what format you want to base your representation on. The most common formats used by APIs today are XML and Jason. But this can really be anything. In fact, you can choose to support multiple based formats. The next question is around what type of state transfer your API will need to support. For some designs the API may be read-only and in that case the question of state transfer is not very relevant. In cases where state is transferred to the service however, as is the case with our bug tracking service, you have the choice between pre-defined and ad hoc. Pre-defined-state transfer relies on the client having prior knowledge of your format. Typically via your media type specification so that it can construct a valid representation. Ad hoc on the other hand means that the server will tell the client in the representation how it needs to go about constructing a valid representation. A good example of this style is html forms. The next question is around how the elements of your domain will map onto your base format. As you can see here there are three ways that you can map the two. In a specific domain style you create domains specific elements and attributes. For example, with the XML based format you would create a bug element with a title attribute etc. the general style is bound to a more generic domain. It's not specific to your business but it's not just a data structure either. A good example of a broadly used domain general media type at the moment is Adam. Finally, an agnostic domain style is a media type that is completely unrelated to any specific business domain. A good example of a domain agnostic media type is html. The final question in representation design, and it is relation to the previous question of domain style, is application flow. Application flow is defined as the elements that allow a client to recognize and respond to possible state transitions. To put it another way, this question is around how the representation format let's a client know that a link exists and what it can do with it. As in the case of the state transfer question, it is possible that a format will have no application flow identifiers. More likely however, will be the choice between intrinsic and applied. In the case of intrinsic the media type will have built in elements for representing length and expressing their semantics. This will be more common with specific and general domain styles. In the case of applied, the media type does not have all of the necessary facilities for fully expressing application flow identifiers. However, it gives the designer the ability to apply attributes to its elements in order to express those concepts. A great example of applied application flow can be seen in the REL and class attributes within html. So based on the options that we have, let's look at the choices that we made in designing the bug tracking service. Because it already has quite a few hypermedia controls I chose html as the base format. This choice also makes testing the service a little easier as I'll show you in a moment. The state transfer style will be ad hoc since we're using html forms to provide the state transfer elements to the client. The client will simply fill in the values and submit the form. Like I said earlier, html is a domain agnostic format. It's not tied to any specific business domain. This means that we will need to apply our custom application flow identifiers using an element of html. Fortunately html gives us a bunch of different options with attributes such as class, REL, ID and name. Based on the application states and transitions that we identified earlier, we can put together a list of the different elements that we're going to need to represent our system. Obviously we'll need a list of bugs and structure for identifying the various data elements of a bug, such as title and description. Additionally for each bug we'll need a few possible non item potent link templates for moving the bug into various states. Now, the key benefit of a hypermedia design is that not all of these link controls will exist for every representation. When a bug is in the backlog state only the link control for moving it to working is presented. When a bug is in the working state the controls from moving to backlog and moving to done are available. And this is how a hypermedia driven design enables the client to discover the right thing to do without having to have prior knowledge or knowing the right procedures to call. In addition to moving a bug around we also need a hypermedia control for adding a new bug. As we mentioned earlier this will be equivalent to creating the bug and putting it into the backlog state. However, all this is opaque to the client. Our service provides an add-bug semantic and the client simply follows the link. And finally all of our representations should have a set of navigation links so that a client can browse bugs in the different bug states. These will simply be outbound links. Now as I mentioned a minute ago, html provides quite a few different options for declaring application flow identifiers. Here is a small sample of how I'm using different html attributes and elements to map my domain concepts onto html. I find this approach to be helpful in documenting my representation along with providing a sample of the representation, which I'll show next. The key thing to remember is that you're capturing all the details that a client will need to recognize. Remember in REST we don't have service descriptions. The primary thing that clients will look to for understanding is the documentation around your representation format and its associated media type. A quick word though, about when to use the different html attributes, if you choose html as your base format. There's much more detail about this in Mike's book, but here is a quick summary. ID is used to identify a unique data element or a block within a representation. For example you'll see that I use it to identify the major state blocks in the bug tracking representation format. Name is used to identify a state transition element within the representation. For example an input element. Class is used to identify any non-unique data element or block within the representation. For example, in the bug tracking representation you can see that I use class to identify data elements like title for each bug in a list of bugs. REL is used to identify a non-unique process flow element within the representation. In the bug tracking representation I use REL with anchor tags to identify the purpose for the link. Note that with both REL and class you can have multiple values separated by a space and this can be very handy when you want to enable your clients to slice and dice available transitions rather than always having a one dimensional list. So the table can provide a lot of good details about your design but it can be a little hard to visualize what an actual representation will look like. As such, I always find it helpful to include a sample of the representation as you can see here. There are a couple things I'll point out. First, notice that I've omitted all of the H ref and action attributes. This is in part to conserve space, but more importantly it's because the actual link values do not and should not matter to the client. In an example, RESTful interaction the client decides that it would like to activate the index link based on the REL value. It then simply issues a request to the value of the associated H ref attribute. It doesn't matter what the value actually looks like. It could be a pretty URL courtesy of a framework like MBC or it could be their Quids strung together. In REST the URL is not the important part. Second, notice how I have class values like MOO space action space next. Remember how I mentioned that you can specify multiple relationship values separated by a space. This gives the client the ability to, for example, to get and present the user with all of the move links found in our representation. I'm also using next here as a hint to my clients to show the optimum path through the workflow. This is by no means a requirement, but it's a helpful tool that you may find that you want to use.
I mentioned earlier that choosing html as my base format brought with it some additional benefits from a testability perspective. Specifically I'm referring to the fact that because the html media type has a known user agent, a Web browser, it becomes easy to test your API by simply navigating to the entry point URL using your Web browser and then simply following links and activating hypermedia state transition controls.
Now, remember from our real world example, when we introduce the iconography for identifying the next state in our workflow it meant that we were able to dynamically introduce a new workflow state without breaking our client. The ability to have this same kind of dynamism in software is in my opinion, one of the coolest things about having a hypermedia driven API. Let's modify our system design by bringing in the QA state for our bug workflow. We then need to modify the state transitions so that a bug flows from working to QA and then from QA to done. Additionally a bug can flow from QA back to working in the event that during the QA process it's determined that the bug is not actually fixed. And a bug can transition from done to working in the event that it is reactivated. So what does this mean for our representation format? And what does it man for clients? On the server it means that I create new resources as you can see from the circle in green. However because clients don't actually need to know about URLs this doesn't really have an impact on them. The only thing that a client really needs to know about are a couple of new link relationship values for both the forms class and the anchors REL attributes we've added a new link relationship value called QA. Additionally while a client should understand these, it shouldn't break if it doesn't. We'll talk about that a little bit more in just a minute. Equally important to adding a new hypermedia control to move the state of a bug from working to QA is the hypermedia controls that are removed. With my modified representation a bug in the working state will no longer have a hypermedia control to move it to the done state. This design only let's the client see the operations that are actually valid for them to do. Making the distributed system far less brittle. I mentioned a minute ago that a client doesn't actually have to understand the new QA link relationship type in order to make progress. This is because of that next hint that I'm using in my link relationship values. Remember that I mentioned I'm adding a link relationship value of next to the hypermedia control that represents the ideal path through the workflow. This means that in the example here the client can present to the user the ideal next state transition and then activate the hypermedia control without even knowing the full meaning of the link relationship. Now certainly this won't work for more complicated state transitions but it's a pretty cool illustration of the power of hypermedia. As holds true for most software development projects, getting version one of a service is usually the easy part. But what about versions two, three and so on? How do we add new features and fix our own bugs after our service has become immensely popular and we have thousands of different clients on different platforms using it? In the Web service world this was a relatively simple question because there was really just one way that versioning was done. And this was by versioning the service, usually taking the form of a version identifier somewhere in the URL. Now this strategy makes perfect sense in a world where the unit of versioning is the service or the service description. In fact, there's not really another option. In REST however, the contract is the uniform interface and this means that you have quite a few different options for versioning based on the specific thing that you're versioning. In many cases you can version within your representation which means that you don't need to do anything from a formal versioning prospective. This is possible when adding things to your representations definition. Your service adds whatever it wants and clients should simply ignore what they don't understand. When the semantics of a representation change as the result of a rename or deletion of an element, you may need to version your representation. In this case you can take advantage of content negotiation to ensure that your clients are able to specify and get the representations that they need in order to function. Note that there is not really any prescription regarding how this representation Meta data should be transferred from the client to the server. Embedding this information as part of a custom media type name, using a parameter on https accept header such as version or profile, or using a custom http header, all of these are valid approaches. Finally if for some reason your server can no longer preserve the semantics of the map between a resource and its associated entities, you may need to version the resource itself which is not versioning, so much as it is creating a new resource. But it will manifest in the resources identifier which in http terms is the URL. So this approach may look most similar to what you've used with Web services in the past.
In this module I presented a mental model for thinking about RESTful service design and we spent some time walking through a sample bug tracking service. We looked at how to extrapolate states and transitions based on requirements and resources from states. We then took a look at some of the basics of representation design. For more on that subject I highly recommend Mike Amundsen's book on designing hypermedia APIs. We finished up by looking at some different strategies for evolving RESTful services and saw that we have more options then we did back in the Web service days. My hope is that at this point you're able to think about your own services through the lens of, what if I didn't have computers? Now that we've looked at RESTful service design, we're going to move over in the next module and look at some design principles for RESTful clients.