Obviously the title of this post doesn’t make sense to be all in same article, yet I’ve still managed to carry an underlying theme interlinking the above topics : -). Effectively I’m hoping to encourage the pursuit of end user value over and above all else.
Although I’m going to focus on Custom Media Types in this instance (since it reflects my most recent public discussion), the essence of my argument can be applied to many other areas, protocols, libraries, patterns, etc being advocated in software development today.
The main problem I have with Custom Media Types, is the same I have with any other technologies asking me to devote more effort to produce to what I ultimately conceive as a less valueable output. Where the only time when it proves to be beneficial over raw canonical data forms is when the consuming client has built-in support for the custom media format saving the integration point since the client already understands how to process the response.
Although this post may contain a critique on certain facets of REST, I’m by no means advocating against it as overall it’s a far superior more elegant approach to designing services than the SOAP and WS-* stack (which I’m happy to see now starting to slowly fade out of existence). In this light I instead consider myself more a devils advocate providing critique on parts which provide minimal value yet require significant more effort to implement. So consider this a public service announcement as due to the taboo nature of REST you’re unlikely to find many posts like this critiquing it which is partly my motivation for writing it.
There are times when custom mediatypes are useful however what’s not being discussed in these cases (and what I continue to have issue with) is the amount of airtime given to what I consider a small % of applicability and real-world use. This has a tendency to result in a significant body of knowledge flowing around new developers information streams and if they didn’t know any better would think that this is the right way to go about developing web services. What we’re going to end up with are solutions looking for problems and a swath of invented custom formats – more then there deserves to be. Given the unhealthy amount of coverage given to REST frameworks used to flex different parts of REST theory, I fully expect solutions developed just because developers want to obtain full-compliance REST reputation badges implementing non-trivial CONNEG + Custom Media Types multi-hop solutions when far simpler, more direct solutions can elegantly solve the same problem space. Of course I fully expect people to argue that a superior solution can be achieved even with full-compliance of REST HATEOS restrictions, and I simply want to re-iterate for developers to consider, to understand and weigh all options, and that Getting Things Done in the minimum LOC and effort possible is also a worthy goal to strive for, as when we get older, time becomes more precious and being able to hone this skill will ultimately pay dividends later : -)
In my 10+ years as a backend systems developer, I can count on one hand the number of times when creating custom media types has proven beneficial or was a customer requirement. The majority of my time is spent developing systems with a primary purpose of servicing clients, thick, thin or otherwise. In my web services world view, not all media types are created equal and its the popular data formats XML, JSON, SOAP, CSV, etc that are ultimately exposed by web services and consumed by its clients. It’s these formats I believe Web Service Frameworks should be optimized for as they provide the most utility, re-usability and value.
As expected my real-world experience is also the primary influence on http://www.servicestack.net where the above data formats are built-in and are configured to work with all your services automatically. Your C# logic remains DRY and is fused with the expected HTTP result providing a free logic block to be able to return any C# POCO or throw any C# Exception. At the same time making all services available on all above data formats. These are the traits I’ve found to be the most productive and ultimately provide a friction-free development experience.
The other notable mediatypes which are actually more prominent than web services data formats are the media types understood by web browsers. By this I mean HTML/CSS/JavaScript /img/vids/etc. The development of web sites have long been the focus for most development platforms where websites have now become the most important deployment target for most new applications. These web formats are first class formats where entire web frameworks are built around and optimized for, which you should be using instead of Web Service Frameworks.
Despite my criticisms, I still think the production of custom formats is important when it’s required, and ServiceStack actually has very good support for it with one of the DRY-est API’s you’re likely to see in accomplishing this task. I just don’t personally give prominence to this functionality since it’s not where I believe web service developers should be speding their time in implementing solutions with. Although for the interested this is how ServiceStack implements the VCard Format. Where its non-invasive layered API makes it trivial to support any custom media format that is well supported since its the same integrated pipeline the newer built-in HTML and CSV formats are using.
Note: at the lowest HTTP API level in every web framework (e.g. IHttpHandler, HttpServletRequest, etc) this has always been possible, so all web service frameworks are showcasing its high-level API to achieve this end. Some frameworks make this functionality available using APIs mapping directly to REST concepts, ServiceStack however prefers to make it as DRY, typed and expressive as possible.
A key objective in SOA systems and web services today is to servicify and free your data making it as universally available and re-usable as possible. Given these objectives it is common to see SOA systems today make use of HTTP for their transport protocol and utilizing either XML or JSON for their data formats. This is largely because the data formats contain the essence of your data where should they need to at a later date, your clients can use this canonical form of your data and create the custom media types themselves. The same is not true the other way around where if the only format your service provided were custom media formats (e.g. the VCard format) it not only takes much more effort to extract the raw data, but the specific media type will only contain the subset of your contact data as supported by the format. So if your developers only have an option to deliver 1 format for your resource, you’re best to make it a raw data format since the development of other formats can still be achieved by your clients long after the developers have moved on.
In summary you should only be accessing the specific VCard format (or other specific mediatype) only if your client already has built-in support for it. Since it requires more effort on the client and server to implement and access otherwise.
It is therefore important to remain vigilant and keep the 80/20 Rule in mind (or in the case of custom media formats the 95/5 rule : -) and treat custom media types as complimentary formats that you should only add in addition to your data formats. When development of your SOA services have finished you want to ensure it’s data is as accessible and re-usable as possible in the formats that provide the most utility to your current and future consumers of your services, i.e. the data formats.
With the current demise of SOAP, REST is making it’s way back into the forefront of IT minds. The value of REST has always been available in HTTP/1.1 since its inception but between the marketing and committees pushing their own agendas, the Enterprise development world was largely oblivious to its possibilities as they were taught if your data didn’t come in a XML decorated envelope it’s not a real web service.
Its come-back can largely be attributed to developers and companies ignoring the blind advice of big iron Company’s and instead discovered that they are better able to provide value by embracing the inherent properties in HTTP rather than the friction-encumbered dev experience of building SOAP web services.
Displacing big iron-backed monolithic standards is hard, it takes courageous companies willing to discard current enterprise teachings who are able to re-think the essence of what they’re trying to achieve.
It’s therefore important we don’t repeat the same mistakes and don’t lose sight of what the value proposition on the technologies we ultimately choose to adopt, can provide. I’m constantly seeing blind embracing of software process, approach or technology where at the most extreme levels its treated akin to a religion, instead of what it should be: productive tools serving a means to an end.
REST at its extreme conformance imposes some hefty HATEOS restrictions which coupled with most Hyper media driven applications fixation on XML, does not leave you with a very productive window to develop in. If you see real-world value in conforming to these restrictions then by all means adopt them, I personally don’t, and since collecting stamps of approvals from the REST community is not my #1 priority, I routinely discard they exist.
XML is another one of those funny technologies handed down from standard committees as the holy grail data format. Unfortunately this only holds true when you don’t have to develop with it, as it’s a poor programattic fit, making it harder to work with than JSON, that’s both more verbose and slower as well. A good reference that explores these differences in more detail is Douglas Crockford’s entertaining talk on Heresy & Heretical Open Source.
XML as a interoperable structured data format has its place, at the same time I’m finding cases where it would be the ideal data format harder to find. I no longer store it in databases, make use of it for efficient messaging or let it go near my mobile or web apps.
The way I defend myself from adopting the wrong technology is to take a cynical view against all promises, hype and marketing materials. I usually form a picture of the easiest way to develop something and use that as a benchmark to compare other solutions against. The onus then becomes on other frameworks to prove their value by either reducing the amount of effort or code required or in someway enhancing re-usability, performance or promote better maintainability by reducing coupling. If it does not provide any realizable real-world value I don’t spend too much time researching it and continue what I was doing before this Hot Tech Tip landed on my lap.
Basically don’t take any advice at face value, understand the benefits and amount of effort to achieve the desired outcome of each approach and choose the one that offers the maximum end user utility given the least effort to implement. If you don’t understand the full value that your architecture/approach/framework is providing, you won’t be able to maximize its benefits and will soon find you’ve inadvertently subscribed to a solution looking for a problem.
So if you’re spending a lot of time fighting with your framework just to try to get it to prescribe to some artificial standard or religion it might be time to explore other approaches/technologies. When evaluating technologies think about the concept of Last principles where you first look at the ultimate end-user result you want to deliver, then look backwards at the tools, technoligies and frameworks that will best meet this end with the least effort and maximum efficiency.
I personally believe that the Java/.NET worlds are plagued with leaky, heavy abstractions that introduce more friction an inertia that they’re trying to solve. The pain in configuring these monstrosities can be attributed to:
“So much complexity in software comes from trying to make one thing do two things.” – Ryan Singer
Whilst the un-intended, unknown or magic behaviour I frequently see at runtime whilst developing in said frameworks I’m attributing to its poor, leaky abstractions.
In recent years I’ve been more active in proactively shunning heavy frameworks and look to libraries allowing me to write less code and configuration. Configuration is actually a form of inferior code which happens to be more awkward to define object graphs with than your general purpose programming language. What’s left is a fragile, un-typed mess that you can usually only determine correctness at runtime.
I have many reasons to prefer shorter code-bases which follow my general mantra of “You never have to maintain any line of code you didn’t write”. However here a couple of my favourite quotes I’ve collected on the subject that put this more eloquently than I:
“I happen to hold a hard-won minority opinion about code bases. In particular I believe, quite staunchly I might add, that the worst thing that can happen to a code base is size.” – Steve Yegge
“A programmer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.” – Antoine St Exupery
With that said, my general preference is for light-weight development libraries and frameworks with thin abstractions allowing me to capture my logic’s intent as DRY as possible. Usually this means these libraries have a low artificial construct / concept count whilst remaining expressive at its core.
Despite its poor runtime performance and no sight of any corporate backing, Ruby has been making deep inroads into developers mindshare simply because it allows developers to be more productive. Although the language itself is quite expressive, I’m attributing it’s large productivity gains on the communities focus on Simplicity, Convention, DRY-ness and it’s ultimate goal of its ability to compress logic into pseudo code that runs. In this goal, it seems either by accident or in a way that was cleverly conceived by the languages author it appears that much of Rubys libraries follow a deep symmetry to Donald Knuth’s Literate programming which enables programmers to develop programs in the order demanded by the logic and flow of their thoughts.
“Meaning belongs to a text or to a program to the extent that it acts upon intelligence in a particular way.” We say that a textual work is “literary” if it contains a high density of meaning. The above quotes are from @jashkenas enlightening talk at Ruby’s SunnyConf 2010 exploring the subject.
Literate programming is an intriguing topic that not only looks to enhance developer productivity but at the same time increases developer enjoyment and satisfaction. I can definitely see its appeal as it seems like a breadth of fresh air especially if you spend your time fighting heavy frameworks in your day job. Having said that Ruby’s lack of good performance characteristics which I consider important keeps me from using the language in any capacity.
Luckily CoffeeScript is a hot new language that looks to embrace this vision and since its built on JavaScript, can take advantage of the arms race going on amongst browser vendors in recent years trying to make JavaScript as fast as possible.
Unlike other JavaScript frameworks tackling this domain, it’s not interpreted at runtime and simply compiles 1:1 with efficient JavaScript (in some cases more efficient than a developer would write).
This gives it some unique qualities where it is able to run in a browser without any plugins and thanks to node.js the same CoffeeScript code can run very efficiently on servers as well.
We’re now starting to see this powerfull and expressive combination powering new servers like Ruby’s new Pow Rack server for OSX. Its recent inclusion in the default Rails install is causing the biggest rift I recall in recent times for a single line of config change.
So while everyone may not be in love with it yet, they at least have an opinion on it :)
What’s yours?