Living on the Bleeding Edge: A Case Study in Using Clojure, AMQP, Chef, Cucumber and JRuby in the Financial Industry
Summary posted to the Emerging Tech Website
Have you been itching to try emerging technologies like Clojure, AMQP, Chef, Cucumber or JRuby? We’ll cover what happened to us when we proposed these hot (unproven!) new technologies, management said yes, and we had to deliver by sharing our story of building a message exchange in the financial industry. We think hearing our story will help give you the confidence to take these technologies into your own enterprise.
“Time to market is critical in the financial industry”
Collateral Messaging is a brand new market.
Our main competitor actually made it to market before we did (gulp).
They are a startup.
Would probably like to rephrase this: The messaging project within Algorighmics has been given the latitude to exhitibt some of the behaviors of a startup.
- we don’t have the pressure (at the moment) of hand-to-mouth
- were given the latitude to chose the technology stack (yay)
- alternative recruting strategies
- Small “company”. This division before messaging contained around 20 people. Now we have more than 30.
Were sitting on the concept for many years
Competitor Emerged in early 2009.
Sparked interest in the organization.
Rapidly built a Prototype (RabbitMQ and Ruby on Rails) to prove to internal Executives that it could be done within the collatearl division (here in Philly). Facilitated by the existance of and heavy leveraging of Open Source (eg: RabbitMQ, Rails). Even though the division is a (somewhat conservative) .NET / C# shop.
Prototype was well received, but lead to questioning by the CTO of the technology choices. This was a huge change from the typical stack.
Choice of stack stressed as enabler, b/c time to market is very important.
Validation of the technology stack by researching various alternatives.
Success: you’ve been approved / funded.
Er, shit, now I (Aaron) have to deliver.
Where am I going to find the good people?
2008 Jan – – Aaron takes over responsibility for organizing Philly Lambda
2009 Feb – 0 – Initial Prototype
2009 Apr – 0 – Convinced CTO / Demo
2009 May – 0 – Demo
2009 Jun – 1 – Received funding
2009 Jul – 3 – first two developers hired
2009 Aug – 4 – thrid developer hired
2009 Nov – 4 – 8th: Linode Experiences Widespread outages (doh!)
2009 Nov – 4 – 9th: We provision and re-deploy to Slicehost for the demo in 1/2 a day (go automation!)
2009 Nov – 4 – 10th: Live Demonstration to Banking Industry in NYC.
Messaging support for two different collateral management
solutions including a web based portal for less sophisiticated users.
Moves Slowly?
(picture of a turtle / tortoise?)
Catching up.
Mainframes?
(find a picture of a mainframe)
COBOL?
Technologically Conservative?
These #‘s are conservative (since the OTC market isn’t regulated):
250 Billion in 20011.3 Trillion in 2006
?? ??illion in 2009
NOTE: a centralized messaging exchange could provide that kind of insight (hint hint)
Local business people (developers, executives)
Developer looking forward to do cool things
Kyle: to hear how we were able to use these technologies, how we were able to get them approved by the company, what the typical arguments are against these choices and what the facts are that dismiss those arguments. How we leveraged these technologies and how they were better than the more traditional counterparts (eg: Clojure vs Java; Rails vs Spring).
That’s quite a mix of technologies, how will you find developers who can work in all of those?
This is as much a story of trying to find good talent as it is the pick of technology. It’s easy to look at this project from both directions. Did he pick a tech stack he thought would attract good developers? Or did he pick the stack based on it’s technical merit, only later hoping he’d find good developers?
Aaron’s goal was to find the best programmers in the area to work with him on this project. He valued experience and the ability to think outside the box. Paul Graham writes about the same ideas in his essays. Graham beleives that better programmers are continually looking to new programming languages in an attempt to imporove their ability to scale themselves. Looking for better tools and more expressive languages. Aaron went with this idea – he picked technologies that held promise for him: Lisp’s expressiveness and ability to model complex problems more easily than other languages; Ruby on Rails productivity as a web development stack; and AMQP through RabbitMQ, build on Erlang, known for its reliability and performance. To build a team, Aaron worked a rather eliete local users group, Philly Lambda, looking for who he felt were the more well regarded developers in the area. He continues to stay active with the group, dedicating a lot of his personal time to it, so that his network, his connections, to all those programmers stays active and refreshed.
Those programmers want to work in these technologies. All technologies have their warts, the best people can make projects work regardless of their shortcomings. Those are the kinds of people Aaron wanted.
Who are they? How do you know then when you see them?
What do they want?
They want their daily lives to be interesting. They want a challenge. Sometimes the technology stack being new is enough of a challenge even if the problem being solved is not. New problem domains, complex problem domains. Domains with enough depth that they will have to continually learn, study and practice.
They want to work with the other ‘best’ programmers. They want to have a tight night team and they want the team members to compliment each other, to continually improve. A great programmer working on a team without other ‘great’ programmers will experience a decay of morale and will often look for a way to change.
They want to work with the best tools in a good environment.
- Clojure
- JRuby
- Rails
- Cucumber
- Chef
- RabbitMQ
- JavaScript
- jQuery
- YUI
For each major player, consider the following aspects:
- How it was chosen
- why was it a good fit?
- what was the case agasinst it?
- what facts helped get it past those concerns?
- What [unforseen] benefits did we end up discovering?
- What warts did we end up discovering or working around?