Using Clojure to build a microservices CMS

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com, or follow me on Twitter.

Many of us become cynical about technologies that promise big breakthroughs in productivity, so we become overly careful in our choices, but this is a good question for managers to always be asking:

“Why would a large organisation with a mix of technologies and legacy systems want to muddy the waters with a completely new language?”

If you want to make the conservative choice, and stick with what you already have, you should be able to articulate the reasons as clearly as you would if you were choosing a new technology.

This certainly hints at the productivity power that I have discovered via Clojure:

“Could Clojure be used to build the bespoke CMS? Is it too bleeding edge? Would his team get it?”

When Colin Steele wrote about his experience as CTO at RoomKey, he also emphasized the incredible productivity benefits of using Clojure:

“As of this writing, we’re on track to having 9 million uniques a month, from zero at the start of 2012. We did so with absolutely no fuss, no late nights, no hand wringing, and a laughably small amount of additional opex spend. ”

http://www.colinsteele.org/post/23103789647/against-the-grai…

And in this article Dave Elliman is describing an architecture very similar to what Colin Steele describes, and with similar gains in productivity:

“After brainstorming some ideas we decided to put forward the idea of a Javascript based Single Page App (SPA) with a Clojure back end and a set of small Clojure based micro services sitting on top of MongoDB hosted in Rackspace.”

I haven’t done the Javascript thing yet, but the Clojure/MongoDb/micro-services approach has worked out very well for me. (I’m using “micro-services” informally, as in “lots of small apps that work together”).

This whole thing was interesting:

Their teams’ current skill-sets leant them towards Java, but the important thing to us was the feeling that we could genuinely deliver quickly and meet their deadlines.

A few months in we were about to start a smaller project within the larger program of work. We needed to add content management to a number of potential web site instances. We began by reviewing available CMS tools in the market. After some research it turned out that none of the off-the-shelf CMS tools available would meet their required approach in the way we needed, so we decided to build our own.

We already had a continuous delivery pipeline set up from the main project that we could adapt, we just needed the right technological choices that would deliver business value in an innovative way to meet our deadline. We knew we had to use the JVM, but building what was to be an app used by partnering digital agencies and client staff, needed to be robust and work intuitively. After brainstorming some ideas we decided to put forward the idea of a Javascript based Single Page App (SPA) with a Clojure back end and a set of small Clojure based micro services sitting on top of MongoDB hosted in Rackspace.

Dan wasn’t easily convinced. He took some persuading…

We had to present the pros and cons of polyglot programming using the JVM as the basis of the agreement, rather than just using Java.

Dan still wasn’t convinced…

We discussed the existing Clojure community, the maturity of the language itself and the momentum we saw in the industry. Companies are seeing speed to market deliveries, that are based on Clojure.

Dan decided to test out the theory.

If he, someone without a development background could get Clojure, then surely his team of Java developers could too.

He had recently built an app to collect data about the various sites the company hosted in different places and had used Java and Spring MVC. He locked himself away for the weekend and rebuilt his app… in a fraction of the code.

The following Monday we got an email from Dan.

“I am more than happy to go ahead with the use of Clojure.”

Win.

Given the go ahead, we started our usual project inception activities and started delivering the application. We decided to split the services so that one service would provide content to its consuming web sites that were responsible for all layout decisions given the content components and some meta data about it, and another service was responsible for maintaining it. The Javascript SPA used the second service directly as well as domain specific services on the main project platform – we were also building in parallel.

The first deadline was 6 weeks away. No-one expected us to hit it. We had started a week late due to internal governance and we were using a new technology the team didn’t know.

We were careful to structure the iterations and get the right things done in the right order. We engaged with user testing in early pre-development to get the designs and user experience conversations going. The users were accustomed to traditional CMS’s, an approach we had eschewed because of the varying outlets for the content – we didn’t want to enter ‘template hell’. We worked with the client to establish acceptance criteria and trained their internal users on how create the necessary content. We released every day to a pre-production environment so the users could test early and often.

The process continued with the team delivering and the users creating content on the application, an application constantly growing in capability.

We hit the deadline a week early.

It was no big deal. We were delivering everyday, multiple times a day. We had delivered everything for phase 1 of the product two months early and way under budget. But how much of this was due to the process? How much to do with it being an SPA, Clojure, or the team?

Post external references

  1. 1
    http://thoughtworks.github.io/p2/issue09/two-months-early/
Source