Smash Company Splash Image

June 11th, 2016

In Technology


AWS is inappropriate for small startups because its complexity demands a specialist

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at:

Sean Hull references a conversation that he and I had in Slack. I would like to expand on the argument that I made then. You might want to read his essay first, where he makes these points about AWS:

1. Over 70 services offered

2. Still complex to build high availability

3. Need a dedicated devops

4. Orchestration involves many moving parts

5. Troubleshooting failed deployments is difficult

At the time of our conversation I had crashed an AWS instance and we were having trouble fixing it. Among the points I made to Sean was that the problem was specific to AWS. If we’d been in the Rackspace cloud, I simply would have rolled back to yesterday’s image. A problem that would have taken 5 minutes to fix instead dragged on for 2 weeks. We became embroiled in a mystery no one could quite figure out. I even went to the great popup office that Amazon now maintains in New York (a fantastic resource), and I spoke in person with 2 AWS engineers, and they were unable to come up with a definite answer about what was wrong. To his enormous credit, Sean had the best guess of anyone, and when I followed his advice, the problems stopped.

At the last few startups when I’ve worked I’ve insisted on using the Rackspace cloud. It is fairly simple and straightforward. The interface is simple. A mediocre developer can figure it out in 20 minutes. This simplicity is a powerful feature! We should not underestimate it!

Though I strive to be the most amazing programmer that I can ever be, I am not striving to be the most amazing devops guy in the world. And yet, at very small startups, the lead developer is typically also the lead devops guy. I’ve already written of the drama that went on at Celolot last year, yet despite the many, many problems that we had there, we never had any devops problems. And that is because I was in charge of devops, and our needs were simple, and I managed everything through Rackspace. Rackspace made it easy for me to create an image of my servers each night, and when we needed additional development servers, we just spun up a new one from an existing image. Simple.

I can handle basic Linux devops. But AWS is not Linux devops. AWS has become its own specialty. With over 70 services offered, only a full-time AWS specialist can properly understand how to build something in AWS.

Some of you, reading this, will be tempted to say “You are only complaining because you are ignorant about AWS!” But that is exactly my point! Knowledge of Linux devops does not translate into knowledge of AWS, and for most small teams, they only need basic Linux devops. What AWS offers is overkill!

Last year we built a system with an architecture similar to what Matthias Nehlsen has written about — we had some Clojure apps that talked to each other over Redis. So I set up a simple Redis server. Please note: I did not setup a cluster. We didn’t need that yet. We just needed a simple Redis server. So I spun up a basic Linux server, installed some packages, spun up Redis, secured it, and I was done. It took less than hour, perhaps as little as 30 minutes.

Recently I was working with a team that had to build a data ingester, to pull data from multiple 3rd parties. I was working with an inexperienced team, so I taught them the basics of how such a system would work. I created this humorous illustration, which we reviewed together:

Again, even though I am not a devops guy, I took on the role of devops to set this system up. But this team had already committed itself to AWS, so I figured I would do this the official AWS way. How do I set up Redis in AWS? Immediately I run into this:

and this:

Ugh! It’s a bit more than I need right now!

I’m not saying AWS is bad! I think AWS is wonderful! For a big company with a dedicated devops guy, AWS is awesome. Or if a medium sized company has the budget to call in someone like Sean, then AWS is awesome. But for a small team with no dedicated devops guy, AWS is inappropriate. Because over and over again it forces you to think about complexities that you don’t yet want or need to deal with.

Last week I was talking with my friend Chris Clarke. He runs devops at Parsely. Their servers are all at AWS. Their frontend is written in Angular, and it makes calls to ElasticSearch. Chris told me he was running a 62 node ElasticSearch cluster, which stores over 100 terabytes of data. No doubt, when you are working at that scale, then AWS is very useful. But you need someone with the incredible skills of Chris Clarke, and that is a rare thing. And specialized. Small startups don’t normally have/need/want a devops guy of that skill level, because the devops needs of small startups tend to be minimal.

And of course, there has been a trend to reduce devops in small startups to nothing at all. Heroku started that trend, mostly for the Rails crowd, but then for other technologies. More recently, there have been companies such as, run by the very brilliant Antonio Pellegrino. If you are working with NodeJS, then you should be using LSQ — they give you a fantastic way to test, deploy and monitor your NodeJS apps.

And yet, the serverless movement is not quite 100% there yet. Heroku and LSQ are awesome, but if you need a Cassandra database then you need to set that up yourself. Someone like me, if I was running a small startup, I would offload what I could to such services as these, but for any special thing that my startup needed, I would spin up some servers in the Rackspace Cloud and I would install whatever packages I needed, and I would customize it myself. I am not a devops guy, but I can do all the basics that are typically needed at a small startup.

What I can’t do is deal with the vast complexity of AWS. I don’t have the time. I’m busy building software, and I need to learn the algorithms and technologies needed for my software. Learning AWS is, at this point, its own specialty, and therefore AWS is only useful to companies once the company is big enough to hire a full time devops AWS specialist, or at least big enough to have the budget to bring in someone like Sean on a regular basis.



Post comments on Hacker News


Check out my book:


February 20, 2019 10:41 am

From Just An Observer on Don't waste your life on Twitter

"A couple of my favorite bloggers started doing twitter. Instead of permanent additions to knowledge, there is..."

February 20, 2019 3:24 am

From Brennan on Did sleep paralysis start the Salem Witch Trials?

"If you have occasional sleep paralysis, you can take steps at home to control this disorder. Start by making s..."

February 19, 2019 11:09 am

From Ryan Earp on Why I prefer dynamic-typing over static-typing: the speed of adapting to change

"If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate Americ..."

February 3, 2019 2:32 pm

From ruurd on Argument about attraction and sexuality and trans

"wait wait wut? what's the liberals doing here?..."

January 18, 2019 10:22 am

From Justin McGuire on When will the era of CyberPunk end?

"The reason cyberpunk doesn't die is because it all came true. From Noah Smith on twitter: "The cool thing a..."

December 16, 2018 9:06 am

From lawrence on Yair Lapid: What does it say about us that Israel has become the only democracy in the world in which Jews don’t have freedom of religion?

"Cat Mara, thank you for catching that. I've fixed it now. (The URL was a "v" by mistake. Looks like I was tryi..."

December 12, 2018 7:50 pm

From lawrence on Object Oriented Programming is an expensive disaster which must end

"Jussi Nurminen, thank you for writing. I believe you are correct, in the sense that Python 2.x had all the bas..."

December 12, 2018 5:13 am

From Jussi Nurminen on Object Oriented Programming is an expensive disaster which must end

"Hello! I've lately became a bit more suspicious of OO designs (including my own), so I read your original 2014..."

December 4, 2018 9:22 am

From lawrence on Docker is the dangerous gamble which we will regret

"GK, thank you for writing, but I don't understand what you mean when you write: "However, at that point you..."

December 4, 2018 7:14 am

From GK on Docker is the dangerous gamble which we will regret

"A development VM is a fine choice, provided that it comes with tools that make it just as easy to run commands..."

November 30, 2018 7:04 pm

From lawrence on Docker is the dangerous gamble which we will regret

"GK, thank you for writing. About this part: "That thing is writing portable shell scripts. The moment you n..."

November 30, 2018 1:41 pm

From GK on Docker is the dangerous gamble which we will regret

"The fat binaries article was nice, but full blown fat binaries are not really necessary. Whats needed is that ..."

November 27, 2018 1:13 am

From lawrence on Object Oriented Programming is an expensive disaster which must end

"Andres Moreno, thank you for writing. Among other points to be said, I'll say I'm almost heart broken about Py..."

November 26, 2018 9:11 pm

From Andres Moreno on Object Oriented Programming is an expensive disaster which must end

"I am stunned! Why did it take so long to show that the Emperor has no clothes? I got bit by the Lisp bug early..."


Pingback: WEB OPERATIONS WEEKLY NO.70 | 科技与人文的十字路口

November 20, 2017
2:21 am

By Mark

I think in this case you are confusing your experience with doing things a certain way, and your lack of experience doing things another way, as meaning one is easier (universally) than the other.

I say this as a developer (ie not dev-ops) that has transitioned from the ‘linux admin’ style setup you mentioned, to the AWS focused one.

Using your own example, now that I have knowledge of both approaches, I would find it much easier to use for example AWS Lambda than whatever approach you use to make workers. Once you have a system for piping your queue to your workers, AWS handles the scaling of the workers itself. It’s insanely easy compared to the ‘old way’.

February 26, 2018
1:52 pm

By Richard Remer

If the AWS way already works for you, then it’s useful. But if the AWS way does not, it’s like pulling teeth to get anything done.

For example, our workers are in PHP, using the Laravel framework. So, if I want to take advantage of the AWS way, I need to figure out how to get Laravel bootstrapped in Lambda, which is excessively complicated, error prone, and a nightmare to maintain.

All of our system configurations are stored in Github and are completely versioned, and restorable. Except for the AWS configuration, which lives in the “cloud” and is impossible to reproduce without creating ever more layers of management software on top.

95% of the time, AWS is the wrong tool.

February 26, 2018
4:58 pm

By lawrence

Richard Remer, thanks for writing. I agree, it is often best to simply have a powerful server. I think the cloud is over-sold. There are some really wonderful tools in AWS, but the real benefits are for companies that are big enough to have full-time devops staff, as keeping up with the ever-changing AWS world is something that only specialists can do.

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>