Smash Company Splash Image

May 13th, 2018

In Technology

2 Comments

Docker protects a programming paradigm that we should get rid of

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com

This comment from 2014 still seems relevant in 2018:

I can see the advantage for dev boxes where a developer might want to setup a load of containers on their machine to emulate a staging or production environment. But I don’t really understand why you’d want to base your entire production infrastructure on it.

The most upvoted response reveals all the problems that I’m critical of:

- An application will not mess with the configuration of another app (that’s solving the problem of virtualenv, rvm and apt incompatibilities).

- The application is not tight to a physical machine, a container can connect to another machine instead of the local one just by changing the routing or environment variables if everything is done properly.

It’s the UNIX philosophy applied to applications, everything must be as small as possible, do only one task and do it properly and I think that’s part of docker’s popularity.

I don’t understand this part:

It’s the UNIX philosophy applied to applications, everything must be as small as possible, do only one task and do it properly and I think that’s part of docker’s popularity.

There might be great reasons to use Docker, but this isn’t one of them. Using Docker, building images of our apps, with an internal illusion of user paths and environment variables, living in isolation from one another, baked into an immutable image — all of that carries us very far away from the good things we can all admire about the original Unix spirit. There is no sense of “small pieces loosely joined,” but rather, Docker takes us into the world of “complex configuration to manage your earlier configuration and standardize it so that even more complex orchestration tools will know how to work with it.”

I’ll respond to this in particular:

- An application will not mess with the configuration of another app (that’s solving the problem of virtualenv, rvm and apt incompatibilities).

Right, so that is why I wrote “Why would anyone choose Docker over fat binaries?“. Rather than use Ruby or Python, and rely on the underlying operating system, why not use an uber binary that has no outside dependencies?

Let’s consider the counter-argument first. Ryan Tomayko wrote I Love Unicorn Because It’s Unix in 2009,

Eric Wong’s mostly pure-Ruby HTTP backend, Unicorn, is an inspiration. I’ve studied this file for a couple of days now and it’s undoubtedly one of the best, most densely packed examples of Unix programming in Ruby I’ve come across…

We’re going to get into how Unicorn uses the OS kernel to balance connections between backend processes using a shared socket, fork(2), and accept(2) – the basic Unix prefork model in 100% pure Ruby.

We should be doing more of this. A lot more of this. I’m talking about fork(2), execve(2), pipe(2), socketpair(2), select(2), kill(2), sigaction(2), and so on and so forth. These are our friends. They want so badly just to help us.

Ruby, Python, and Perl all have fairly complete interfaces to common Unix system calls as part of their standard libraries. In most cases, the method names and signatures match the POSIX definitions exactly.

This was the attitude that drove Ruby and Python and Perl during the last 30 years: that the operating system was powerful and a light weight scripting language should exist partly to offer a convenient wrapper over OS calls. Larry Wall said that Perl was appropriate for any app that was too big for Bash but didn’t need to be in C. That is a huge space. And Ruby and Python and Perl all were built with the idea that the developer should rely on the OS as much as possible. This was brilliant in the 90s, but it means that the apps written in these languages tend to be dependent on file paths and environment variables and user paths and OS permissions — the app is heavily dependent on the overall context of the machine, because the app is supposed to be a lightweight wrapper around all the functionality already provided by the OS.

This paradigm used to be brilliant but it becomes pathological when it tries to transition to cloud computing.

There is a devops joke that says that when you’ve got a handful of servers you name them like pets: bob, alice, li, lo. But when you’ve got hundreds of servers, you simply number them, like widgets coming off the assembly line at the widget factory.

The paradigm that Ryan Tomayko praises is well suited to a world where the servers are named like pets. But calling fork() and join() from your Ruby app, when you’ve got a thousand instances of your Ruby app running on a thousand servers, is a very bad idea. At that point you need higher level frameworks for dealing with concurrency.

I loved Ryan Tomayko’s essay at the time and I sent it to all of my friends. I wanted everyone to understand and appreciate it. It influenced how I wrote Ruby. But as I worked on bigger and bigger systems, I realized, that paradigm needs to die. fork() and join() can not control the concurrency of your system when your system is spread across a large number of servers. To handle these bigger systems, many new frameworks have emerged. Ruby programmers can now use Celluloid an actor framework “which lets you build multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects.” Developers who write Scala are in love with Akka, which appears to be very good. In the world of Clojure, Michael Drogalis has lead the way with the Onyx framework. And the Go language has very good primitives for certain kinds of concurrency, and it has the Circuit framework for large scale distributed computing.

What is Docker for? You can take your old Python and Ruby and Perl apps and wrap them up in Docker, and thus those old apps can make the transition to the modern world of cloud computing. In that sense, Docker allows you to take apps developed with a paradigm from the 1990s, and deploy it in 2018.

Many people regard this as one of the greatest things about Docker, but I regard the entire effort as an example of what is wrong with the tech industry. We suffer an unwillingness to confront the reality of the emerging situation, and commit to new paradigms that are well adapted to the new situation. Instead we commit to very complex technologies that allow us to wallow in the past. This is “conservative” in the negative sense: rigid, nostalgic, reactionary.

I know a great many developers are going to dismiss this blog post, but as a thought experiment, you might want to consider two different companies. One spends the next 5 years committing to those languages and eco-systems that have been built for the era. The other spends the next 5 years using Docker so they can keep using script languages from the 1990s. Now it is the year 2023, and a crisis happens at both companies. Which of those two companies do you think will be more ready to adapt to the crisis?

Source



Check out my book:





RECENT COMMENTS

May 21, 2018 2:39 am

From xulin on Docker is the dangerous gamble which we will regret

"Elixir & Erlang..."

May 16, 2018 3:28 pm

From Steffen on Docker is the dangerous gamble which we will regret

"I'll never regret the use of docker cause I never used it (and never will). My advise is the the anti docker ..."

May 15, 2018 1:36 pm

From Clayton Gulick on Docker is the dangerous gamble which we will regret

"Great article. The current Docker craze, imho, is the same old problem where engineers get a hammer and every ..."

May 15, 2018 10:38 am

From lawrence on Docker is the dangerous gamble which we will regret

"Yawar, yes, I have looked at Elixir. And as I mentioned in the article: "They are now examining Go and E..."

May 15, 2018 10:32 am

From lawrence on Docker protects a programming paradigm that we should get rid of

"Agam Brahma, thank you. I have now fixed the typo...."

May 15, 2018 12:11 am

From Agam Brahma on Docker protects a programming paradigm that we should get rid of

"Great article, so true @ "unwillingness to confront the reality of the emerging solution". Minor typo: s/On..."

May 14, 2018 8:41 pm

From Yawar on Docker is the dangerous gamble which we will regret

"Just wondering, have you looked at Elixir? It seems to fit perfectly with the ideals you describe: based on de..."

May 14, 2018 7:00 pm

From lawrence on The Coral Reef Pattern of incremental improvement

"Andrew Whitworth, thank you for writing. About this: "Rails might not have been such a big problem for u..."

May 14, 2018 4:05 pm

From Andrew Whitworth on The Coral Reef Pattern of incremental improvement

"I like the coral reef metaphor, but this and your tangent about using separate languages both suffer from the ..."

May 14, 2018 12:43 pm

From Mickael on Docker is the dangerous gamble which we will regret

"Hi, I agree with some of your points, especially that docker is maybe more a dev tool than a production tool...."

May 14, 2018 6:25 am

From Eduardo Moroni on Docker is the dangerous gamble which we will regret

"Hey dude, Nice work. pretty awesome essay...."

May 14, 2018 6:18 am

From Pietro on Docker is the dangerous gamble which we will regret

"Hi, I wish there were more people like you around. The real mess I see coming, or, I should say, which has ..."

May 14, 2018 5:54 am

From Jongilanga Guma on Docker is the dangerous gamble which we will regret

"Awesome Essay, believing in Technology Hyper or fashion has showed us results that we regret. I think am gonna..."

May 14, 2018 3:27 am

From Nonax on Docker is the dangerous gamble which we will regret

"Hi! really nice article and great analysis! I'm sharing lot of your view concerning docker and its environm..."

May 14, 2018 12:34 am

From Anon on Docker is the dangerous gamble which we will regret

"Docker as a tool breaks down at one of the core linux philosophies. Lots of different tools doing lots of diff..."

2 COMMENTS

May 15, 2018
12:11 am

By Agam Brahma

Great article, so true @ “unwillingness to confront the reality of the emerging solution”.

Minor typo: s/Onxy/Onyx/

May 15, 2018
10:32 am

By lawrence

Agam Brahma, thank you. I have now fixed the typo.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>