Modern programming depends on DSLs, and Lisp is the best as DSLs

Anders Hovmöller seems to be stuck with a mentality shaped by the C++ language, and so he doesn’t get the important of DSLs.

In C++ land it’s well known that you must keep to a subset of the language in a code base to keep sane. It’s also known that every time you increase the size of the subset you are making the situation worse even if it solves a small problem at hand.

In Lisp land on the other hand, all users can increase the size of the language themselves with macros.

Modern productivity in a language depends on composability. What you do is important to me, what I do is important to you. Issac Newton said “If I have come far, it is because I was standing on the shoulders of giants.” Programmers must also stand on the shoulders of giants. I want to use other people’s code, so I can write less. I want to use 3rd party libraries that have amazing APIs, I want to use my own code from 5 years ago, and I want to use the code of my co-workers. I don’t want their work slowing me down.

If this is difficult in C++, then that is simply an argument against C++.

Languages that allow a high level of meta-programming (Ruby, Lisps, maybe Javascript) make it easier to compose work together.

Yet it remains true that each data structure is a mini-language to itself. We can, at best, hope to develop a great API for that data structure. There are 2 ways to do this:

1.) rely on generic features. Clojure does great at this, allowing the whole of its core library to do data manipulation on sequences.

2.) when something more specific is needed, develop a Domain Specific Language. All languages at some point need a special language for dealing with special data structures, the only question is what the right approach for these DSLs?

You also often hear from Lisp proponents that Lisps are great because code and data are described in the same way. The implication being that it’s simpler because it’s one thing to learn instead of two and that when you learn to transform data you learn to transform code, enabling powerful tooling. There are at least two problems with this thinking:

The simplicity argument: Different things should be different. Having fundamentally different things looking very similar is in fact a risk and a problem, not a strength. The human brain easily handles the idiosyncrasies of English, it can handle the small rule sets of sane programming languages. It has huge problems with simplicity though, as evidenced by the perpetual resistance to the theory of evolution.

Enabling tooling: if this hypothesis is correct, where are the tools? It should be that everyone could write them but somehow there is more and better tooling (like refactoring tools, IDEs) for e.g. Python, a language Lisp proponents are horrified by because of things like significant white space and syntax with lots of special rules.

About “Different things should be different” isn’t this exactly what makes C++ so difficult? Isn’t this exactly what destroys the ability of C++ programmers to re-use each others work?

This next bit almost sounds sensible, until you realize he is criticizing the very thing that programming is more and more reliant upon:

Language design is hard, don’t let everyone do it
The features you have in your language and the features you choose not to have are difficult decisions that subtly and not so subtly shape the community, libraries and other code written in the language. Lisps punt on these hard questions in favor of allowing users to write their own languages on top of Lisp. This creates a Wild West where Domain Specific Languages (DSLs) abound. You will find that in Lisp land this is seen as a strength but if you ask most developers which they would prefer of: a) a project written in one language, or b) a project written in fifty languages you will be hard pressed to find anyone preferring fifty. But that is exactly what DSLs everywhere means, although I’m being kind when I say fifty instead of hundreds or thousands.

Obviously one should exercise care when extending the language, but all other approaches to programming also force us to use DSLs in different ways, and so the real questions are:

1.) how easy is it to create DSLs?

2.) how flexible are those DSLs?

3.) how fragile are those DSLs?

Regarding the generic approach, Clojure’s opinion on this is well known:

While datatypes and protocols have well-defined relationships with host constructs, and make for a great way to expose Clojure functionality to Java programs, they are not primarily interop constructs. That is, they make no effort to completely mimic or adapt to all of the OO mechanisms of the host. In particular, they reflect the following opinions:

Concrete derivation is bad
you cannot derive datatypes from concrete classes, only interfaces

You should always program to protocols or interfaces
datatypes cannot expose methods not in their protocols or interfaces

Immutability should be the default
and is the only option for records

Encapsulation of information is folly
fields are public, use protocols/interfaces to avoid dependencies

Tying polymorphism to inheritance is bad — protocols free you from that

If you use datatypes and protocols you will have a clean, interface-based API to offer your Java consumers. If you are dealing with a clean, interface-based Java API, datatypes and protocols can be used to interoperate with and extend it.

Some languages try to use object oriented programming to enforce “objects” as the unit of DSL, with each object being its own mini DSL, with its own unique set of functions and therefore its own unique interface. In contrast, Clojure emphasize a generic approach:

Object Orientation is overrated

Born of simulation, now used for everything, even when inappropriate
Encouraged by Java/C# in all situations, due to their lack of (idiomatic) support for anything else

Mutable stateful objects are the new spaghetti code
Hard to understand, test, reason about
Concurrency disaster

Inheritance is not the only way to do polymorphism

“It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.” – Alan J. Perlis

Clojure models its data structures as immutable objects represented by interfaces, and otherwise does not offer its own class system.

Many functions defined on few primary data structures (seq, map, vector, set).

Write Java in Java, consume and extend Java from Clojure.

It would be interesting to read a thoughtful critique of Clojure’s approach but Anders Hovmöller doesn’t offer one. He seems to have written his critique before he had considered what the appropriate Clojure approach would be.