How ignorant am I, and how do I formally specify that in my code?

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com

For decades, computer programmers have argued with each other over the issue of strict data-type enforcement, versus dynamic data-type unenforcement. And after 16 years of computer programming, I have come down firmly in the middle: I like optional typing.

In computer programming, the static type advocates wear a perpetual sneer, claiming that many problems in programming would be solved if only we adopted their approach. And yet, many languages are statically typed, and software written with those languages continue to have a large number of bugs (for example, think of any open source Java project over time). So clearly, static typing does not get us to Nirvana.

The most obvious argument against static typing is that it claims a level of confidence that it can not possibly have. Developers using static typing specify all types at the beginning of their project, before they have learned anything about the problem they need to solve. The advocates of static typing will make the counter-argument that it is easy to change types as one learns more about the problem. But I would assert that it is misleading to pretend that you understand the system at the beginning. I want to be honest about how ignorant I am when I begin to work on a problem that I have never encountered before.

All the same, I have struggled to explain what I see as the advantages of dynamic typing. And this article does a good job of getting at part of the truth:

I have the strange feeling that types hampers programmer’s
creativity.

The underlined sentence is a compact summary of the reluctance that programmers often feel in migrating to statically typed languages – that they are losing something, a degree of freedom that the writer identifies as hampering creativity.

Is this true? I will argue, to a degree – yes. A type checker for a functional language is in essence, an inference engine; that is to say, it is the machine embodiment of some formal system of proof. What we know, and have known since Godel’s incompleteness proof [9] [11], is that the human ability to recognise truth transcends our ability to capture it formally. In computing terms our ability to recognise something as correct predates and can transcend our attempt to formalise the logic of our program. Type checkers are not smarter than human programmers, they are simply faster and more reliable, and our willingness to be subjugated to them arises from a motivation to ensure our programs work.

That said, not all type checkers are equal. The more rudimentary and limited our formal system, the more we may have to compromise on our natural coding impulses. A powerful type system and inference engine can mitigate the constraints placed on what Racketnoob terms our creativity. At the same time a sophisticated system makes more demands of the programmer in terms of understanding. The invitation of adding types was thus taken up by myself, and the journey to making this program type secure in Shen emphasises the conclusion in this paragraph

The war between static-type advocates and dynamic type advocates tends to be unrelenting — programmers prefer one style or the other, never both. And yet, as I work on a project, I will add in type information to each function, as soon as I feel sure of it; and advocates of static typing, when they see this, laugh with contempt, and claim that I am creating a broken, ad-hoc, pseudo type system. Why does this approach feel like the right way to go for me?

So now I have a new theory about the motivations of the 2 types of programmers. It has to do with 2 attitudes to truth:

1.) one group of programmers are willing to start off by saying “This is the truth”. As they get new information, they will change their definition of what is true, but at any given time they will insist on having a complete statement of what they think is true. These are the people who love static typing.

2.) the other group of programmers start off saying “There is a great deal that I do not know about this particular problem, and I am unwilling to commit myself to absolute statements of truth, while I’m still groping in the dark.” They proceed with small experiments, they see where they fail, they learn about the problem, they discover the solutions, and they add in type information incrementally, as they feel more confident that they understand the problem and possess a workable solution. These are the dynamic type advocates.

I know some people have tried to argue that the difference is one of conceptualists versus empiricists, but I don’t think that is 100% true.

I might start off with a simple database query such as:

(defn persist-this-item [item]

(let [item (if (nil? (:created-at item))

(assoc item :created-at (tyme/current-time-as-datetime))

item)

item (assoc item :updated-at (tyme/current-time-as-datetime))

item (assoc item :_id (ObjectId.))]

(mc/insert “tma” item)))

As I learn more about what patterns are going to be common for this app, I will add a large number of :pre and :post assertions, forcing the code to check types and data content, such as:

(defn persist-this-item [item]

{:pre [

(map? item)

(= (type (:item-name item)) java.lang.String)

(= (type (:item-type item)) java.lang.String)

(if (:created-at item)

(= (type (:created-at item))
org.joda.time.DateTime)
true)

]}

(let [item (if (nil? (:created-at item))

(assoc item :created-at (tyme/current-time-as-datetime))

item)

item (assoc item :updated-at (tyme/current-time-as-datetime))

item (assoc item :_id (ObjectId.))]

(mc/insert “tma” item)))

This is called “design by contract” — the assertion statements help all future programmers understand how this code is suppose to be used.

Are the advocates of static typing correct to claim that I am creating a broken, ad-hoc, pseudo type system? No, that is irrelevant. I am documenting what I have learned about my system so far, I am encoding certain patterns that I am sure are true. If you look at my code, after I have worked on a project for a few months, you will see that some of my functions enforce type assertions, and some don’t. Thus you are seeing my exact level of ignorance, and my exact level of certainty. Any function with type assertions is a function I’m sure of, and those functions without type assertions are the ones that I am still figuring out. But if I took a static-typing approach, then every function would specify all of its types from the moment I wrote my first function. If you look at statically typed code, every function has ever type specified, so you have no idea what the programmer is confident about, and where they still feel ignorant.

I dislike that approach. I want a system that allows me to formally specify my ignorance and my confidence, and dynamic typing, with optional typing, allows me to do that.

I posted this example to Hacker News and got his response:

I like Clojure, but this looks rather painful.
If you’re dead set on adding type declarations, perhaps you’d be better off with Haskell or something?

and also:

Maybe I’m misunderstanding, but it looks like every time these functions get called, they’re type checking their inputs and outputs? That seems like a waste of resources. Why not just write unit tests that ensure that they will always be called with the types you expect (or fail at an earlier step if not)? Identify the places where there might be ambiguity in something’s type (e.g. when a JSON blob arrives, it might map a key to a list instead of a dictionary, or if a function might return multiple types, etc), and use unit testing to ensure that the type you expect will emerge from those places, or else a failure will occur.

and someone followed up with:

Unit testing cannot prove the absence of bugs and shit happens in spite of their presence. That’s why I still value (good) static type systems. But besides a good static type system that can prove certain properties about the code (and I’m not talking about Java’s type system here), you can also do design by contract [1], which is precisely what @krubner is doing and I find that to be pretty cool.

Well, it would be cooler if the compiler could check those contracts at compile-time and issue some helpful warnings, but at runtime they are still valuable because the code will fail sooner rather than later. Plus you can probably disable them completely, should you experience problems with performance in production.

They also serve as documentation for other developers, documentation that you’re forced to keep in sync. This documentation is not about the actual business logic, that ends up being laid out in tests, but rather about interface specifications and invariants.

So there you have it – testing serves a different purpose.
[1] https://en.wikipedia.org/wiki/Design_by_contract

Of course, one can get the compiler to turn off the assert statements, so they don’t run in production. I use them in development to help with debugging, and they also serve as documentation for future programmers.

The person who suggested Haskell was perhaps mislead by seeing only one example. To see the full pattern of my usage, you have to realize that I have type assertions on some function, but not on others. In that way, the assertions make clear what I am confident about, and what I am still ignorant about.

Source