Smash Company Splash Image

December 12th, 2017

In Technology

7 Comments











If you enjoy this article, see the other most popular articles




















If you enjoy this article, see the other most popular articles




















If you enjoy this article, see the other most popular articles

Why I prefer dynamic-typing over static-typing: the speed of adapting to change

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com, or follow me on Twitter.

I am a fan of dynamic typing. So is corporate America. The widespread use of PHP, Ruby, Python and Javascript suggest that dynamic typing is useful. Static typing tends to be used in areas where legal regulations create needs that outweigh programmer productivity. If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

In How ignorant am I, and how do I formally specify that in my code? I said that I liked to add run-time contracts to a function as I better understand it. When I first write a function, I may not know for sure how I will use it. In my mind, the architecture is still in flux. Adding contracts comes later, when I am confident of the design. Perhaps this approach only appeals to developers who are often creating ad-hoc architectures?

There are many arguments in favor of static-typing. Once you have confidence in your architecture, then it is good to lock it down with contracts that warn about breaking changes. I imagine there are many developers who only do specific kinds of programming (CRUD apps for a database, “business intelligence” reports and dashboard, etc) and in those cases they use one architecture for their whole careers. In that case, perhaps they have a good reason to use static-typing. They are already confident with the architecture, as they have used it a dozen times before.

For the programming that I do, I am often creating new architectures. Therefore I need dynamic typing.

This is a standard defense of static typing:

“A large class of errors are caught, earlier in the development process, closer to the location where they are introduced.”

I would re-state this as:

“A large class of errors are introduced, which otherwise would not exist.”

I have dealt with terrible APIs that were outside of my control. When I worked at Timeout.com, we had to pull in data from Ticketmaster and also Booking.com. These APIs were inconsistent, and they would change over time. They also left out fields, so all fields had to be treated as an optional, which it made it difficult to enforce a meaningful contract.

Consider dealing with JSON in Java. Every element, however deeply nested, needs to be cast, and miscasting leads to errors. Given JSON whose structure changes (because you draw from an API which leaves out fields if they don’t have data for that field) your only option is to cast to Object, and then you have to guess your way forward, figuring out what the Object might be.

Consider the Salesforce clone of Java, Apex, which I have had to work with.

The if() statements here are the same one’s that I would have to write in Ruby or Python or PHP, but meanwhile I’ve had to do a bunch of other, useless work:

    public Object deserializeJson(String sandi_data) {
        Object objResponse = JSON.deserializeUntyped(sandi_data);
        
        if (objResponse instanceof Map<String, Object>) {
            Map<String, Object> mapResponse = (Map<String, Object>)objResponse;                
            List<Object> dataList = (List<Object>)mapResponse.get('data');
            if(dataList == null) {
                String err = 'The Sandi API field for data was null';
                System.debug(err); 
                ApexPages.Message msgErr = new ApexPages.Message(ApexPages.Severity.ERROR, err);
                ApexPages.addmessage(msgErr);
                return null;
            } else if (dataList.isEmpty()) {
                String err = 'The Sandi API field for data was empty';
                System.debug(err); 
                ApexPages.Message msgErr = new ApexPages.Message(ApexPages.Severity.ERROR, err);
                ApexPages.addmessage(msgErr);
                return null;
            } else {
                System.debug('dataList:');
                System.debug(dataList);
                return dataList; 
            }
        }
        return sandi_data;
    }

And then, downstream of this:

            List<Object> dataList = (List<Object>)deserializeJson(sandi_data);

            for(Integer i=0; i < dataList.size(); i++) {                
                Map<String, Object> dataMap = (Map<String, Object>)dataList[i];
                String response = fetchCompany(dataMap);
                SearchResult__c profile = saveProfileResult(response);
                cr.add(profile);
            }

I’m leaving out the code that is downstream of this function, but it is full of more of the same: guessing at fields, guessing at how they should be cast, using if() to guard against null or empty. Tons of unnecessary bloat. Lots of easy errors to make.

Again, some of the if() statements need to be made in Ruby or Python or PHP, but the rest of it is just pure bloat. Verbose, unneeded and unhelpful.

In a dynamic language I could simply work with a deeply nested data structure of maps and lists, and I’d handle the casting at the very end of the process. In a dynamic language, I could treat everything as a string till the very end, and then cast to integers or dates or floats or strings as needed. In a dynamic language, I could write the code faster, with less errors, and with less code.

Static typing does not live up to its promises. If it did live up to its promises, we would all use it, and the fact that we don’t all use it suggests how often it fails.

Since I am so often misunderstood on this subject, I hope you’ll forgive me if I indulge in a certain amount of redundancy:

To be clear, this argument applies when we have no control over the API that we draw from. We are drawing from the API of a different company. I wish they didn’t use JSON. If they have to use JSON, I wish they at least enforced a consistent schema. But they don’t. And that is why static type checking fails: because the real world is chaotic, and when you have to interact with that real world, you are often forced to do so dynamically, because of the mistakes that other companies have made. The real world is dynamic. It is full of mistakes. Dynamic typing recognizes this, much more so than static typing.

The idea that you can know an external API perfectly is a fantasy. The real world is messy. The real world does not always conform to a strict schema.

The notion that An External API Is Reliable is as stupid as the notion The Network Is Reliable:

https://blog.fogcreek.com/eight-fallacies-of-distributed-computing-tech-talk/

Many people feel that it is a serious mistake to wait to cast data. The argument is something like: “Using dynamic typing will just hide the problems under the rug, and they will explode in your face later on. Static typing makes those problems explicit at the beginning.”

What I wrote was:

“In a dynamic language I could simply work with a deeply nested data structure of maps and lists, and I’d handle the casting at the very end of the process”

I’ll simplify this: there are 3 times when we can enforce a schema:

1.) when the API call returns with a string

2.) on every line, scattered through dozens of functions

3.) at the end, when I have the data that I want

I advocated for #3. Here are the reasons I don’t like the first 2 options:

#1 – the external API is bloated, so writing a schema for the whole thing would be difficult to justify in terms of business. We only need a tiny slice of the data. More so, when the API ambushes you with an unexpected change, you have a lot of work to do to adapt to the change.

#2 – having casting discovery information scattered through dozens of functions makes the code brittle and refactoring difficult. More so, when the API ambushes you with an unexpected change, you have a lot of work to do to adapt to the change.

With Ruby or Python or PHP or any dynamic language I have the option of #3: grab the data, cast everything as a string, grab the tiny sliver of data I actually need, and then enforce the schema on that tiny sliver. This is the data that I can cast to integers, floats, dates, etc — whatever is actually needed. More so, when the API ambushes you with an unexpected change, you only need to change the casts you have in the final stage. And this is the strongest argument for dynamic typing: the minimal amount of work needed to adapt to change.

In static-type languages such as Java, I’m forced to go with either #1 or #2, and they are both bad options.

When I first wrote about this, a number of people responded by saying the problem was with JSON. On Hacker News I saw the comment “It is only the usage of an awful JSON library and a not so nice language that makes this a problem.”

But bad JSON is part of the real world. If your static-type language can not handle bad JSON, then it can not handle the real world. That is my point: static-type checking is too academic, too pure, for the real world.

As to “not so nice language”, this is an example of the No True Scotsman fallacy, which goes like this: no True statically typed language would be this bad! Somewhere there is a True statically-typed language of such beauty and purity, it overcomes all of these problems!

The real world is messy and full of mistakes. Dynamic-typing adapts to this much better than the academic purity of static typing.

[ [ UPDATE 2019-02-20 ] ]

See the conversation on Hacker News

.

Source



Check out my book:





RECENT COMMENTS

July 14, 2020 8:53 am

From lawrence on How I recovered from Lyme Disease: I fasted for two weeks, no food, just water

"In my experience, yes. I have made it common to do a 3 day fast a few times a year. Anecdotally, I would say i..."

July 13, 2020 3:07 pm

From Tes on How I recovered from Lyme Disease: I fasted for two weeks, no food, just water

"Thank you so much for sharing your story. I have recently been diagnosed with Lyme disease and want to treat i..."

May 31, 2020 9:03 am

From Gábor Hidvégi on What does PHP offer now?

"Dear Lawrence, I can agree with you partially, but not in the whole at all. I agree with you that even P..."

May 22, 2020 12:43 pm

From Frank Hileman on Object Oriented Programming is an expensive disaster which must end

"Good article. However I hate to see so called "SOLID Principles" conflated with OOD. SOLID is a form of cargo ..."

May 22, 2020 10:29 am

From ByterBit on Object Oriented Programming is an expensive disaster which must end

"The first time I looked closely at OOP I was astonished at the – idiocy, the absence of any usefulness, the ad..."

May 13, 2020 5:03 pm

From Just+An+Observer on His Girl Friday

"Closest match to the intensity of this is the movie "Twentieth Century" with John Barrymore and Carole Lombard..."

October 19, 2019 3:08 am

From Bernd Schatz on Object Oriented Programming is an expensive disaster which must end

"I really enjoyed your article. But i can't understand the example with the interface. The example is reall..."

October 17, 2019 4:50 pm

From Anderson Nascimento Nunes on The conventional wisdom among social media companies is that you can’t put too much of the onus on users to personalize their own feeds

"Can't speak for anyone else, but on my feed reader: 5K bookmarked feeds, 50K regex on the killfile to filter o..."

October 9, 2019 3:08 pm

From Dan Campbell on Object Oriented Programming is an expensive disaster which must end

"Object-Oriented Programming is Bad https://www.youtube.com/watch?v=QM1iUe6IofM..."

October 4, 2019 8:44 pm

From lawrence on My final post regarding the flaws of Docker / Kubernetes and their eco-system

"Gorgi Kosev, I am working to clean up some of my Packer/Terraform code so I can release it on Github, and then..."

October 4, 2019 5:14 pm

From Gorgi Kosev on My final post regarding the flaws of Docker / Kubernetes and their eco-system

"> Packer, sometimes with some Ansible. The combination of Packer and Terraform typically gives me what I ne..."

October 4, 2019 12:40 pm

From lawrence on My final post regarding the flaws of Docker / Kubernetes and their eco-system

"Gorgi Kosev, about this: "I would love if you could point out which VM based system makes it simpler and..."

October 4, 2019 7:31 am

From Gorgi Kosev on My final post regarding the flaws of Docker / Kubernetes and their eco-system

"I won't list anything concrete that you missed, because that will just give you ammunition to build the next a..."

October 4, 2019 1:39 am

From lawrence on My final post regarding the flaws of Docker / Kubernetes and their eco-system

"Gorgi Kosev, also, I don't think you understand what a "straw man argument" is. This is a definition from Wiki..."

October 4, 2019 1:23 am

From lawrence on Why I prefer dynamic-typing over static-typing: the speed of adapting to change

"Alistair Bayley, that is valid criticism. Still, in my essay on Object Oriented Programming, I did suggest the..."

7 COMMENTS

Pingback: New top story on Hacker News: Why I Prefer Dynamic Typing Over Static Typing (2017) – Hckr News

February 19, 2019
11:09 am

By Ryan Earp

If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

This is called Argumentum ad populum.

June 29, 2019
8:52 pm

By Andrés Rodríguez

In static-type languages such as Java, I’m forced to go with either #1 or #2, and they are both bad options.

How is #2 any different from dynamic languages? You have a type that defines syntactically valid JSON, and you have functions that allow you to traverse it and blow up if they don’t behave the way you expect them to just like with a dynamic language accessing a JSON field of something that is a String blows up.


interface ParsedJSON {
ParsedJSON get(String fieldName); // blows up if not an object
ParsedJSON idx(int index); // blows up if out of bounds or not an array
String getString(); // blows up if not a string, or you can even have a JS-like weakly typed conversion for this
int getNumber(); // blows up if not a number
boolean getBool(); // you get the idea
boolean isUndefined();
boolean isNull();
}

This definition should cover all JSON types unless I’m forgetting something, and this is if you want the same super loose access behavior as a dynamic language.

A better solution would be to just use a JSON library that allows you to define a fraction of the fields, so that you only type the things you actually access, so that you don’t have to define an entire schema for every object. This is the default behavior when using TypeScript by the way, something such as:


type MyJSON = {
fieldOne: {
fieldINeed: string
anotherFieldINeed: { id: string }
// actual JSON has 210 more fields
}
// actual JSON has 24 more fields
}

With this, if you actually access the same field more than once the compiler has your back when it comes to typos; and if the JSON ever changes, you just change the type and the compiler lets you know every single place where it broke, instead of having to chase it down. Failure mode is exactly the same one as a dynamic lang, but fixing it is much easier.

Also, while Java is getting better with the latest versions, it is *far* from having a *good* type system and the 90’s OOP culture it carries around affects the ecosystem negatively when it comes to teaching users how to harness the usefulness of types.

If you want something mainstream with a powerful type system, TypeScript is a nice enough option. If you have to work with .NET Core F# is great where things such as type providers means you don’t have to type your database or JSON manually and most types are inferred. And that is just the tip of the iceberg, I would bet you a beer your mind would be blown if you had the chance to try out the ideas described in the book “Type-Driven Development with Idris”, where you first describe the structure of your problem with types, then the actual behavior becomes almost obvious to the programmer and sometimes even *to the machine*, where it can write the code for you.

June 29, 2019
9:28 pm

By lawrence

Andrés Rodríguez, thank you for writing. The obvious response would be that a lot is handled as a string in dynamic languages, until such time as it needs to be something else. But the moment that things blow is not inevitable. Sometimes you are just passing information along, and don’t need the type, you can treat it as a string. It depends on what kind of work you are doing. If you are writing software for a bank, then yes, you need to know if “11” can be cast to an integer. But if you are, say for instance, running NLP scripts on text, then it is entirely reasonable to treat everything as a string. Likewise, if you are just putting together a website, you can get away with treating things as a string, for a long time. I believe that one of the reasons that PHP became so popular, back in 1999 and 2000, is that it made it easy to treat everything as a string, till you wanted to treat it as something else, and then it often allowed some of the more obvious casts to happen automatically.

And again, about the bank example that I just gave, that is the kind of well known problem where I think static typing can be useful.

July 7, 2019
12:54 pm

By lawrence

Ryan Earp, thank you for writing.

You have suggested that this is “argumentum ad popularum,” but I don’t think you understand what argumentum ad popularum is. A medical survey with a million people in it will likely have a smaller margin of an error than a medical survey with a thousand people in it. This has nothing to do with popularity, it has everything to do with making sure that sub-populations that appear to be outliers represent something honestly different, and are not merely statistical noise.

So too with software. Surveying the trends in business is not about discovering what’s popular, it is about figuring out what the real trends are, so we don’t get distracted by statistical noise.

Dynamic languages, such as Ruby, Python, PHP, and Javascript, have gained market share over the last 30 years, in businesses both large and small. This suggests some real trend that is more than just statistical noise. Again, we do not care what is popular or not, but we want a wide survey to be sure we understand the real trends.

From this we must conclude that static typing failed to deliver on some of its promises. Otherwise we would not see the trend that we do.

October 3, 2019
9:13 pm

By Alistair Bayley

If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

This same argument would appear to apply to object-oriented languages i.e. if object-oriented programming lead to greater programmer productivity, then corporate America would only use object-oriented languages. Which they by-and-large do.

So using this argument appears to refute your essay criticising OO programming.

October 4, 2019
1:23 am

By lawrence

Alistair Bayley, that is valid criticism. Still, in my essay on Object Oriented Programming, I did suggest there were some other ideas driving OOP forward during the era when it saw corporate adoption:

And there was an explicitly political idea that drove OOP to its peak in the 1990s: the idea of outsourcing. The idea of outsourcing software development rested on some assumptions about how software development should work, in particular the idea of the “genius” architect, backed by an army of morons who act as secretaries, taking dictation. OOP was the software equivalent of a trend that became common in manufacturing during the 1980s: design should stay in the USA while actual production should be sent to a 3rd World country. Working with UML diagrams, writing code could be reduced to mere grunt work, whereas the design of software could be handled by visionaries, possessed with epic imaginations, who could specify an OO hierarchy which could then be sent to India for a vast team to actually type out. And the teams in India (or Vietnam, or Romania, etc) were never trusted, they were assumed to be idiots, and so, for a moment, there was a strong market demand for a language that treated programmers like idiots, and so the stage was set for the emergence of Java.

I think the ideas around UML were very interesting and I try to stay open minded about the possible emergence of a purely declarative language that would truly allow an architect to specify a design, which others could then fill in. We now know that UML and OOP didn’t fulfill that promise, but it’s an interesting promise, and so perhaps some day something like that will exist. And I’m supportive of efforts to create a language that would allow non-technical people to better specify what they want from the software developers. I once worked at a shop that had a fantastic project manager who was not afraid of the technical side of things, so we tried to use Cucumber/Gherkin and Behavior Driven Development. Cucumber/Gherkin is an interesting experiment, that doesn’t quite get us to the hoped for utopia that UML was supposed to give us, but I am glad to see the tech industry still attempting to find the right way forward. “The Future of UML” is from 2003, and perhaps obsolete, but offers an interesting history:

https://www.artima.com/weblogs/viewpost.jsp?thread=8671

During the 1990s, many languages were competing to be seen as the purest Object Oriented Languages (or the one that made the most pragmatic compromises, as Java did by allowing primitive types). Since somewhere around 2005, there has a retreat from Object Oriented Programming, not so much in what software developers do on a daily basis, but rather on the intellectual side. Most languages are now proudly multi-paradigm, I can not think of any that still market themselves on the basis of their OOP purity.

But there is still a prestige that attaches to the word “Object Oriented”. That is not an interesting technical fact, but it is an interesting sociological fact. For instance, Scala developers will, depending on the conversation, either argue that Scala is Object Oriented, or they will argue that Scala is multi-paradigm. Since it is clearly multi-paradigm, I find it intellectually dishonest when they argue that Scala is Object Oriented. Martin Odersky offered some comments in a thread that discussed my essay on Object Oriented code, and I found his responses to be a bit misleading:

http://www.smashcompany.com/technology/martin-odersky-has-some-comments-about-object-oriented-programming

So, to answer you directly, you say of corporate use of OOP: “Which they by-and-large do.” I agree that most corporations use languages that have an OOP heritage, but most of the languages have been moving in the direction of being multi-paradigm. There last few years have seen an explosion of articles about the Functional paradigm, and how you can write in the Functional style even if you are using Java or Javascript.

Back in 2001, when the momentum towards OOP was at its peak, Jonathan Rees listed the 9 attributes people associate with OOP, and he added:

Because OOP is a moving target, OOP zealots will choose some subset of this menu by whim and then use it to try to convince you that you are a loser… I have heard OO defined to be many different subsets of this list.

http://www.paulgraham.com/reesoo.html

We are left with a confusing situation where the meaning of a phrase such as “Object Oriented Programming” is hotly contested.

I apologize for writing “If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages.” Such a naked statement will never be exactly true. It might have been more accurate to write:

“If static typing lead to greater programmer productivity (via a reduction in bugs) then over time we would expect corporate America to move towards using statically-typed languages more and more often, whereas in fact we see the opposite.”

Even that is not 100% true, but it is difficult to write any assertion that is 100% true, and while writing an endless set of qualifiers does help make statements slightly more accurate, it also leads to very dull writing. I try to strike some balance between qualifying statements enough that they are mostly accurate, but without writing so many qualifiers that my writing becomes dense.

Looking at trends over the long term probably does tell us something about what is successful for software developers, but even that is not always a reliable guide, as anyone who criticized Object Oriented Programming during the 1990s could tell us, and how anyone who criticizes Dockers/containers right now can tell us.

Perhaps the only really reasonable lesson to draw is that when the tech industry commits itself to a paradigm, pursues it for more than 10 years, invests billions, re-writes the industry standards to conform to that paradigm, and still the effort is seen as a failure, such that the tech industry eventually retreats from the effort, then we have learned the limits of that particular paradigm. We can say that much about both Object Oriented Programming and static type checking.

Leave a Reply

Your email address will not be published. Required fields are marked *