Smash Company Splash Image

December 12th, 2017

In Technology

5 Comments











If you enjoy this article, see the other most popular articles




















If you enjoy this article, see the other most popular articles




















If you enjoy this article, see the other most popular articles

Why I prefer dynamic-typing over static-typing: the speed of adapting to change

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com, or follow me on Twitter.

I am a fan of dynamic typing. So is corporate America. The widespread use of PHP, Ruby, Python and Javascript suggest that dynamic typing is useful. Static typing tends to be used in areas where legal regulations create needs that outweigh programmer productivity. If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

In How ignorant am I, and how do I formally specify that in my code? I said that I liked to add run-time contracts to a function as I better understand it. When I first write a function, I may not know for sure how I will use it. In my mind, the architecture is still in flux. Adding contracts comes later, when I am confident of the design. Perhaps this approach only appeals to developers who are often creating ad-hoc architectures?

There are many arguments in favor of static-typing. Once you have confidence in your architecture, then it is good to lock it down with contracts that warn about breaking changes. I imagine there are many developers who only do specific kinds of programming (CRUD apps for a database, “business intelligence” reports and dashboard, etc) and in those cases they use one architecture for their whole careers. In that case, perhaps they have a good reason to use static-typing. They are already confident with the architecture, as they have used it a dozen times before.

For the programming that I do, I am often creating new architectures. Therefore I need dynamic typing.

This is a standard defense of static typing:

“A large class of errors are caught, earlier in the development process, closer to the location where they are introduced.”

I would re-state this as:

“A large class of errors are introduced, which otherwise would not exist.”

I have dealt with terrible APIs that were outside of my control. When I worked at Timeout.com, we had to pull in data from Ticketmaster and also Booking.com. These APIs were inconsistent, and they would change over time. They also left out fields, so all fields had to be treated as an optional, which it made it difficult to enforce a meaningful contract.

Consider dealing with JSON in Java. Every element, however deeply nested, needs to be cast, and miscasting leads to errors. Given JSON whose structure changes (because you draw from an API which leaves out fields if they don’t have data for that field) your only option is to cast to Object, and then you have to guess your way forward, figuring out what the Object might be.

Consider the Salesforce clone of Java, Apex, which I have had to work with.

The if() statements here are the same one’s that I would have to write in Ruby or Python or PHP, but meanwhile I’ve had to do a bunch of other, useless work:

    public Object deserializeJson(String sandi_data) {
        Object objResponse = JSON.deserializeUntyped(sandi_data);

        if (objResponse instanceof Map<String, Object>) {
            Map<String, Object> mapResponse = (Map<String, Object>)objResponse;
            List<Object> dataList = (List<Object>)mapResponse.get('data');
            if(dataList == null) {
                String err = 'The Sandi API field for data was null';
                System.debug(err);
                ApexPages.Message msgErr = new ApexPages.Message(ApexPages.Severity.ERROR, err);
                ApexPages.addmessage(msgErr);
                return null;
            } else if (dataList.isEmpty()) {
                String err = 'The Sandi API field for data was empty';
                System.debug(err);
                ApexPages.Message msgErr = new ApexPages.Message(ApexPages.Severity.ERROR, err);
                ApexPages.addmessage(msgErr);
                return null;
            } else {
                System.debug('dataList:');
                System.debug(dataList);
                return dataList;
            }
        }
        return sandi_data;
    }

And then, downstream of this:

            List<Object> dataList = (List<Object>)deserializeJson(sandi_data);

            for(Integer i=0; i < dataList.size(); i++) {
                Map<String, Object> dataMap = (Map<String, Object>)dataList[i];
                String response = fetchCompany(dataMap);
                SearchResult__c profile = saveProfileResult(response);
                cr.add(profile);
            }

I’m leaving out the code that is downstream of this function, but it is full of more of the same: guessing at fields, guessing at how they should be cast, using if() to guard against null or empty. Tons of unnecessary bloat. Lots of easy errors to make.

Again, some of the if() statements need to be made in Ruby or Python or PHP, but the rest of it is just pure bloat. Verbose, unneeded and unhelpful.

In a dynamic language I could simply work with a deeply nested data structure of maps and lists, and I’d handle the casting at the very end of the process. In a dynamic language, I could treat everything as a string till the very end, and then cast to integers or dates or floats or strings as needed. In a dynamic language, I could write the code faster, with less errors, and with less code.

Static typing does not live up to its promises. If it did live up to its promises, we would all use it, and the fact that we don’t all use it suggests how often it fails.

Since I am so often misunderstood on this subject, I hope you’ll forgive me if I indulge in a certain amount of redundancy:

To be clear, this argument applies when we have no control over the API that we draw from. We are drawing from the API of a different company. I wish they didn’t use JSON. If they have to use JSON, I wish they at least enforced a consistent schema. But they don’t. And that is why static type checking fails: because the real world is chaotic, and when you have to interact with that real world, you are often forced to do so dynamically, because of the mistakes that other companies have made. The real world is dynamic. It is full of mistakes. Dynamic typing recognizes this, much more so than static typing.

The idea that you can know an external API perfectly is a fantasy. The real world is messy. The real world does not always conform to a strict schema.

The notion that An External API Is Reliable is as stupid as the notion The Network Is Reliable:

https://blog.fogcreek.com/eight-fallacies-of-distributed-computing-tech-talk/

Many people feel that it is a serious mistake to wait to cast data. The argument is something like: “Using dynamic typing will just hide the problems under the rug, and they will explode in your face later on. Static typing makes those problems explicit at the beginning.”

What I wrote was:

“In a dynamic language I could simply work with a deeply nested data structure of maps and lists, and I’d handle the casting at the very end of the process”

I’ll simplify this: there are 3 times when we can enforce a schema:

1.) when the API call returns with a string

2.) on every line, scattered through dozens of functions

3.) at the end, when I have the data that I want

I advocated for #3. Here are the reasons I don’t like the first 2 options:

#1 – the external API is bloated, so writing a schema for the whole thing would be difficult to justify in terms of business. We only need a tiny slice of the data. More so, when the API ambushes you with an unexpected change, you have a lot of work to do to adapt to the change.

#2 – having casting discovery information scattered through dozens of functions makes the code brittle and refactoring difficult. More so, when the API ambushes you with an unexpected change, you have a lot of work to do to adapt to the change.

With Ruby or Python or PHP or any dynamic language I have the option of #3: grab the data, cast everything as a string, grab the tiny sliver of data I actually need, and then enforce the schema on that tiny sliver. This is the data that I can cast to integers, floats, dates, etc — whatever is actually needed. More so, when the API ambushes you with an unexpected change, you only need to change the casts you have in the final stage. And this is the strongest argument for dynamic typing: the minimal amount of work needed to adapt to change.

In static-type languages such as Java, I’m forced to go with either #1 or #2, and they are both bad options.

When I first wrote about this, a number of people responded by saying the problem was with JSON. On Hacker News I saw the comment “It is only the usage of an awful JSON library and a not so nice language that makes this a problem.”

But bad JSON is part of the real world. If your static-type language can not handle bad JSON, then it can not handle the real world. That is my point: static-type checking is too academic, too pure, for the real world.

As to “not so nice language”, this is an example of the No True Scotsman fallacy, which goes like this: no True statically typed language would be this bad! Somewhere there is a True statically-typed language of such beauty and purity, it overcomes all of these problems!

The real world is messy and full of mistakes. Dynamic-typing adapts to this much better than the academic purity of static typing.

[ [ UPDATE 2019-02-20 ] ]

See the conversation on Hacker News

.

Source



Check out my books:





RECENT COMMENTS

August 21, 2019 8:47 am

From Jorge Castro on Docker is the dangerous gamble which we will regret

"Hi there and I agree completely but I can resume as follow: Docker promises simplicity, i.e. IT IS EASY. ..."

August 20, 2019 2:29 pm

From lawrence on If you want to go dancing in New York City, consider Silvana

"Which is fine. Like I said, there are dance scenes that have strict “no alcohol” rules. That might appeal to y..."

August 20, 2019 1:35 pm

From Just An Observer on If you want to go dancing in New York City, consider Silvana

""Promise of an early bed" - the whiff of danger keeps me away from many venues like the one you describe...."

August 20, 2019 12:22 am

From lawrence on If you want to go dancing in New York City, consider Silvana

"I think any time you go to any club there is the possibility of running into an angry person, maybe a person w..."

August 19, 2019 7:56 pm

From Just An Observer on If you want to go dancing in New York City, consider Silvana

"I'm confused. You and your friends went out, had a fight, and it's still a great place to go to? Maybe..."

August 18, 2019 8:57 pm

From Michael L on Americans increasingly hate each other

"You seem to have little patience for people who choose different tech paths than you. Although it looks like o..."

August 18, 2019 8:53 pm

From Michael L on Many of my Stackoverflow questions have been marked as duplicates even though they were not

"You don't suffer slights well, do you? Others who choose to waste time with dumb tech, do they keep you up at ..."

August 18, 2019 8:34 pm

From Michael L on Docker protects a programming paradigm that we should get rid of

"Me again. I've worked for a company that focuses on containerized applications for some time now. There is abs..."

August 18, 2019 8:00 pm

From Michael L on Docker protects a programming paradigm that we should get rid of

"To build on my last statement, I'm not trying to show that I'm "smarter." I'm probably not, or if I am, who gi..."

August 18, 2019 7:48 pm

From Michael L on Docker protects a programming paradigm that we should get rid of

"You think that containerization is going anywhere? I agree that it isn't strictly necessary, but you mistake y..."

August 18, 2019 7:40 pm

From Michael L on Docker protects a programming paradigm that we should get rid of

"If you weren't criticizing Docker, why did your frame the article as anti the Docker/Kubernetes/Python troika?..."

August 13, 2019 1:34 pm

From William Hatch on Docker is the dangerous gamble which we will regret

"No tool is every perfect, and you're certainly free to use whatever you want. But, I doubt very much the huge ..."

August 5, 2019 5:23 pm

From lawrence on Why does he want to throw his reputation away?

"DangerNorm, as to your last point, I was recently consulting with a startup that focused on the privacy of med..."

August 5, 2019 3:55 pm

From DangerNorm on Why does he want to throw his reputation away?

"This is tangential to the above point, but on the subject of mental health, I'll also take this chance to say ..."

August 5, 2019 3:51 pm

From DangerNorm on Why does he want to throw his reputation away?

"Your description of most people's response is correct, but I consider it to be a problem, rather than the corr..."

5 COMMENTS

Pingback: New top story on Hacker News: Why I Prefer Dynamic Typing Over Static Typing (2017) – Hckr News

February 19, 2019
11:09 am

By Ryan Earp

If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

This is called Argumentum ad populum.

June 29, 2019
8:52 pm

By Andrés Rodríguez

In static-type languages such as Java, I’m forced to go with either #1 or #2, and they are both bad options.

How is #2 any different from dynamic languages? You have a type that defines syntactically valid JSON, and you have functions that allow you to traverse it and blow up if they don’t behave the way you expect them to just like with a dynamic language accessing a JSON field of something that is a String blows up.


interface ParsedJSON {
ParsedJSON get(String fieldName); // blows up if not an object
ParsedJSON idx(int index); // blows up if out of bounds or not an array
String getString(); // blows up if not a string, or you can even have a JS-like weakly typed conversion for this
int getNumber(); // blows up if not a number
boolean getBool(); // you get the idea
boolean isUndefined();
boolean isNull();
}

This definition should cover all JSON types unless I’m forgetting something, and this is if you want the same super loose access behavior as a dynamic language.

A better solution would be to just use a JSON library that allows you to define a fraction of the fields, so that you only type the things you actually access, so that you don’t have to define an entire schema for every object. This is the default behavior when using TypeScript by the way, something such as:


type MyJSON = {
fieldOne: {
fieldINeed: string
anotherFieldINeed: { id: string }
// actual JSON has 210 more fields
}
// actual JSON has 24 more fields
}

With this, if you actually access the same field more than once the compiler has your back when it comes to typos; and if the JSON ever changes, you just change the type and the compiler lets you know every single place where it broke, instead of having to chase it down. Failure mode is exactly the same one as a dynamic lang, but fixing it is much easier.

Also, while Java is getting better with the latest versions, it is *far* from having a *good* type system and the 90′s OOP culture it carries around affects the ecosystem negatively when it comes to teaching users how to harness the usefulness of types.

If you want something mainstream with a powerful type system, TypeScript is a nice enough option. If you have to work with .NET Core F# is great where things such as type providers means you don’t have to type your database or JSON manually and most types are inferred. And that is just the tip of the iceberg, I would bet you a beer your mind would be blown if you had the chance to try out the ideas described in the book “Type-Driven Development with Idris”, where you first describe the structure of your problem with types, then the actual behavior becomes almost obvious to the programmer and sometimes even *to the machine*, where it can write the code for you.

June 29, 2019
9:28 pm

By lawrence

Andrés Rodríguez, thank you for writing. The obvious response would be that a lot is handled as a string in dynamic languages, until such time as it needs to be something else. But the moment that things blow is not inevitable. Sometimes you are just passing information along, and don’t need the type, you can treat it as a string. It depends on what kind of work you are doing. If you are writing software for a bank, then yes, you need to know if “11″ can be cast to an integer. But if you are, say for instance, running NLP scripts on text, then it is entirely reasonable to treat everything as a string. Likewise, if you are just putting together a website, you can get away with treating things as a string, for a long time. I believe that one of the reasons that PHP became so popular, back in 1999 and 2000, is that it made it easy to treat everything as a string, till you wanted to treat it as something else, and then it often allowed some of the more obvious casts to happen automatically.

And again, about the bank example that I just gave, that is the kind of well known problem where I think static typing can be useful.

July 7, 2019
12:54 pm

By lawrence

Ryan Earp, thank you for writing.

You have suggested that this is “argumentum ad popularum,” but I don’t think you understand what argumentum ad popularum is. A medical survey with a million people in it will likely have a smaller margin of an error than a medical survey with a thousand people in it. This has nothing to do with popularity, it has everything to do with making sure that sub-populations that appear to be outliers represent something honestly different, and are not merely statistical noise.

So too with software. Surveying the trends in business is not about discovering what’s popular, it is about figuring out what the real trends are, so we don’t get distracted by statistical noise.

Dynamic languages, such as Ruby, Python, PHP, and Javascript, have gained market share over the last 30 years, in businesses both large and small. This suggests some real trend that is more than just statistical noise. Again, we do not care what is popular or not, but we want a wide survey to be sure we understand the real trends.

From this we must conclude that static typing failed to deliver on some of its promises. Otherwise we would not see the trend that we do.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>