## Technology

August 12th, 2019

# How Homakov hacked Github

Apparently I never linked to this? I was impressed with this. A few minor hacks that Github already knew about, and had looked at, and had decided they weren’t important. But Homakov found a way to link them together into a serious attack.

Source

August 8th, 2019

# Please don’t touch the keyboard

Come on. This technique can be improved.

Source

July 18th, 2019

# Ethereum is dangerous and Pact offers a much better blockchain smart contract language

What strikes me is how carefully they’ve thought about the problems with Ethereum, and other blockchain technologies. They deliberately make sure that Pact is NOT Turing complete, so they can avoid some of the hacks that took place with Ethereum. They emphasize a declarative style. And yet they also make it so the contracts can be fairly upgraded.

This is from the article:

However, the added versatility offered by Solidity smart contracts comes at a great cost in safety and complexity, jeopardizing ...

July 4th, 2019

# The right way to do Agile

The best implementation of Agile that I’ve seen was when I was at ShermansTravel.com and Mark Herschberg was CTO. The system went like this:

Herschberg reckoned that each developer could get 5 hours of real work done each day (the rest of the day got eaten up by email, conversations, research, and other incidentals). We had a crew of 5 programmers, so we had 125 hours a week of real programming. The sprints were 2 weeks long, so we had 250 ...

June 17th, 2019

# A real life problem with our Docker use that cost us 3 days of lost work

Sadly, no one saw this because I originally posted it as an update to “Docker protects a programming paradigm that we should get rid of” but just now, on Hacker News, steve_taylor wrote:

docker-compose up -d

My response? This is the sales pitch for Docker:

docker-compose up -d

But this Slack conversation is the reality (we lost 3 days before we were able to fix the issue, a complex interplay of the Docker cache and the way the Dockerfile ...

June 15th, 2019

# How does your frontend know the id of a document if you’re using an async architecture? Rely on UUIDs.

Building an async system is more complicated than building a sequential system, but it can offer competitive advantages. For instance, you can build mobile apps that work offline, and later sync their data with your system, and this ability to function to work offline can enable uses that your competitors don’t have. This is what allowed Handshake CRM to thrive, even though they started after the CRM market had seemingly consolidated down to a handful of winners. Given the dominance ...

June 15th, 2019

# Caches are cheap, build a lot of them for the frontend

There was a stretch, maybe from 2005 to 2015, which will be remembered as the era of simple Ajax uses. The era started when Sam Stephenson released the Prototype library, but it really got going once jQuery was released. It was an era of Javascript fetching data via Ajax, and relying on the backend code to decide what data is sent in response to each request.

That era is evolving into something different. Frontends are now complicated enough that the ...

June 15th, 2019

# Why are software developers confused by Kafka and immutable logs?

Simplicity. Why is it so hard to keep things simple? There are many concepts that we use in software which arise from common sense which we then encumber with needless complexity. Why?

For instance, logs. It’s great to have a record of things. If you’ve built some software that gathers information, it is often very useful to know the order in which information arrived in your system. That is, it is insightful to know the history of events that built ...

May 30th, 2019

# The collision of capital and idealism over a technology

This is a good article, but is it appropriate to say that a technology is simply going through some growing pains when the technology is 25 years old? Doesn’t the long history suggest the problem here is a complex one, with a long history of its own? Python was a dying language until Google decided to make large investments in it. Now it is the main language for Data Science. This is a language that started off with a certain ...

May 30th, 2019

# Billions were wasted on Hadoop startups, and the same will eventually be true of Docker

The hype over Hadoop has died away and many of the investments now seem to have been terrible investments:

MapR Technologies Inc., one of the troika of unicorn startups that emerged from the early days of the big-data movement, may cut up to 122 jobs and shut down its Santa Clara, California headquarters if it can’t secure additional funding.

May 11th, 2018

# Systems of process orchestration

I get that Linux Cgroups might be acceptable, in the sense that “perfect is the enemy of the good” and Cgroups are maybe good enough for now. I’d prefer it if the industry could rethink what a multi-user computer is, get rid of Unix, take the best ideas forward into a new system, and build something where something as clunky as Cgroups and Docker are not needed — it should be possible to have a “fat process” which does what ...

May 11th, 2018

# What are Linux cgroups

This is not the way I would have gone, but whatever. Here we are now:

cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes.

Engineers at Google (primarily Paul Menage and Rohit Seth) started the work on this feature in 2006 under the name “process containers”.[1] In late 2007, the nomenclature changed to “control groups” to avoid confusion caused by multiple ...

May 8th, 2018

# So many things can go wrong with Docker

Sean Hull always has interesting things to say, and so it is worth reading what tripped him up with a recent project where he had to Dockerize everything:

When a service is run, ECS wants to have *all* of the containers running together. Just like when you use docker-compose. If one container fails, ecs-agent may decide to kill the entire service, and restart. So you may see weird things happening in “docker logs” for one container, simply because another failed. What ...

May 7th, 2018

# Is there anyone who can keep up with everything AWS offers?

As I’ve suggested before, unless you a full time specialist tracking AWS, you won’t be able to keep track of all the services coming out of AWS. The one I just learned about is ALB:

Amazon Web Services (AWS) just announced a new Application Load Balancer (ALB) service.

I spent some time playing with the new service to understand what it offers and to see how it fits into our cloud architecture.

In summary, ALB is a massive improvement over ELB in almost ...

April 30th, 2018

# Confluence is very difficult to set up

I had endless errors with Confluence. Each time I started over, I set up a new EC2 instance. I eventually learned that it was possible to restart the setup process by deleting the file confluence.cfg.xml so I did this:

/etc/init.d/confluence stop

rm /var/atlassian/application-data/confluence/confluence.cfg.xml

/etc/init.d/confluence start

but now I’m trying to finish the setup, and all I get is a blank page:

Does anyone know how to fix this?

Source

April 12th, 2018

# Excessive abstraction in PHP frameworks

I’m working with Laravel now, a PHP framework. I see this on StackOverflow:

With Laravel, everything is more complex than it needs to be. Yesterday I was dealing with the issue of logging and errors. How to get Laravel to output errors somewhere that I could see them? Well, that involves overriding some logging classes. The tone of the documentation is always “See, it’s very simple, and flexible, just override the default classes, inherit from the parent, override the method, choose ...

April 9th, 2018

# Do the staff at Jezebel manually manage the trolls in the comments?

Apparently so, this is interesting.

Someone named Jinni writes:

Our Muqaddimah of Code 45* is still banned, alas. A bright point in my own days feels notably dimmed. For those of you who didn’t read (or quite ‘get’) Code 45*; it was a hallucinatory tour into the subconscious of Donald Trump as evidenced by the odd and illogical punctuation of his disordered tweets. Code 45*, like a creative, and slightly mad psychotherapist, attempted to wrest vestiges of meaning from the morass of ...

April 8th, 2018

# PHP code for Apple Push Notification Service

I just lost a week of my life trying to send a message from my server to the APNS servers. A previous team had given us some bad PHP code which I could not get working. I finally tore it out and replaced it with the example I found on StackOverflow:

/* We are using the sandbox version of the APNS for development. For production environments, ...

April 8th, 2018

# A pem tool to generate the pem file for Apple Push Notification Service

Having lost the last week of my life trying get a pem file that will let me talk to APNS, I will link to this, because it seems to be a solution for the future:

Tired of manually creating and maintaining your push notification profiles for your iOS apps? Tired of generating a pem file for your server?

pem does all that for you, just by simply running pem.

Source

April 6th, 2018

Apparently there are a lot of ways to do this? I am suspicious, because I lost days trying to get this to work. This article says:

Now we need to encode these two files in Base64:

openssl base64 -in cert.pem -out cert.txt

openssl base64 -in key.pem -out key.txt

But this gist says:

Step 1: Create Certificate .pem from Certificate .p12

Command: openssl pkcs12 -clcerts -nokeys -out apns-dev-cert.pem -in apns-dev-cert.p12

Step 2: Create Key .pem from Key .p12

Command : openssl pkcs12 -nocerts -out apns-dev-key.pem -in apns-dev-key.p12

None ...

April 5th, 2018

# Apple Push Notifications converting p12 to pem files

None of the commands listed on this page worked for me, and yet that is the topped ranked page in Google. What did work for me:

openssl pkcs12 -in apns_certificates.p12 -out apns_certificates.pem -nodes openssl pkcs12 -in apns_key.p12 -out apns_key.pem -nodes chmod 0400 apns_key.pem chmod 0400 apns_certificates.pem openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert apns_certificates.pem -key apns_key.pem cat apns_certificates.pem apns_key.pem > Peeroapp.pem Source

April 3rd, 2018

# The difficulties of starting a career in tech

This is a good summary of some issues with starting a career in tech. This one jumps out at me, though I realize the issue of portfolios is controversial in some quarters.

More than a couple of times have I met developers looking for web development gigs without a proper portfolio to show off.

Having a portfolio helps to demonstrate what you are capable of doing.

You’ve probably been told that having a personal domain puts you a mile ahead in your ...

March 27th, 2018

# I just lost a week of my life dealing with one bug in a Cordova/Ionic mobile app

Over the last week I had two marathon days where I worked more than 12 hours, and I had 4 normal days where I worked 6 to 8 hours, and all of it was wasted on one bug.

I’ve a friend who wanted an app built that would work for both iOS and Android. He contracted with a team in India to build it. They used Cordova and Ionic.

When they were done with the project they sent it to ...

March 25th, 2018

# Apple takes two weeks to update a forgotten password?

I am amazed by this. On Friday I tried to log into:

https://developer.apple.com

None of my regular passwords worked. Only later did I realize I’d probably set up a separate account for this, using some unusual password. Anyway, I figured this would not be a big deal. I clicked “I forgot password”. I put in my email and then my phone number.

Apple then sent a text message to my phone, with a code that I was to type into the website. ...

March 22nd, 2018

# Even a basic iOS app generates a huge number of errors in Xcode

I’ve decided I should learn how to build native iOS apps. So I am following a basic tutorial for Xcode and iOS.

As soon as I’ve created the project, I try to run it. It does compile and run, but the log is full of hundreds of errors. I’m putting a small sampling below.

Does Apple realize how confusing this is for someone who is just beginning?

I learned how to program back in 1999, writing PHP scripts. One great ...

March 19th, 2018

# CSS is terrible because it is global state

CSS is broken because the style sheets are global. Once upon a time, this didn’t seem terrible because it was thought that CSS would operate as CONSTANTS. But they are not constants. On any modern website or app, the styles are always changing. So the styles are really a global variable, and they have all the problems that 60 years of computer science has taught us to expect from global variables.

There are other problems with CSS. Perhaps the biggest ...

March 19th, 2018

# Online education in the 1960s

Interesting:

As Dear tells it, students were far more interested in using PLATO to chat with one another than to complete their lessons. They even created their own emoticons by taking advantage of a feature within PLATO’s operating system that allowed them to stack letters on top of each other instead of typing them side by side. Typing the word “WOBTAX” in a stack produced a smiley face.

“This was a great pastime of many undergraduates, many of whom would flunk out ...

March 10th, 2018

# The lack of volume in a high-dimensional cube

I wish this felt more intuitive to me:

This is a starting point for intuition:

To provide some intuition consider the situation in two dimensions, as shown in Figure 10. For a point on the circle to be close to the equator, its y-coordinate must be small.

So, this is sort of an expression of the Curse Of Dimensionality?

But perhaps the most amazing thing is that we can not figure out how many spheres might surround a central sphere, in such a ...

March 8th, 2018

# In Apache you can setup a reverse proxy with either RewriteRule or ProxyPass

With Apache I have always done something like this:

ProxyPass / http://127.0.0.1:8080/ ProxyPassReverse / http://127.0.0.1:8080/

But I just saw this interesting bit on StackOverflow:

RewriteEngine On RewriteRule ^(/.*)$http://app-server:8181/jellyfish$1 [P] ProxyPassReverse / http://app-server:8181/jellyfish/

Which does the same thing. The RewriteRule says “grab everything after the ...

February 12th, 2018

# Containers like Docker will not fix the sick politics of your company

Interesting:

Deciding what’s yours and what’s theirs is the basis of every sibling-rivalry détente. In Eric Brewer’s CAP theorem you can pick two of consistency, availability, and partition tolerance as long as one of them is partition tolerance, because, as distributed systems expert Caitie McCaffrey puts it, “physics and math.” In a distributed system that contains humans in multiple time zones, you’re inevitably going to have partitions, and waiting 10 hours for headquarters to wake up and make a decision is ...

February 3rd, 2018

# When things go wrong on a software project

I like this:

Whenever things are not progressing as expected, it is a natural instinct to fall back to micromanagement, but it seldom truly cures the ill. It’s probable that your team disagrees with your vision or its feasibility – and that you’re either not listening to their feedback, or they don’t think you’d care. It’s good to assume that most of your employees are as smart or smarter than you; barking your orders at them more loudly or more frequently ...

February 2nd, 2018

# A very long debate about when to use a library and when to write your own code

The whole thing is interesting, but Jens Stimpfle’s comment comes close to representing my view:

“I see the value in learning by building yourself, but from a software engineering point of view, using a tried and tested framework is likely to give you higher quality product in less time.”

Well, the framework I was to use last (Qt) had bugs that simply can’t be fixed by users (memory leaks, double free leading to segfault when exiting after reloading QML engine) and immature ...

January 18th, 2018

# The continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers

Interesting:

The conclusion seems inescapable that with at least certain kinds of large programs, the continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers who are closely and continuously connected with them.

Source

January 17th, 2018

# Chris Zheng is angry at the Clojure community

This initially sounds like it is going to be interesting:

I say it with much admiration and respect to all the members of community. I’ve learnt so much from being in this microcosm of dynamism, ideas and learning over the past 8 years. Clojure has allowed me to get to know so many amazing people, to travel to a whole bunch of places and to do things that I had never thought possible.

Having said that, I’ve gone from a wide-eyed, ...

January 16th, 2018

# What is up with Narrative.io?

When I go to the website of Narrative.io all I see is black. Is this because I have an ad blocker installed? If so, this is the most dramatic ad-block caused bug that I’ve seen. Something like 20% of everyone on the Web is now using ad-blocking. Narrative.io should keep that in mind when they design their site.

Source

January 6th, 2018

# Yegor Bugayenko: 5 things to improve Java

Yegor Bugayenko offers suggestions for making Java better:

Global Variables. There are Singletons in Java, which, as we all know, are nothing but global variables. Wouldn’t it be great to enable global variables in Java and get rid of Singletons. PHP, JavaScript, Ruby and many other languages have them, why doesn’t Java?

Global Functions and Namespaces To group static methods together we create utility classes, where we have to define private constructors to prevent their instantiation. Also, we have to remember which ...

January 5th, 2018

# How should your company handle data safety and version control?

I’ve been writing software professionally since 1999. The first few years, I didn’t work at a company that used version control. Colin Steele mentioned Subversion to me in 2005. I used it till 2011. Since 2011, every company I’ve worked at has used Git.

From 2005 to 2011 we used Subversion to keep track of all projects. This was at both startups that I helped run, and also while doing most client work. During these years, I only had one ...

January 5th, 2018

# The problem with HTML

(originally from a longer essay)

Just to offer some perspective about where I’m coming from, I’ll start by saying that I think of these as tools that made it easy for beginners to be productive:

Hypercard

Visual Basic (obviously I mean the classic versions, before .NET. The stuff in the 1990s was genius)

I spent 1995 learning HTML and putting together simple websites. I was thinking this is something I’d like to ...

December 26th, 2017

# Jakub Holý on the awesome productivity of Clojurescript

I have found the REPL useful when working with Clojure. I plan to write about this soon. Jakub Holý has a great write-up regarding both Clojure and Clojurescript:

Aside – about us: Telia Norge is Norway’s second largest mobile operator. Our team has spent the past three years with Node.js, React, and (eventually) Redux creating a great webshop (for hardware and subscriptions) to replace an off-the-shelf one, with a great success. Now we set out to make also the lives ...

December 25th, 2017

# Jakub Holý: how to build a shopping cart

Interesting:

However there are other concerns than simplicity and time-to-production that we could have been taken into account, I believe. But I failed to communicate those in time. The current solution would have been perfect if the network transfer times were negligible and if the backend service was 100% reliable. None of that is true. Every call takes at least 300ms and the backend can be overloaded or (as yesterday) unavailable due to network issues. I there is a high ...

December 25th, 2017

This is a very good point:

Imagine you need to call .indexOf on product_code. You might be tempted to add

if (!_.isString(product_code)) return false;

Now you are safe and your code can’t fail. But when I read this code, I immediately think:

Oh, I believed product_code to be always a string but obviously it is not always the case. It must have happened in the past or the author wouldn’t have added this check. I wonder what are the possible non-string values. And what ...

December 18th, 2017

# The penalty of saying something controversial on HackerNews

I made a comment on HackerNews and it was heavily downvoted. I made a good point about the limitations of Git, but people did not want to hear it. This raises an interesting issue about voting systems, in general, is there any way to find stuff that is true, reliably, when the voters might have incorrect beliefs? I think Slashdot went furthest, they took voting systems seriously, and they tried to weight the votes of those who were themselves upvoted. ...

December 12th, 2017

# The Krubner Law Of Ugly Code

The Krubner Law Of Ugly Code states that any computer programming language that has a “return” keyword will tend to be ugly. “Tend” means the tendency is there, though of course, as I’m sure hundreds will rush to remind me, you can write beautiful code in any language.

Here is a simple example from a well-respected project. Here the return is used in a guard clause. This is good code, for the Java language, in that it puts the guard ...

December 12th, 2017

# To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice?

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

http://www.artima.com/intv/interop.html

Bill Venners: To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice? If everyone fully understands and adheres to the contract of the functions of a particular DLL, shouldn’t updating that DLL in theory not break any code?

Anders Hejlsberg: ...

December 12th, 2017

# Camille Fournier (Rent the Runway) – Consensus Systems for the Skeptical Architect

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

ZooKeeper owns your availability. Worth watching. I thought it was amazing when she said that even 5 years ago, even at Google, most programmers were not thinking in terms of distributed computing. Instead, its been the boom in microservices that has driven the adoption of ...

December 12th, 2017

# Sam Newman’s book Building Microservices

I’m posting here some quotes I like from this book:

Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a grid to create a pool of pricing workers, allowing ...

December 12th, 2017

# Why I prefer dynamic-typing over static-typing: the speed of adapting to change

I am a fan of dynamic typing. So is corporate America. The widespread use of PHP, Ruby, Python and Javascript suggest that dynamic typing is useful. Static typing tends to be used in areas where legal regulations create needs that outweigh programmer productivity. If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

In How ignorant am I, and how do I formally specify that in ...

December 11th, 2017

# Watching the AI bots on a game fight each other is like testing a data analysis model

I was researching something else and I followed a link to the game Wesnoth. I downloaded it and spent some time testing it. The game is entirely built by volunteers. The game is slow and somewhat tedious, but they did a great job with the art and they did a great job enabling a variety of scenarios.

They claim they put some time into the AI engine that lets bots play a character. I was curious about that. I set ...

November 25th, 2017

# That’s supposed to be an umlaut in the title

This sounds like a very serious criticism of WordPress:

Yes, I know that’s supposed to be an umlaut in the title. I just can’t persuade WordPress to do it.

As far as I know, WordPress has handled UTF8 for a long time. Does anyone know what is going on?

Source

November 25th, 2017

# VisualBasic could have saved Microsoft Excel

Companies are rebelling against Excel:

Finance chiefs say the ubiquitous spreadsheet software that revolutionized accounting in the 1980s hasn’t kept up with the demands of contemporary corporate finance units. Errors can bloom because data in Excel is separated from other systems and isn’t automatically updated.

Older versions of Excel don’t allow multiple users to work together in one document, hampering collaboration. There is also a limit to how much data can be pulled into a single document, which can slow down analysis.

“Excel ...

November 2nd, 2017

# Denormalization to diverse apps is the correct path forward for media companies

I am very pleased to see that the New York Times now has this cutting edge architecture, which almost certainly the right way forward at this time:

The denormalized log and Kafka’s Streams API

The Monolog is great for consumers that want a normalized view of the data. For some consumers that is not the case. For instance, in order to index data in Elasticsearch you need a denormalized view of the data, since Elasticsearch does not support many-to-many relationships between ...

October 31st, 2017

# Matthew Vagnoni suggests the blockchain can resolve the issue of patient consent

Back in 2015, when I worked at Rollio, we hired Matthew Vagnoni for 2 weeks to come and teach us about Natural Language Processing. Vagnoni is a fantastic person to work with: he is a true expert in his field, yet he manifests very little ego about his extraordinary skill. He was friendly and calm and also authoritative, as he walked us through the various strategies we might take, when connecting to Salesforce, using NLP.

I only recently realized that ...

October 31st, 2017

# XLSX2CSV in Java or Clojure

This page does not seem to rank very high on Google, which makes it difficult to find a good example of how to use Java to convert an Excel file to CSV. Last year, when I worked on this, I recall it took me many days of searching before I found this example.

Also good:

http://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/poi/ss/examples/ToCSV.java

Source

October 27th, 2017

1 Comment

# A developer asks for help, then gets angry at those who help them

This is pretty wild. Some idiot named “Amo” asks why Supervisord is using 100% of the CPU. A fellow named Paul Calabro then gives what sounds like the correct answer. Instead of saying “Thank you” Amo then threatens to downvote Calabro. Either Amo is vicious or they completely misunderstood the answer. Either way, Amo is an idiot.

If I was Amo, I would delete the question, and thus make it private, because the whole thing is embarrassing. In case ...

October 25th, 2017

# Why would anyone choose Docker over fat binaries?

[ [ In this essay, I use the phrase "fat binary" to refer to a binary that has included all of its dependencies. I am not using it to refer to the whole 32 bit versus 64 bit transition. If I was only writing about the world of Java and the JVM, I would have used the word "uberjar" but I avoided that word because I also wanted to praise the Go language and its eco-system. ] ]

They are ...

October 24th, 2017

# You can set environment variables from the Supervisord config

I often tell clients to consolidate their config files, from all their various apps, inside of a folder inside of /etc (I assume Linux servers).

But an interesting alternative is to set the config in the Supervisord config, and thus consolidate all logging and config info in one place.

[program:django] environment = SITE=domain1, DJANGO_SETTINGS_MODULE=foo.settings.local, DB_USER=foo, DB_PASS=bar command = python manage.py command

One could simply have different entries for development, certain branches, ...

October 24th, 2017

# Infinite scroll is a terrible idea 99% of the time

Pinterest introduced the modern infinite scroll, so women would keep scrolling their home page. For a while, Pinterest was the fastest growing large site in the world. Infinite scroll is useful if you’re simply encouraging people to browse over random items. It is a terrible design idea if you are trying to communicate information. As Edward Tufte says, long lists are fantastic if you are trying to communicate choices or many types of information.

So check out the Web ...

October 15th, 2017

# The resulting passive-aggression kills productivity faster than any methodology or technology decision

These are the truest 6 paragraphs that I’ve ever read about software development ideologies:

Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results.

In his 2003 thesis People and methodologies in ...

October 15th, 2017

# What are the benefits of the fictions that guide software development?

Very interesting and well written:

I’m not going to beat up on any of these paradigms, because what’s the point? If software methodologies didn’t exist we’d have to invent them, because how else would we work together effectively? You need these fictions in order to function at scale. It’s no coincidence that the Agile paradigm has such a quasi-religious hold over a workforce that is immensely fluid and mobile. (If you want to know what I really think about software ...

October 11th, 2017

# NLP challenges: word ordering reverses meaning

Another good example:

When words are used in conjunction with each other, they express an idea that is bigger or more meaningful than each word in isolation. The two clauses I’m not happy I’m working and I’m happy I’m not working contain the sames words, in close proximity, but have quite different meanings.

Source

October 5th, 2017

1 Comment

# AWS does not protect you from devops

Once upon a time, people made fun of Linux GUIs because they would give you a graphical way to edit a configuration file, but you still had to know everything in the configuration file, and what the settings did. People made the reasonable point that slapping a pretty graphical interface on a configuration file did not make it any easier to work with Linux.

Nowadays, the clients I interact with say things like “We want to use AWS because then ...

September 20th, 2017

# Where should a spec be defined relative to a function

This is an interesting conversation on the Clojure mailist, and this question is relevant for more than just Clojure. It touches on the main question of language design.

Didier feels it should be possible to define the spec with the function, and this functionality should be added to the Clojure core, so that this would become idiomatic Clojure:

Gary Trakhman argues that this would make the core “defn” macro too slow, and therefore everything about spec should be left to ...

September 18th, 2017

# How to use Zach Tellman’s Clojure library Manifold

These examples are great, and I wish this was more highly ranked in Google, so I’m linking to it.

Source

September 11th, 2017

# Niche facts that Google can’t find

Google works so well that I’m shocked when it doesn’t work. I recall reading an early RFC, from the early 1970s, when someone proposed allowing variable length bytes. I tried searching Google for this. No luck. Google only shows me the famous RFCs, not the failed experiments.

Source

September 11th, 2017

# Why can’t modern tools match Smalltalk?

I often have the impression that Smalltalk in the late 80s and early 90s hit a peak that has yet to be matched. Great effort was instead put toward languages that are less intelligent, and which limit the computer programmer. Does this have something to do with the deskilling of computer programming, as Stanislav says?

“It amazes me just how blindly complacent programmers have been in the face of the ongoing and very successful deskilling of their profession. ...

September 8th, 2017

# The limitations of decision trees

If you are going to buy a house, you will make a list of pros/cons. That much is common sense. It is also an implicit decision tree. Most of these concepts are common sense. But I am struck, yet again, and how quickly this can seem to be unimaginably complex, once you involve the abstract jargon of math:

And once again, I note the sheer awfulness of math notation. How much faster ...

September 8th, 2017

# Gini Impurity is quick to measure and easy to adapt to

I’m trying to convince a client that we need to match records using a Random Forest approach. They have hit the limits of what can be done with simple string matching. I like basic articles like this, for explaining things clearly, and making it obvious how easy Random Forests can be (especially compared to Neural Nets).

Which is the better split? This is a subjective question. In practice, people use different metrics for evaluating splits. The most commonly used metric ...

September 8th, 2017

# Logs are the future of software

Not sure why it has taken so many decades to become obvious, but clearly a history of all events is better than a relational database for keeping track of data. There is no reason why any company should ever use a relational SQL database. The source of truth should be the log, and one’s consumers should build their own denormalized datasets — and these will often work best in a document store, rather than a relational database.

I am very ...

September 7th, 2017

# Peter Williams on Humane Interfaces

Still caught by the nostalgia I mentioned in the 2nd previous post, I re-read Peter Williams post, linked via the same debate. I recall reading this in 2005.

Martin Fowler has posted a nice article on humane interface design (as opposed to minimal interface design). I am definitely on the side of right and good (read: humane interfaces) in this debate. Nothing takes the fun out of programming faster than having to write a bit of code that you know ...

September 7th, 2017

# The failure of the X Window system paved the road for the rise of the Web

This is an amazing article about the absolute failure of the X Window system. Copyright (C) 1994, so this was written just as the Web was gaining momentum, but before it was obvious that the Web would become the main way that different devices could establish a single interface. Among the many good bits, there is this:

Figuring out where a particular resource value is for a running application is much fun, as resource can have come from any of ...

September 7th, 2017

# The loneliness of advanced skills

I decided to re-read The departure of the hyper-enthusiasts. I’ve linked to that essay many times before. It was formative for me, at a formative moment for the Web, so it was formative in many ways, and it captured something important about that moment, and how things were changing.

I find myself getting nostalgic. That is not interesting in and of itself. To be nostalgic for a formative time, about a formative essay, is ordinary enough.

September 7th, 2017

# Blue or Not Blue decision trees

This is a point worth remembering:

“Before we get started I need to clarify something. Theoretical decision trees can have two or more branches protruding from a single node. However, this can be computationally expensive so most implementations of decision trees only allow binary splits.”

Source

September 7th, 2017

# If NodeJS is so great, why are the problems so common?

Callback hell. Take a deep breath and ask yourself why this website needs to exist:

Source

August 12th, 2017

# Supervisord has spent 5 years discussing whether or not it should support a timing mechanism?

I’m not sure what to make of this incredible thread in the issue tracker on Github for the Supervisord repo. Someone asked, in 2012, for a way to time when various apps start, so that they could start app A, B and C in order. On April 16 of this year, 2017, someone wrote:

This issue has been open since 2012, does anyone have ownership over it? It’s clearly important to many users; is there any consensus amongst supervisor contributors?

I ...

August 4th, 2017

# A pure GUI for IP/TCP, a technology with a single mandate, with no burden of also offering semantics or structure or hierarchy

Someone on Hacker News said most developers don’t “get” the Web, and I posted this in response:

My history is surprisingly similar to yours, I started in 1999, I used Notepad as my first text editor, and by 2003 I got caught up in the movement towards making markup strict, which I felt was the mark of professionalism. However, by 2006 I had mostly rejected the notion of “strictness”. There were several things that turned me against strictness. One of ...

August 2nd, 2017

# Cultural differences between Clojure and NodeJS

Right now I have to work with NodeJS, because I have a client who uses NodeJS. I am getting used to the asynchronous style of NodeJS. I notice a subtle cultural difference between Clojure and NodeJS. Most of the tutorials for NodeJS assume the person reading is a beginner, someone who may not understand big complex words like “asynchronous”. In the world of Clojure, there is the assumption that the person reading the article has some experience. With Clojure, there ...

July 24th, 2017

This is a good example of the limits of NodeJS:

async.parallel([ function(callback){ setTimeout(function(){ callback(null, 'one'); }, 200); }, function(callback){ setTimeout(function(){ callback(null, 'two'); ...

July 19th, 2017

# No one should ever use NodeJS, part MMCVXII

I previously wrote about a very surprising bug I discovered in NodeJS/HapiJS.

But today I face a new issue: How to parallelize work in NodeJs? I discovered this fantastic library called paralleljs. It assigns work to child processes. Fantastic idea, very similar to Python. And how do I get required libs into the child process? That is very difficult. From the issues on Github:

to make things a bit more concrete, this snippet would be representative of what I want ...

July 18th, 2017

# Do people love DigitalOne servers?

I’m looking to host my next web site at DigitalOne. If you have experience with them, please let me know.

Source

July 18th, 2017

# When Nginx becomes the bottleneck

This makes me sad. I love Nginx as a reverse proxy, so long as it is invisible and I never have to think about. Realizing that it, too, can be a source of problems really is discouraging.

47,135 connections in TIME_WAIT! Moreover, ss indicates that they are all closed connections. This suggests the server is burning through a large portion of the available port range, which implies that it is allocating a new port for each connection it’s handling. Tweaking ...

July 18th, 2017

# Trying to explicitly enumerate everything that is true is hopeless

This is good:

Let’s take a look at some existing representations. The most famous representation is WordNet. In WordNet, the symbols are groups of words that have the same meaning, called synsets. One synset could be the set consisting of “car” and “automobile.” Each word can be in multiple synsets. For example, “bank” could be in the synset that means river bank and also in the synset that means a place where money is deposited. There are a few kinds ...

July 8th, 2017

# A simple intro to tokenizing with OpenNLP in Clojure

A nice intro:

Finding sentences

Words (tokens) aren’t the only structures that we’re interested in, however. Another interesting and useful grammatical structure is the sentence. In this recipe, we’ll use a process similar to the one we used in the previous recipe, Tokenizing text, in order to create a function that will pull sentences from a string in the same way that tokenize pulled tokens from a string in the last recipe.

We’ll need to include clojure-opennlp in our project.clj file:

(defproject com.ericrochester/text-data ...

July 8th, 2017

# Neural Networks beat Support Vector Machine beat Random Forests

I’ve written about my experience at a startup that used NLP to translate a salesperson’s text message into an entry in Salesforce. At the time, our NLP developer used the Stanford NLP library to try to build a model, but mostly they used a lot of regex and string matching.

I’ve been thinking about how we might have done that project faster and better. I’ve recently been thinking the right approach would have been Random Forests. We were lucky ...

July 8th, 2017

# The struggle to find meaning in words, via word counts in matrixes

Interesting comment:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is the ...

July 8th, 2017

# The problems with DynamoDB

Everything I’ve read suggests to me that companies are drifting towards Cassandra as the best of the NoSQL databases. But some of my clients give me directives such as “Please restrict yourself to AWS technologies”. So I was considering using DynamoDB. And then I read this.

Interesting:

When a customer does an action, it is reasonable to think that this kicks off a number of actions in your system. One action might spawn a number of events to be sent into ...

June 29th, 2017

# The curse of too many dimensions

Damn, this is interesting:

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic optimization.

There are multiple phenomena referred to by this name in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining, and ...

June 29th, 2017

# Morphism is the starting point of category theory

I guess I knew this, but still it is interesting to read:

In algebra, a homomorphism is a structure-preserving map between two algebraic structures of the same type (such as two groups, two rings, or two vector spaces). The word homomorphism comes from the ancient Greek language: ὁμός (homos) meaning “same” and μορφή (morphe) meaning “form” or “shape”.

Homomorphisms of vector spaces are also called linear maps, and their study is the object of linear algebra.

The concept of homomorphism has been ...

June 29th, 2017

# All the weird characters in Clojure explained

This is a very useful guide.

Source

May 31st, 2017

# Cost cutting at British Airways caused the catastrophic failure of its database service

This is obviously a lie:

BA insisted that the computer system outages that grounded hundreds of flights were not linked to ferocious cost-cutting and the outsourcing of work to contractors in its IT department.

There is absolutely no way to get a melt down of this scale unless you’ve just fired your key staff, so you’ve nobody left who knows how to bring the system back online. I’ve worked at very small startups, with just 3 people on staff, and we ...

May 27th, 2017

# Is Clojure dying?

Interesting.

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure ...

May 9th, 2017

# Don’t use PM2 with NodeJS

Interesting:

I don’t use any of the “node” process managers because they don’t solve the problem of starting your process at server boot. I use init scripts, or upstart, or runit or systemd like you’re supposed to for a service. Anyone using forever or pm2 likely doesn’t understand sysops.

Source

May 5th, 2017

# Very long GC pauses when NodeJS is about to die from memory exhaustion

I sometimes see stuff like this in my logs, when my NodeJS app has reached critical:

856870 ms: Mark-sweep 5647.1 (6029.8) -> 5625.8 (6029.8) MB, 10733.9 / 0.0 ms [allocation failure] [GC in old space requested]. 867774 ms: Mark-sweep 5625.8 (6029.8) -> 5625.8 (6030.8) MB, 10903.4 / 0.0 ms [allocation failure] [GC in old space requested]. 878580 ms: Mark-sweep 5625.8 (6030.8) -> 5625.8 (5996.8) MB, 10806.3 / 0.0 ms [last resort gc]. 889353 ms: Mark-sweep ...

April 26th, 2017

# More NLP for Clojure

This looks fantastic:

To do its magic, postagga extracts the phrase structure of your input, and tries to find how do this structure compare to its many semantic rules and if it finds a match, where in this structure shall he extract meaningful information.

Let’s study a simple example. Look at the next sentence:

“Rafik loves apples” That is our “Natural language input”

First step in understanding this sentence is to extract some structure from it so it is easier to interpret. One common way ...

April 26th, 2017

# TCP can fail in many, many ways

Amazing that the world depends on a technology with so many quirks:

So, if we read that data first, and LINGER, are we good to go? Not really. The close() call really does not convey what we are trying to tell the kernel: please close the connection after sending all the data I submitted through write().

Luckily, the system call shutdown() is available, which tells the kernel exactly this. However, it alone is not enough. When shutdown() returns, we still have no indication ...

April 23rd, 2017

# Thinking about Concurrency, Raymond Hettinger, Python core developer

A basic overview of concurrency, most of which I knew, but he had one good line at the end: “If you add enough locks to your concurrent code, it is no longer concurrent. It is completely sequential, you’ve simply made it a lot more complicated than single threaded code.”

Source

April 13th, 2017

# Python has greenlets

I’m sad that Python is doing so well. It gets all the attention that should go to Clojure. They are both great for AI and NLP, but Python doesn’t have as many great ideas for concurrency. Or rather, the language doesn’t, but the eco-system makes up for that with a wealth of interesting ideas — and it is sad to see so much effort go to see making Python work, rather than just starting over with something that has a ...

April 13th, 2017

# Katharine Jarmul describes the bias in Google’s Word2Vec software

Interesting:

I must warn you that parts of this post are disgusting, disturbing and awful. If you are having a rough day, feel free to save for another time. If you are already sick of seeing hateful language, this is likely not a post to read at present. That said, I feel my duty as a former journalist to look at it, expose it, and hope to spark better conversations around how we handle both implicit and explicit bias and prejudice ...

April 11th, 2017

# Why are people ignoring the problems with NodeJS?

Two weeks ago I wrote “A surprising NodeJS failure mode: deterministic code becomes probabilistic under load“. Since then I’ve been reading up on NodeJS and learning more about its substantial failure modes. I’m left feeling very surprised at the success that NodeJS is having.

Just to give you a sense of what I mean, this is how The New Stack sums up the success of NodeJS:

Ready for a Long Term Node Relationship?

In just seven short years, Node.js has gone ...

March 30th, 2017

# A surprising NodeJS failure mode: deterministic code becomes probabilistic under load

I have a client. Let’s say they run some scripts that scrape the web for data, then they run some NLP scripts to pull out the facts they need, then they need to insert that into their MySQL database. When they gather facts, they need to know if the facts are about an organization that is already in the database. So they asked me to create an API that could take their new facts, and try to match those facts ...

March 29th, 2017

# Skip Lists: A Probabilistic Alternative to Balanced Trees

Interesting:

Skip lists are a data structure that can be used in place of balanced trees. Skip lists use probabilistic balancing rather than strictly enforced balancing and as a result the algorithms for insertion and deletion in skip lists are much simpler and significantly faster than equivalent algorithms for balanced trees

A node that has k forward pointers is called a level k node. If every (2i)th node has a pointer 2i nodes ahead, then levels of nodes are distributed in a simple pattern: 50% are level 1, 25% ...

March 29th, 2017

# Novelty seeking multi agent systems

Interesting:

This paper considers novelty-seeking multi-agent systems as a step towards more efficient generation of creative artifacts. We describe a simple multi-agent architecture where agents have limited resources and exercise self-criticism, veto power and voting to collectively regulate which artifacts are selected to the domain i.e., the cultural storage of the system. To overcome their individual resource limitations, agents have a limited access to the artifacts already in the domain which they can use to guide their search for novel artifacts. Creating geometric images called spirographs as a case study, we show ...

March 29th, 2017

# Bushy join trees for snowstorm queries

I am taken aback by the amount of jargon here:

Many workloads for analytical processing in commercial RDBMSs are dominated by snowstorm queries, which are characterized by references to multiple large fact tables and their associated smaller dimension tables. This paper describes a technique for bushy join tree optimization for snowstorm queries in Oracle database system. This technique generates bushy join trees containing subtrees that produce substantially reduced sets of rows and, therefore, their joins with other subtrees are generally much ...

March 29th, 2017

# How to combine novelty search with fitness-based evolution

I need to go back and read this whole article:

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It is still unclear, however, how some key parameters and algorithmic components influence the evolutionary dynamics and performance of novelty search. In this paper, we conduct a comprehensive empirical study focused on novelty search’s algorithmic components. We ...

March 29th, 2017

# Novelty search

Interesting:

Novelty search is a recent algorithm geared toward exploring search spaces without regard to objectives. When the presence of constraints divides a search space into feasible space and infeasible space, interesting implications arise regarding how novelty search explores such spaces. This paper elaborates on the problem of constrained novelty search and proposes two novelty search algorithms which search within both the feasible and the infeasible space. Inspired by the FI-2pop genetic algorithm, both algorithms maintain and evolve two separate populations, ...

March 27th, 2017

# What every object on a network needs

There is no need to rely on the Web for these things:

The problem with operations is that we have to define, a-priori, the semantics of each individual operation. We cannot tell, unless we know beforehand, whether the operation is safe to call multiple times, whether the result of an operation will be the same every time we call it, whether the result can be cached, and if so for how long. Many years of building distributed applications have told ...

March 27th, 2017

# Use pseudo URIs for an id

A long article, but this is interesting:

When facing the problems above, my team at SoundCloud started exploring alternatives that would allow for us to have simple, scalar values that were still rich enough to act as good identifiers across our hundreds of microservices. Reading through decades of industry work on the matter, we found something simple that could help us: Uniform Resource Names, or URNs. URNs were a type of Uniform Resource Identifiers (URIs) that, as opposed to URLs, were ...

March 27th, 2017

# Is NewSQL the next big thing?

I like the fact that Kinesis has an SQL interface:

I can see this being useful for Business Intelligence. But that seems like a niche to me. Both SQL and NoSQL solved big, universal problems.

But maybe the SQL is just window dressing? Maybe the real breakthrough is a distributed system with strong consistency guarantees? It would be a very big deal if someone found a way around the CAP Theorem. But otherwise, NewSQL is just more bad marketing for ...

March 27th, 2017

# Designing a neural net is really hard

Interesting:

2bitencryption says:

Designing a neural network is a thousand times harder than I imagined.

After AlphaGo, I tasked myself with creating a neural network that would use Q-Learning to play Reversi (aka Othello).

At that point, I had already utilized Q-Learning (the tabular version, not using a neural network) for some very simple and mostly proof-of-concept projects, so I understood how it worked. I read up only perceptrons, relu, the benefits/disadvantages of having more/fewer layers, etc.

Then I actually started on the project, thinking ...

March 26th, 2017

# Training a neural net

A great intro:

Training

Training is how we teach a neural network what we want it to learn. It follows a simple five step process:

1.) Create a training data set, which we will call x and load its labels as targets y

2.) Feed the x data forward through the network with the result being predictions y’

3.) Figure out the “loss” of the network, which is the difference between the predictions y’ and the correct targets y

4.) Compute the “gradient” of the loss ...

March 26th, 2017

# Transform real data into a tensor, that is, tensor flow

A good introduction:

Source

March 17th, 2017

# What if we had computer bits that allowed more than two values?

The Red-Black tree is interesting because it can store the color using only bit, and one bit can have one of two values, often represented as either a 0 or a 1. That is the nature of our silicon hardware. But if we had hardware where the basic charge could be one of several values, then it would possible to do multi colored trees. If we had hardware that could hold one of 20 values, it would be possible to ...

March 17th, 2017

# An InputStream requires more memory as it has to buffer the whole file

This is exactly the opposite of what I’ve always understood, and what I’ve read elsewhere:

When opening a workbook, either a .xls HSSFWorkbook, or a .xlsx XSSFWorkbook, the Workbook can be loaded from either a File or an InputStream. Using a File object allows for lower memory consumption, while an InputStream requires more memory as it has to buffer the whole file.

The stream requires more memory? That is crazy. It’s a stream. And what is a File object if not ...

March 8th, 2017

# Joe Armstrong: computer programming is the alteration of values in a key value database

An incredible assertion that would be dismissed if it was coming from someone less talented than Joe Armstrong. It is interesting to think that RAM is a key value store, and so anything that updates RAM is altering the values of a key value store.

He starts by saying that he often ends up with miscellaneous code:

When I’m programming I often get to the point were I say there shoulda function foo/2 in lists.erl but their isn’t. There should be ...

March 1st, 2017

# How to get Nginx to wait for your app

I’ve been struggling with a slow NodeJS app which is behind a reverse proxy handled by Nginx.

I found this bit useful:

Syntax: proxy_max_temp_file_size size; Default: proxy_max_temp_file_size 1024m; Context: http, server, location When buffering of responses from the proxied server is enabled, and the whole response does not fit into the buffers set by the proxy_buffer_size and proxy_buffers directives, a part of the response can be saved to a temporary file. This directive sets the maximum size of the temporary file. The size of data written to ...

February 28th, 2017

# How do we teach self-driving cars to avoid hitting people on bicycles?

This is a good conversation:

ChuckMcM :

“Now Johnny, when you ride your bike you must wear your I-am-a-bike vest and follow these patterns or the cars are likely to kill you.” :-) Summary is that “people riding bikes” (PRB) is a much denser image set than “people in cars” (PIC) and “people walking or jogging” (PWJ), and the PRB objects have a much higher dynamic angular vector capability (they can change direction extremely quickly) combined with a wider dynamic velocity vector ...

February 14th, 2017

# Different NLP approaches

A nice summary:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is ...

February 11th, 2017

# Tony Arcieri’s very good defense of Rails

I just had a project where I had to write an app that could fire several million requests against an API, with data pulled from a database. I wrote the whole thing in NodeJS and HapiJS, and it was an agony. The uncontrolled async of NodeJS tripped me up. If I need to fire 10 million HTTP requests, how do I do that without blowing the stack? As it was, the functions would pile up, and then I’d get a ...

February 7th, 2017

# systemd is no longer interested in the grubby details of reality

Interesting:

Unfortunately, since then systemd developers have shown an unfortunate and increasing streak of idealism. More and more, systemd seems not to be interested in dealing with the world as it actually is, with all of its grubby inconvenient mess; instead it simply assumes some pure and ideal version of things. If the world does not measure up to how it is supposed to be, well, that is not systemd’s problem. Systemd will do the nominally right thing no matter how ...

January 28th, 2017

# NLP as a simple four-step formula: embed, encode, attend, predict

Interesting:

Let’s say you have a great technique for predicting a class ID from a dense vector of real values. You’re convinced you can solve any problem that has that particular input/output “shape”. Separately, you also have a great technique for predicting a single vector from a vector and a matrix. Now you have the solutions to three problems, not two. If you start with a matrix and a vector, and you need a class ID, you can obviously compose the ...

January 27th, 2017

# Paul Phillips denounces Scala

Pacific Northwest Scala 2013 We’re Doing It All Wrong by Paul Phillips For a computer programming tech conference, this talk has a surprising amount of emotional rawness. Phillips is denouncing Scala both on a personal level and on a technical level.

Source

January 27th, 2017

# One team that gave up on Scala

It’s interesting how Scala was initially seen as “A Better Java” but now is seen as having been to experimental in its approach.

I remember when I first saw the potential issues of scaling Scala at Gravity back in 2009/10ish. It was close to the end of the day when we had a major issue reported in production that was affecting our large customers. Several of us started investigating and were able to track the source of the issue. The ...

January 27th, 2017

# Scale programmers do not trust Martin Odersky

This whole conversation is incredible. Here is a forum full of Scala programmers, and they seem to be talking against Odersky’s proposal. And with some anger. I’ve never seen a Rubyist talk about Matz like that. I’ve never heard a Clojurist talk about Rich Hickey that way. Java programmers are, at most, muted in their criticism of Gosling, with the overall recognition that “mistakes were made” but made in good faith. I suppose you could argue that Pythonists are split ...

January 27th, 2017

# The failure of hierarchy based on category

This is good:

The Hierarchy Problem

Every time I start at a new company, I struggle with the problem when I’m creating a place to put my Company Documents, e.g. the Employee Handbook.

Do I create a folder called Documents and then create a folder called Company in that?

Or do I create a folder called Company and then create a folder called Documents in that?

Both work. But which is right? Which is best?

The idea of Categorical Hierarchies was that there were Base Classes ...

January 27th, 2017

# Logicians hate randomness

My strongest subject was always logic, even though I was terrible at most other forms of math. And though I have doubts about randomness, I certainly love inconsistency, which can be approached in a logical way. Every computer programmer who writes concurrent code has to deal with inconsistent data, and it would be great if logicians thought more deeply about how to formalize this.

With Gödel, at first there was a lot of shock. As a student in the late ...

January 24th, 2017

# Movio.co decides to switch from Scala to Go, and they reference my essay

Since they cite me (and my essay from 2014) as part of their decision making process, I want to throw in 2 cents here. They ended up deciding on Go, whereas I have ended up preferring Clojure, yet I agree with a lot of what they say, so I’ll try to clarify why I ended up with a different decision than what they made.

I understand what they mean when they write:

I think the first time I appreciated the positive ...

January 18th, 2017

# Modern programming depends on DSLs, and Lisp is the best as DSLs

Anders Hovmöller seems to be stuck with a mentality shaped by the C++ language, and so he doesn’t get the important of DSLs.

In C++ land it’s well known that you must keep to a subset of the language in a code base to keep sane. It’s also known that every time you increase the size of the subset you are making the situation worse even if it solves a small problem at hand.

In Lisp land on the other hand, all ...

January 8th, 2017

# This year’s goal: move away from WordPress

Like a lot of people, I first used WordPress because it was quick and easy to get going, and because my designer knew it very well. But I dislike the dependence on MySQL, I dislike its slowness, and I dislike the security problems that I’ve had (this site has been hacked twice).

For my modest goals, I only need a static site, so I plan to switch to Cryogen. I was interested by this article:

For a while I ...

January 8th, 2017

# Different people have different strengths

I love this :

I’ve had similar self-doubt in the past, I considered myself fast but sloppy. I changed my opinion after a workshop on different team roles at my previous job. A lot of it was boring workshop fluff, but I loved the core message: that many personality traits aren’t purely positive or negative. Perfectionists are nitpickers. Fast developers are sloppy. Experienced ones overthink stuff. Bleeding-edge evangelists ruin long-term stability etc. There are two sides to every coin. We tend to ...

January 8th, 2017

# Can event logs, as a software architecture, really work?

This is an interesting bit of criticism of this architecture, an architecture which certainly has gotten a lot of attention over the last 4 years:

I have worked on, or cleaned up, 4 different CQRS/ES projects. They have all failed. Each time the people leading the project and championing the architecture were smart, capable, technically adept folks, but they couldn’t make it work.

There’s more than one flavor of this particular arch, but Event Sourcing in general is simply not very ...

December 15th, 2016

# Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

I posted this question to StackOverflow and I would appreciate any help figuring out what the memory issue is:

Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

Source

December 14th, 2016

# Ugly engineering

This sounds hideous:

I used to work on the DynamoDB team. Throwaway account because my normal account can be tied back to my real name.

“Each hash key resolves to a number of possible servers the data can be on. Data is replicated across several of these servers. For redundancy. The hash key determines which shard to use. On individual machines, each set of data is stored by a compound key of hash key and sort key (if there is a sort ...

December 13th, 2016

# Beautiful engineering

Wow. Sometimes I witness some engineering that leaves me feeling joy and inspiration. I felt that way about the Golden Gate Bridge. But sometimes I feel that way about code. For instance, right now, I just learned that the defun macro for Clojure allows this function for calculating a fibonacci number.

A fibonacci function: (defun fib ([0] 0) ([1] 1) ([n] (+ (fib (- n 1)) (fib (- n 2))))) Output: (fib 10) ;; 55

That ...

December 7th, 2016

# The crazy process of trying to release a Salesforce app

Right now I’m helping a client release an app for Salesforce in the Salesforce App Exchange. The process for releasing an app is completely insane and very poorly documented (or rather, there is an abundance of documentation for various parts of the process, but there is no summary of the process). I was lucky to stumble upon this post, which offers the nearest thing to a summary of the process:

1) Create a Dev Org

2) Become a Salesforce.com Partner

3) Create Test ...

November 26th, 2016

# Clojure for XML

This is a fantastic overview of different approaches:

Zippers are probably the easiest way to manage xml – once you grok them.

Zippers are a strange beast. Wikipedia describes them as:

A technique of representing an aggregate data structure so that it is convenient for writing programs that traverse the structure arbitrarily and update it’s contents…

I like to think of a zipper as a kind of pointer to part of a tree – at any time if you have a tree of nodes ...

November 26th, 2016

# How far can code be optimized

This is funny and interesting:

When you’re competing with another player, they will probably find a way to beat your score by just a few points. Let’s say my score is 340 and a friend beats me with a score of 335. (lower is better. The score is just the number of executed instructions) What follows is a bunch of head-scratching about how you could possibly get any more cycles out of the algorithm. After an hour of staring and trying ...

November 14th, 2016

# NLP with Clojure and OpenNLP

This is a fantastic introduction:

Natural Language Processing (NPL) opens the door to the possibility of turning otherwise inert text into meaningful or, more interestingly, actionable information. It is the latter that I am interested in and what this installment will focus on. I will explore the basics of NLP using the OpenNLP library and Clojure to convert a sentence into a useful structure to store or act on. More specifically, my goal is to take simple sentences that indicate the ...

November 13th, 2016

July 21st, 2016

# The difference between Kafka and Kestrel

For my purposes, for the next 6 months, it seems that Kestrel will be all I need. I’d have to be very successful before I would need anything as complex as Kafka.

The biggest conceptual difference is that Kestrel is a simple stand alone queue where as Kafka is a full fledged queuing system.

Kestrel runs on a single machine and has no concept of clustering or failover or any other features you might expect in a queuing system. Instead the clients ...

July 21st, 2016

# RabbitMQ is difficult, Kafka is awesome, Kestrel is beautifully simple

A great review of these 3 queues:

Not mentioned below is that RabbitMQ works hard to guarantee delivery of a message, so it is slow, but that is because it is in some ways doing more than Kafka.

RabbitMQ:

I created 4 queues, wrote a ruby client and started inserting messages. I got a publishing rate of about 20k/s using multiple threads but I got a few stalls caused by the vm_memory_high_watermark, from my understanding during those stalls it writing to disk. ...

July 21st, 2016

# Kestrel is as simple as Redis but Kestrel is an actual queue

I feel stupid that I didn’t look at Kestrel sooner. I feel especially stupid that I’ve been using Redis when I didn’t actually need it.

Redis has a beautifully flexible API which makes it tempting to use Redis for everything, but Redis is really a cache that focuses on speed above all else. Everything in Redis has to fit in memory, and Redis will drop anything that can’t fit in memory. Also, the stuff in Redis can be mutated, ...

July 21st, 2016

# Only use Amazon SQS if you need a high-latency high-concurrency service

Also interesting, don’t use Amazon SQS unless your needs fit this very specific model where you can deal with the latency and make up for it by being highly concurrent.

I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is ...

July 21st, 2016

# For Apache Storm, use Kestrel

There is a lot in this thread that is interesting. This is coming from Nathan Marz, the guy who invented Storm. He says he uses Kestrel, so that is a big endorsement.

You want to make sure that your spout source can support the out of order acking that Storm requires for guaranteed message processing. We use Kestrel because it has this property and is the simplest. RabbitMQ is another good one to consider.

Source

July 20th, 2016

# The problem with RabbitMQ

Seems like RabbitMQ is good but not perfect:

Much like you’d chuck memcached on each of your web servers and access them in a ring, Darner can occupy a small niche on each box in your fleet’s resources. Tens of MB of RAM and negligible CPU opens up hundreds of gigabytes of queue spool per node. As queue size grows, memory usage remains constant.

Contrast this with Redis, which is speedy but limited in queue size to what will fit ...

July 20th, 2016

# ZeroMQ is the opposite of a queue

Reading this again, several years after the first time I read it, and now it occurs to me: I would use ZeroMQ only in exactly those situations where I would not use a queue. If I only need a queue, I can use Kestrel. But if I need some messaging pattern that is not supported by any queue, then I would use ZeroMQ. That is, if I had a truly unique situation that needed a unique pattern, then ZeroMQ would ...

July 20th, 2016

# Everything you need to know about queues

This looks like an amazing resource and I’m sure I’ll go back and read more soon.

Source

July 20th, 2016

# The New York Times uses RabbitMQ

RabbitMQ is written in Erlang, so I was inclined to think well of it, though I heard criticism of it. And then the New York Times used it. A very surprising vote of confidence in RabbitMQ:

This architecture – Fabrik – has dozens of RabbitMQ instances spread across 6 AWS zones in Oregon and Dublin. The instances are organized into “wholesale” and “retail” layers. Connection to clients is via websockets/sockjs.

Upon launch today, the system autoscaled to ~500,000 users. Connection times ...

July 20th, 2016

# Gauges uses Kestrel which is based on work from Blaine Cook

Why does Kafka get all the attention if Kestrel is so reliable? I assume this is because Kafka can do so much more, though of course the devops work of Kafka can be frightening. I didn’t know about the Blaine Cook connection (of Twitter fame):

Before I get too far a long with this fairy tail, let’s talk about Kestrel — what is it and why did I pick it?

Kestrel is a simple, distributed message queue, based on Blaine Cook’s ...

July 20th, 2016

# Kestrel versus Resque

Interesting note about these queues, which certainly convinces me to use Kestrel:

There are a lot of things that could be done with either Kestrel or Resque. Because Resque is backed by Redis, you have to remember that all of the messages waiting to be processed have to be able to fit in the RAM of the Redis server, with Kestrel you could queue millions or billions of messages and then start to pull them off.

The biggest difference between the ...

July 20th, 2016

Very worrisome, though no sane person would use MongoDB to track something involving money. That’s simply not what MongoDB is for.

Read uncommitted allows all kinds of terrible anomalies we probably don’t want as MongoDB users.

For instance, suppose we have a user registration service keyed by a unique username. Now imagine a partition occurs, and two users–Alice and Bob–try to claim the same username–one on each side of the partition. Alice’s request is routed to the ...

July 20th, 2016

# How to return a disallowed field in GraphQL

Best Practices for GraphQL are still in a very immature stage, but these two ideas both have some merit:

First, return null for the requested field. This seems to work well in cases where there is no real harm in asking for a particular set of data and no real harm in denying it.

A good example would be asking for the email of a user where the backend only provides the user’s email to that user themselves. If I request my ...

July 20th, 2016

# Why has Google released gRPC

This is interesting:

Google has been using a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across our data centers for over a decade. Our internal systems have long embraced the microservice architecture gaining popularity today. Having a uniform, cross-platform RPC infrastructure has allowed for the rollout of fleet-wide improvements in efficiency, security, reliability and behavioral analysis critical to supporting the incredible growth seen in that period.

Stubby has many great features – ...

July 13th, 2016

# Why I hate Ruby On Rails

I’m trying to help a friend with a very small Ruby On Rails project, where she needs a few elements of the interface tweaked. But this is Ruby On Rails, so of course I’m losing 3 hours trying to get setup. I’ve been drawn into the endless swamp of upgrading various libraries, which seems to be an automatic part of dealing with Ruby On Rails. If this was Clojure, or NodeJS, setup would have taken 5 minutes and I would ...

July 13th, 2016

# Mixins are harmful — the Javascript edition

Over the years there have been arguments against mixins, but the dominance of Object Oriented Programming meant that mixins still survived and had theoretical justification. Now, the spread of the Functional Paradigm has put renewed pressure on mixins. Dan Abramov at Facebook just came out with an essay against mixins:

“How do I share the code between several components?” is one of the first questions that people ask when they learn React. Our answer has always been to use ...

July 10th, 2016

# Edges is just a wrapper around the real results that adds extra data used for slicing the result cursor

I think the “edges” and “nodes” terminology is one of the most confusing things about React / Relay / GraphQL, at least at first:

Relay connection defines a relationship an an object that has two root fields: edges and pageInfo. Edges is just a wrapper around the real results that adds extra data used for slicing the results (cursors), PageInfo has metadata related to the current page. This is how it looks like:

connectionName { edges { ...

July 10th, 2016

# Make the Web compatible?

It must be 1999 again because someone is writing about browser incompatibilities:

Users won’t switch browsers, they’ll switch sites

You might think that users will switch browsers to use your site. But many won’t or can’t.

Users have no patience for things that don’t work, and they’ll just go to a competitor’s site instead. Failing at a critical point could turn a potential user away forever. According to Akamai,

32% of users who encounter a problem on your site are less likely to make ...

June 30th, 2016

1 Comment

# The Agile process of software development is often perverted by politics

For those of you who don’t want to read this whole essay, here is the summary:

The word “agile” has a plain meaning in standard English, and that meaning was considered something positive by software developers, so much so that the most successful new development process of the last 30 years calls itself “Agile”. However, at many of the companies that I have worked, actual agility is suppressed because of various political factors. Fear wins out over trust. Instead of actual ...

June 29th, 2016

# How much time and energy should a computer programmer spend learning a proprietary server system?

By itself, this is an anecodote about Google’s particular system: (but see my point at bottom)

Is there a lesson in this? Well, if I were in the business of programming Google App Engine, a few days’ effort up front to get it going might seem not to be a big deal. However, when I visit a team who have just added a new person, I invariably find that person struggling to set up their workstation. There’s usually someone in the room ...

June 29th, 2016

# Is Phoenix going to take over the tech industry?

If not, why? This has been one of the big puzzles of the last 30 years: if some technologies are clearly superior, then why don’t they take over? Apparently because they lack an element that appeals to management in big corporations? That’s always been the argument against Lisp: it is great for the individual master craftsman, but it doesn’t work in a big corporation full of badly paid, mediocre programmers.

Erlang has been the most safe, resilient technology out there ...

June 23rd, 2016

# Pagination with React / Relay / GraphQL

This is an awesome article:

Relay proposes a standard to define a has-many relationship for a GraphQL field. This standard defines a common structure that allows Relay to paginate and filter the results in an efficient way by using cursors, which I’ll explain in a bit.

This is the definition of a Relay connection (from the Relay connection specs):

Relay’s support for pagination relies on the GraphQL server exposing connections in a standardized way. In the query, the connection model provides a standard ...

June 15th, 2016

# Demanding ssh key login to a server is safer than allowing passwords

This is good and true:

Require ssh key authentication

We tend to avoid passwords for logging into servers. There was a lot of discussion around this after Bryan’s original guide came out, but I tend to fall into this camp as well. Here are a few notes on this:

Passwords can be brute forced. Guessing a public key is so essentially impossible that they can be considered perfectly secure

What ...

June 14th, 2016

# Object/Relational Mapping is rarely worth it

I think this can be stated simply: for your basic operations (CREATE, UPDATE, READ, DELETE) then an ORM offers some convenience. But ORMs break down as soon as you have a complex query. And yet, if you start using an ORM because it helps you with the simple queries, it will seem natural to you to extend it and try to use it for the complex queries.

This is a strained analogy to the Vietnam war:

Object systems are typically characterized ...

June 11th, 2016

# AWS is inappropriate for small startups because its complexity demands a specialist

Sean Hull references a conversation that he and I had in Slack. I would like to expand on the argument that I made then. You might want to read his essay first, where he makes these points about AWS:

1. Over 70 services offered

2. Still complex to build high availability

3. Need a dedicated devops

4. Orchestration involves many moving parts

5. Troubleshooting failed deployments is difficult

At the time of our conversation I had crashed an AWS instance and we were having trouble fixing ...

June 11th, 2016

# GraphQL is the best thing about React / Relay / GraphQL

React / Relay / GraphQL:

The big breakthrough here is obviously GraphQL. Not React or Relay, since there are many competing implementations that do similar things, but GraphQL. Even though there are graph query languages out there (Gremlin, etc) they were not suited to querying JSON over the wire. GraphQL is ideal. For my next project, I hope to do a Clojure implementation of most of GraphQL, because I think it can be married to Om.Next in a very powerful way.

Actually, ...

June 10th, 2016

# Krubner, you saved my day

Aw man, after all the mistakes I make, it is nice to know I help people sometimes:

Source

June 10th, 2016

# Sometimes I make a bad decision

It’s very, very, very rare, almost unheard of, but sometimes I make a mistake.

I was helping at Open Road this spring. Sean Hull was also helping. Open Road runs a bunch of WordPress sites, such as The Line Up. All of the images for these sites were stored on a central NFS server. All of the machines were starving for hard disk space. The previous Director Of Technology had made many, many bad decisions, including the decision to save ...

June 7th, 2016

# The Belief that Tests are Smarter than Code Telegraphs Latent Fear or a Bad Process

This part is really good:

Programmers have a tacit belief that they can think more clearly (or guess better) when writing tests when writing code, or that somehow there is more information in a test than in code. That is just formal nonsense. The psychological perspective is instructive here, and it’s important because that — rather than any computational property — most drives developer behaviour. If your coders have more lines of unit tests than of code, it probably means one of several things. They may be ...

June 7th, 2016

# The problem with unit tests

Interesting:

Unit testing is of course not just an issue in object-oriented programming, but the combination of object-orientation, agile software development, and a rise in tools and computing power has made it de rigueur. As a consultant I often get questions about unit testing, including this real one from a recent client of mine, Richard Jacobs at Sogeti (Sogeti Nederland B.V.): My second question is about unit tests. If I remember correctly you said that unit tests are waste. First, I was surprised by that. Today however, my ...

June 6th, 2016

# Ruby On Rails is obsolete

This is a strange conversation to be having in the year 2016. Ruby On Rails changed the technology industry in 2004, but it can not claim to be winning in 2016. It still lacks a story for concurrency. It is being undermined by 2 forces:

1.) the need for concurrency, which is offered by other eco-systems, such as those that run the JVM. A language like Clojure offers the high level of meta-programming that Rubyists love, but with the vast arsenal ...

June 2nd, 2016

# Sometimes peer review delays progress for many decades

This is what real leadership looks like:

Morris intentionally consumes H. pylori. Like Marshall, he becomes ill, but unlike Marshall, he is not completely cured by antibiotics. The infection will remain with him for three years.

I wish he had been given a medal for his self-sacrifice. He was attempting to save millions from suffering.

Interesting, though very disappointing:

1881 Klebs notes the presence of bacteria-like organisms in the lumen of the gastric glands.

1889 Walery Jaworski describes spiral organisms in sediment washings ...

May 22nd, 2016

# Why does Rails become awful?

Interesting:

Let me tell you a story about what can happen in a convoluted Rails codebase. Once, I joined an existing project. It was a huuuuge app which was running an on-line shopping community website. Complicated sales model, complicated promotions, complicated product setups, coupons, user groups, messages – it had it all. I joined them to help ship a few new features. One of my early tasks was to…add a link to something on some page. It took me few days ...

May 15th, 2016

# The dumbest name for a good architecture is “serverless”

I strongly agree that the idea is good, but the name “serverless” is really stupid:

So… this is backend infrastructure (i.e. it runs on a server) to let your team deploy “severless” apps…

So they’re serverless. Except the server you’re running it on. And the lamba/style code they wrote and uploaded to it.

But serverless, because no ops staff required. Except the ones who installed and maintain this.

This is like a snake eating its own tail and wondering what hurts.

Edit: despite my sarcasm, ...

May 10th, 2016

# The sad, slow way a system of cron scripts becomes ugly

Do you have a chore that needs to run in the background, maybe once a day, or once an hour? Cron scripts will save you! They are the most amazingly amazing thing God has invented since Adam and Eve! For sheer wonderfulality they have no equal among the products of mortal fallen flesh!

At least at first.

The simple cron script is wonderfully direct and efficient. But the first can lead to a second. The second can lead to a third. ...

April 10th, 2016

# What do AWS services actually do?

I love this:

Data Pipeline Should have been called Amazon ETL

Glacier Should have been called Really slow Amazon S3

Kinesis Should have been called Amazon High Throughput It’s like Kafka

I have used this but I did not know the name:

Snowball Should have been called AWS Big Old Portable Storage Use this to Get a bunch of hard drives you can attach to your network to make getting large amounts (Terabytes of Data) into and out of AWS It’s like Shipping a Network Attached Storage device to AWS

These are useful names that actually describe what ...

March 27th, 2016

# The fanaticism of “test driven development” is slowly fading

This was one of those trends that went too far and I’m glad to see people talking about reasonable limits:

If you’re part of the Ruby On Rails community for a long time, you’ve probably read tons of articles about testing Rails application (less these days, though). Although there always have been diverging opinions on the matter, it seems the common wisdom was to say that you had to test everything: models, controllers, views and full-stack tests. Oh, and you had ...

March 26th, 2016

# RESTful APIs are dead, long live GraphQL

We can argue whether RESTful APIs ever existed, since very few ever implemented HATEOAS. Ruby On Rails claimed it offered RESTful APIs, but it never offered HATEOAS. After 10 years of failure, the world is looking for something new.

Interesting:

Imagine we have a simple application that fetches a list of stories, and some details about each one. Here’s how that might look in resource-oriented REST:

// Fetch the list of story IDs but not their details: rest.get('/stories').then(stories => // This ...

March 26th, 2016

# Picking React isn’t a technology decision, it’s a business decision

Interesting:

The clear winner right now, is React. Components all the way down makes your application much easier to reason about. The learning curve is very flat. The important APIs would fit on one page. JSX is awesome. You get all the power of JavaScript and its tooling when writing your markup. It is the natural match for Flux and Redux (more on that later). The React community is amazing, and produced many best of breed tools such as Redux (also more on that later). Writing high ...

March 26th, 2016

# Pay discrimination in tech

Women who write the software that runs on mainframe computers earn on average 72 cents per dollar earned by their male counterparts, according to research conducted by Glassdoor Inc., the online job information firm. That pay gap exists even after controlling for age, education, experience, job title, employer and location.

This suggests that the gap widens with age:

However, not all tech jobs pay men and women so differently. Among mobile developers, there is just a 2.9% gap between ...

March 26th, 2016

Everyone is laughing over this. An 11 line function as its own NPM module? I am unsure how I feel. The trend has been to smaller and smaller pieces of functionality. I don’t find this outrageous. Still, the parody is funny:

c4n4rd is game:

This is really exciting!!! I was a bit disappointed that the right-pad will be out only in 2017. I am looking forward to that release because there is a high demand for it now.

What kind of ...

March 26th, 2016

# null

Testing WordPress to see if a title of “Null” works.

Source

March 18th, 2016

# The end of the test-first mania

It worries me that the tech industry is driven by such intense manias, which have cost me several jobs. For a long time you were not allowed to question Object Oriented Programming. I still go to job interviews and I’m asked the basics of Object Oriented Programming, and it is clear I have to say nice things about Object Oriented Programming or I won’t get the job. Thankfully none of these people ever check Wikipedia, where I am cited as ...

March 17th, 2016

# Onyx tested by Jespen is genius tested by genius

Good lord, this makes me happy. I think Michael Drogalis is a very smart guy and everyone knows that Kyle Kingsbury is a very, very smart guy. Kingsbury’s work on Jespen is the finest work that anyone has ever done on the problems of distributed data. Onyx is exciting as an a Clojure answer to Apache Storm. Sad to say, Storm is written in Scala. The idea of testing Onxy with Jespen is one of the finest ideas ...

March 14th, 2016

# Unix processes have gotten bigger and slower over the years

Interesting:

The obvious question thus is: Why state machines? Why not processes or threads?

And the obvious answer is: Performance.

When UNIX was still young, scheduling was supposed to be done by the OS on per-process basis. When implementing a network server, for example, you were supposed to fork a new instance of the process for each TCP connection and rely on the OS scheduler to switch between the processes.

I guess it made sense from performance point of view back then. All ...

March 10th, 2016

# Jakub Holý on copying :pre and :post conditions

Obviously I have read this article before (I offered response and was quoted before) but I’ve gone back to read it again and this jumped out at me:

Do you repeat the same checks again and again? Then you could either copy them using with-meta (they end-up in metadata anyway) or reuse the explicitly:

(defn with-valid-car [f] (fn [car] {:pre [:make :model :year]} (f car))) (def count-price (with-valid-car (fn [car] (do-something car)))) ;; or make & use a macro to make it nicer

That ...

February 15th, 2016

# The lack of software leadership is the driving force of verbose ceremonies and placating rituals

Good code communicates the essence of its task

Good code does not hide its task behind a slab of boilerplate code whose only reason for existing is to satisfy some complex syntax requirements of the language. Stuart Halloway said this beautifully:

Good code is the opposite of legacy code: it captures and communicates essence, while omitting ceremony (irrelevant detail). Capturing and communicating essence is hard; most of the code I have ever read fails to do this at even a basic ...

February 13th, 2016

# Immutability changes everything

If you work in a language where everything happens on one thread, in one process, then you don’t really need immutability. But most complex situations require some kind of concurrency. Either you have one instance of one app, but that app is multi-threaded, or you have a simple app with only one thread, but you run multiple instances of the app — for instance, a simple Python app that pulls a message off a queue and processes it; you might ...

February 13th, 2016

# Functional programming is not the same as static data-type checking

Duck-typing versus Label-typing

If it waddles like a duck, and it quacks like a duck, and it swims like a duck, is it a duck?

Suppose you get a box that has the label “Hammers”. Do you believe the contents are hammers?

Figuring out a duck is a duck can take some effort. After all, you have to check for waddling and quacking and swimming. That’s not too bad, really, but what if you had to check for 20 different features? Or 30? ...

February 10th, 2016

# Interlude

I am mostly done writing the second half of this series, however, it needs to be edited, some code needs to be written, all of the code needs to be checked, and I need to work with Leah McCloskey to develop further cartoons for illustrating the points made in this series. Also, I have a full time job. So it will likely be 2 or 3 months before I can publish the second half of this series.

The series is ...

February 10th, 2016

# Get rid of all Dependency Injection

This is part 6 of a 12 part series:

1.) Quincy’s Restaurant, a parable about concurrency

2.) Why I hate all articles about design patterns

3.) Clojure has mutable state

4.) Immutability changes everything

5.) Mutable iterators are the work of the Devil

6.) Get rid of all Dependency Injection

7.) Sick politics is the driving force of useless ceremony

8.) Functional programming is not the same as static data-type checking

Interlude

9.) Inheritance has nothing to do with objects

10.) Is there a syntax for immutability?

11.) Immutability enables concurrency

12.) Quincy’s ...

February 10th, 2016

# Mutable iterators are the work of the Devil

The assignment from Hell

Assume you die, and then you wake up in a beautiful place, floating on a cloud, and you hear a trumpet in the distance, and you hear beautiful singing. Then a voice from up on high says to you, “I would like you to count all of the positive integers. I will give you all eternity to do it.” You assume you are now in heaven, and God is giving you your first heaven-based assignment. Can you ...

February 10th, 2016

# Clojure has mutable state

Warning: This post is an intro to mutable state in Clojure. If you already know Clojure, you can skip this. If you have no interest in learning Clojure, you can skip this.

This is part 3 of a 12 part series:

1.) Quincy’s Restaurant, a parable about concurrency

2.) Why I hate all articles about design patterns

3.) Clojure has mutable state

4.) Immutability changes everything

5.) Mutable iterators are the work of the Devil

6.) Get rid of all Dependency Injection

7.) Sick politics is the ...

February 10th, 2016

# Why I hate all articles about design patterns

Sometimes play is a lie

Imagine two children, both nine years old, decide to play tea. They rummage around in the closets of their respective parents. They dress up as they’ve seen their parents dress up. The one goes over to the other’s house.

“Mr. Worthington, it is so nice to see you,” says the girl who is acting as host. “Would you like some tea?”

“Why, yes, Mrs. Kentworth, I would be delighted to have some tea.”

The host brings an empty cup ...

February 10th, 2016

# A parable about concurrency (demonstrated with comical cartoons)

(Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!)

This is part 1 of a ...

February 9th, 2016

# Why does Erlang allow so many processes to crash and how does promote reliability?

Why does Erlang encourage crashes?

Back-burning and controlled burns are a real world example of fighting fire with fire. In Saguenay–Lac-Saint-Jean, the region I come from, blueberry fields are routinely burnt down in a controlled manner to help encourage and renew their growth. To prevent forest fires, it is fairly frequent to see unhealthy parts of a forest cleaned up with fire, so that it can be done under proper supervision and control. The main objective there is to remove ...

February 6th, 2016

# Hindley-Milner and the lambda calculus

A nice attempt to explain the crazy syntax of the math that formalizes the idea that the type of an expression can be deduced from the expression itself:

Okay, so we want to talk about expressions. Arbitrary expressions. In an arbitrary language. And we want to talk about inferring types of these expressions. And we want to figure out rules for how we can infer types. And then we’re going to want to make an algorithm that uses these rules to ...

February 3rd, 2016

# Beware of Ruby libraries that generate way too many objects

This is several years old and Ruby garbage collection has gotten better, but still, the point about certain libraries being excessive remains valid.

Be aware that you are allocating objects, for instance something as simple as 100.times{ ‘foo’ } allocates 100 string objects (strings are mutable and therefore each version requires its own memory allocation).

Make sure to evaluate the libraries you use, for instance switching a Sinatra XML rendering action from Builder to Nokogiri XML Builder saved us about ...

February 3rd, 2016

# Pat Shaughnessy dissects how much work Ruby has to do to give you a string

Easy is difficult, and this is a great look at how much work Ruby has to do so that you, the software developer, can change your mind about what kind of string you want:

The standard and most common way for Ruby to save string data is in the “heap.” The heap is a core concept of the C language: it’s a large pool of memory that C programmers can allocate from and use via a call to the malloc ...

February 3rd, 2016

# Software development is a complex system of multiple poorly understood feedback loops and interactions

I wouldn’t use exactly the same words that this article uses, but I agree with the gist of this part, especially where small, fast-moving startups are involved :

Finely grained management of software developers is compelling to a business. Any organization craves control. We want to know what we are getting in return for those expensive developer salaries. We want to be able to accurately estimate the time taken to deliver a system in order to do an effective cost-benefit analysis ...

February 2nd, 2016

# Leonardo Borges’s Imminent for futures and promises

Very interesting:

For the impatient, I’ve included a couple of examples below. I’ve chosen to translate the examples presented by Ben Christensen – of RxJava – in this gist. Albeit them being in Java, they highlight perfectly the problem with blocking futures. Here’s their Clojure equivalent:

;; ;; Example 1, 2 & 3 are handled by the approach below ;; Original examples: https://gist.github.com/benjchristensen/4671081#file-futuresb-java-L13 ;;

(defn example-1 [] (let [f1 (remote-service-a) f2 ...

February 2nd, 2016

# Why the tech community rejected XML

Interesting:

Java was a very limited language, and it was extremely verbose. There were type declarations and coersions everywhere. Almost all looping was managed with for loops (there were no higher order functions, and recursion wasn’t tail cail optimized… also there was no loop-recur macro…).

So XML wasn’t just there to be data, it was also code.

Everyone wrote little mini-languages into XML, because some things were so damn painful to express in Java that it was easier to just implement ...

February 2nd, 2016

# Clojure community problems

Interesting:

What an amazing language. Relatively frequent, consistently stable releases. A pleasure to use. A friendly, smart community. I feel very lucky to be a Clojure user! Thank you for all of your hard work.

The Clojure contrib process frustrates me more than any technical or community aspect of the language.

Clojure gets a lot right, but as has been repeatedly discussed the pace of evolution and the maintainership’s dim view of 3rd party non-bugfix work flatly leads to worthy but minor ...

January 31st, 2016

# We want loose-coupling and high cohesion

This seems like a great rule of thumb for microservices:

“We want to avoid dumb, anemic services that are little more than CRUD wrappers”

But it doesn’t cover the old territory which, if you were using Ruby or PHP, you would cover with a cron script. I suppose all the cron scripts must become functions that live inside the “service” which deals with a given part of the datastore. But that is not how my friends talk about “microservices”.

Page 58, ...

January 31st, 2016

# A message hospital (or dead letter queue), where messages got sent if they failed

I like this book very much. I also like the idea of a “message hospital”.

Page 57, Building Microservices, Sam Newman:

Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a ...

Check out my books:

August 18, 2019 8:57 pm

"You seem to have little patience for people who choose different tech paths than you. Although it looks like o..."

August 18, 2019 8:53 pm

"You don't suffer slights well, do you? Others who choose to waste time with dumb tech, do they keep you up at ..."

August 18, 2019 8:34 pm

"Me again. I've worked for a company that focuses on containerized applications for some time now. There is abs..."

August 18, 2019 8:00 pm

"To build on my last statement, I'm not trying to show that I'm "smarter." I'm probably not, or if I am, who gi..."

August 18, 2019 7:48 pm

"You think that containerization is going anywhere? I agree that it isn't strictly necessary, but you mistake y..."

August 18, 2019 7:40 pm

"If you weren't criticizing Docker, why did your frame the article as anti the Docker/Kubernetes/Python troika?..."

August 13, 2019 1:34 pm

"No tool is every perfect, and you're certainly free to use whatever you want. But, I doubt very much the huge ..."

August 5, 2019 5:23 pm

"DangerNorm, as to your last point, I was recently consulting with a startup that focused on the privacy of med..."

August 5, 2019 3:55 pm

"This is tangential to the above point, but on the subject of mental health, I'll also take this chance to say ..."

August 5, 2019 3:51 pm

"Your description of most people's response is correct, but I consider it to be a problem, rather than the corr..."

August 5, 2019 3:28 pm

"DangerNorm, thank you for writing. I think you are missing the moral element here. You should understand the d..."

August 5, 2019 3:07 pm

"It is true that a murderer is different from the flu virus in that murderers are moral agents and the flu viru..."

August 5, 2019 1:19 pm

"ruurd, I'm not sure if you're responding to me or to DangerNorm, but this is true: "If you don’t get it the..."

August 5, 2019 1:09 pm

"Oh yeah that argument. If you don't get it there is something wrong with you. Yeah right. I'll tell you what's..."

August 4, 2019 9:43 pm

""Mass shootings are just not enough of a problem to be worth the attention given to it", is in fact an importa..."