April 12th, 2018

# Excessive abstraction in PHP frameworks

I’m working with Laravel now, a PHP framework. I see this on StackOverflow:

With Laravel, everything is more complex than it needs to be. Yesterday I was dealing with the issue of logging and errors. How to get Laravel to output errors somewhere that I could see them? Well, that involves overriding some logging classes. The tone of the documentation is always “See, it’s very simple, and flexible, just override the default classes, inherit from the parent, override the method, choose ...

April 9th, 2018

# Do the staff at Jezebel manually manage the trolls in the comments?

Apparently so, this is interesting.

Someone named Jinni writes:

Our Muqaddimah of Code 45* is still banned, alas. A bright point in my own days feels notably dimmed. For those of you who didn’t read (or quite ‘get’) Code 45*; it was a hallucinatory tour into the subconscious of Donald Trump as evidenced by the odd and illogical punctuation of his disordered tweets. Code 45*, like a creative, and slightly mad psychotherapist, attempted to wrest vestiges of meaning from the morass of ...

April 8th, 2018

# PHP code for Apple Push Notification Service

I just lost a week of my life trying to send a message from my server to the APNS servers. A previous team had given us some bad PHP code which I could not get working. I finally tore it out and replaced it with the example I found on StackOverflow:

/* We are using the sandbox version of the APNS for development. For production environments, ...

April 8th, 2018

# A pem tool to generate the pem file for Apple Push Notification Service

Having lost the last week of my life trying get a pem file that will let me talk to APNS, I will link to this, because it seems to be a solution for the future:

Automatically generate and renew your push notification profiles

Tired of manually creating and maintaining your push notification profiles for your iOS apps? Tired of generating a pem file for your server?

pem does all that for you, just by simply running pem.

Source

April 6th, 2018

# Why don’t any articles about Apple Push Notifications agree about what commands to use?

Apparently there are a lot of ways to do this? I am suspicious, because I lost days trying to get this to work. This article says:

Now we need to encode these two files in Base64:

openssl base64 -in cert.pem -out cert.txt

openssl base64 -in key.pem -out key.txt

But this gist says:

Step 1: Create Certificate .pem from Certificate .p12

Command: openssl pkcs12 -clcerts -nokeys -out apns-dev-cert.pem -in apns-dev-cert.p12

Step 2: Create Key .pem from Key .p12

Command : openssl pkcs12 -nocerts -out apns-dev-key.pem -in apns-dev-key.p12

None ...

April 5th, 2018

# Apple Push Notifications converting p12 to pem files

None of the commands listed on this page worked for me, and yet that is the topped ranked page in Google. What did work for me:

openssl pkcs12 -in apns_certificates.p12 -out apns_certificates.pem -nodes openssl pkcs12 -in apns_key.p12 -out apns_key.pem -nodes chmod 0400 apns_key.pem chmod 0400 apns_certificates.pem openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert apns_certificates.pem -key apns_key.pem cat apns_certificates.pem apns_key.pem > Peeroapp.pem Source

April 3rd, 2018

# The difficulties of starting a career in tech

This is a good summary of some issues with starting a career in tech. This one jumps out at me, though I realize the issue of portfolios is controversial in some quarters.

More than a couple of times have I met developers looking for web development gigs without a proper portfolio to show off.

Having a portfolio helps to demonstrate what you are capable of doing.

You’ve probably been told that having a personal domain puts you a mile ahead in your ...

March 27th, 2018

# I just lost a week of my life dealing with one bug in a Cordova/Ionic mobile app

Over the last week I had two marathon days where I worked more than 12 hours, and I had 4 normal days where I worked 6 to 8 hours, and all of it was wasted on one bug.

I’ve a friend who wanted an app built that would work for both iOS and Android. He contracted with a team in India to build it. They used Cordova and Ionic.

When they were done with the project they sent it to ...

March 25th, 2018

# Apple takes two weeks to update a forgotten password?

I am amazed by this. On Friday I tried to log into:

https://developer.apple.com

None of my regular passwords worked. Only later did I realize I’d probably set up a separate account for this, using some unusual password. Anyway, I figured this would not be a big deal. I clicked “I forgot password”. I put in my email and then my phone number.

Apple then sent a text message to my phone, with a code that I was to type into the website. ...

March 22nd, 2018

# Even a basic iOS app generates a huge number of errors in Xcode

I’ve decided I should learn how to build native iOS apps. So I am following a basic tutorial for Xcode and iOS.

As soon as I’ve created the project, I try to run it. It does compile and run, but the log is full of hundreds of errors. I’m putting a small sampling below.

Does Apple realize how confusing this is for someone who is just beginning?

I learned how to program back in 1999, writing PHP scripts. One great ...

March 19th, 2018

# CSS is terrible because it is global state

CSS is broken because the style sheets are global. Once upon a time, this didn’t seem terrible because it was thought that CSS would operate as CONSTANTS. But they are not constants. On any modern website or app, the styles are always changing. So the styles are really a global variable, and they have all the problems that 60 years of computer science has taught us to expect from global variables.

There are other problems with CSS. Perhaps the biggest ...

March 19th, 2018

# Online education in the 1960s

Interesting:

As Dear tells it, students were far more interested in using PLATO to chat with one another than to complete their lessons. They even created their own emoticons by taking advantage of a feature within PLATO’s operating system that allowed them to stack letters on top of each other instead of typing them side by side. Typing the word “WOBTAX” in a stack produced a smiley face.

“This was a great pastime of many undergraduates, many of whom would flunk out ...

March 10th, 2018

# The lack of volume in a high-dimensional cube

I wish this felt more intuitive to me:

This is a starting point for intuition:

To provide some intuition consider the situation in two dimensions, as shown in Figure 10. For a point on the circle to be close to the equator, its y-coordinate must be small.

So, this is sort of an expression of the Curse Of Dimensionality?

But perhaps the most amazing thing is that we can not figure out how many spheres might surround a central sphere, in such a ...

March 8th, 2018

# In Apache you can setup a reverse proxy with either RewriteRule or ProxyPass

With Apache I have always done something like this:

ProxyPass / http://127.0.0.1:8080/ ProxyPassReverse / http://127.0.0.1:8080/

But I just saw this interesting bit on StackOverflow:

RewriteEngine On RewriteRule ^(/.*)$http://app-server:8181/jellyfish$1 [P] ProxyPassReverse / http://app-server:8181/jellyfish/

Which does the same thing. The RewriteRule says “grab everything after the ...

February 12th, 2018

# Containers like Docker will not fix the sick politics of your company

Interesting:

Deciding what’s yours and what’s theirs is the basis of every sibling-rivalry détente. In Eric Brewer’s CAP theorem you can pick two of consistency, availability, and partition tolerance as long as one of them is partition tolerance, because, as distributed systems expert Caitie McCaffrey puts it, “physics and math.” In a distributed system that contains humans in multiple time zones, you’re inevitably going to have partitions, and waiting 10 hours for headquarters to wake up and make a decision is ...

February 3rd, 2018

# When things go wrong on a software project

I like this:

Whenever things are not progressing as expected, it is a natural instinct to fall back to micromanagement, but it seldom truly cures the ill. It’s probable that your team disagrees with your vision or its feasibility – and that you’re either not listening to their feedback, or they don’t think you’d care. It’s good to assume that most of your employees are as smart or smarter than you; barking your orders at them more loudly or more frequently ...

February 2nd, 2018

# A very long debate about when to use a library and when to write your own code

The whole thing is interesting, but Jens Stimpfle’s comment comes close to representing my view:

“I see the value in learning by building yourself, but from a software engineering point of view, using a tried and tested framework is likely to give you higher quality product in less time.”

Well, the framework I was to use last (Qt) had bugs that simply can’t be fixed by users (memory leaks, double free leading to segfault when exiting after reloading QML engine) and immature ...

January 18th, 2018

# The continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers

Interesting:

The conclusion seems inescapable that with at least certain kinds of large programs, the continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers who are closely and continuously connected with them.

Source

January 17th, 2018

# Chris Zheng is angry at the Clojure community

This initially sounds like it is going to be interesting:

I say it with much admiration and respect to all the members of community. I’ve learnt so much from being in this microcosm of dynamism, ideas and learning over the past 8 years. Clojure has allowed me to get to know so many amazing people, to travel to a whole bunch of places and to do things that I had never thought possible.

Having said that, I’ve gone from a wide-eyed, ...

January 16th, 2018

# What is up with Narrative.io?

When I go to the website of Narrative.io all I see is black. Is this because I have an ad blocker installed? If so, this is the most dramatic ad-block caused bug that I’ve seen. Something like 20% of everyone on the Web is now using ad-blocking. Narrative.io should keep that in mind when they design their site.

Source

January 6th, 2018

# Yegor Bugayenko: 5 things to improve Java

Yegor Bugayenko offers suggestions for making Java better:

Global Variables. There are Singletons in Java, which, as we all know, are nothing but global variables. Wouldn’t it be great to enable global variables in Java and get rid of Singletons. PHP, JavaScript, Ruby and many other languages have them, why doesn’t Java?

Global Functions and Namespaces To group static methods together we create utility classes, where we have to define private constructors to prevent their instantiation. Also, we have to remember which ...

January 5th, 2018

# How should your company handle data safety and version control?

I’ve been writing software professionally since 1999. The first few years, I didn’t work at a company that used version control. Colin Steele mentioned Subversion to me in 2005. I used it till 2011. Since 2011, every company I’ve worked at has used Git.

From 2005 to 2011 we used Subversion to keep track of all projects. This was at both startups that I helped run, and also while doing most client work. During these years, I only had one ...

January 5th, 2018

# The problem with HTML

(originally from a longer essay)

Just to offer some perspective about where I’m coming from, I’ll start by saying that I think of these as tools that made it easy for beginners to be productive:

Hypercard

Visual Basic (obviously I mean the classic versions, before .NET. The stuff in the 1990s was genius)

(Of these links, I especially recommend the one about Hypercard.)

I spent 1995 learning HTML and putting together simple websites. I was thinking this is something I’d like to ...

December 26th, 2017

# Jakub Holý on the awesome productivity of Clojurescript

I have found the REPL useful when working with Clojure. I plan to write about this soon. Jakub Holý has a great write-up regarding both Clojure and Clojurescript:

Aside – about us: Telia Norge is Norway’s second largest mobile operator. Our team has spent the past three years with Node.js, React, and (eventually) Redux creating a great webshop (for hardware and subscriptions) to replace an off-the-shelf one, with a great success. Now we set out to make also the lives ...

December 25th, 2017

# Jakub Holý: how to build a shopping cart

Interesting:

However there are other concerns than simplicity and time-to-production that we could have been taken into account, I believe. But I failed to communicate those in time. The current solution would have been perfect if the network transfer times were negligible and if the backend service was 100% reliable. None of that is true. Every call takes at least 300ms and the backend can be overloaded or (as yesterday) unavailable due to network issues. I there is a high ...

December 25th, 2017

# Jakub Holý: don’t add unnecessary checks to your code

This is a very good point:

Imagine you need to call .indexOf on product_code. You might be tempted to add

if (!_.isString(product_code)) return false;

Now you are safe and your code can’t fail. But when I read this code, I immediately think:

Oh, I believed product_code to be always a string but obviously it is not always the case. It must have happened in the past or the author wouldn’t have added this check. I wonder what are the possible non-string values. And what ...

December 18th, 2017

# The penalty of saying something controversial on HackerNews

I made a comment on HackerNews and it was heavily downvoted. I made a good point about the limitations of Git, but people did not want to hear it. This raises an interesting issue about voting systems, in general, is there any way to find stuff that is true, reliably, when the voters might have incorrect beliefs? I think Slashdot went furthest, they took voting systems seriously, and they tried to weight the votes of those who were themselves upvoted. ...

December 12th, 2017

# The Krubner Law Of Ugly Code

The Krubner Law Of Ugly Code states that any computer programming language that has a “return” keyword will tend to be ugly. “Tend” means the tendency is there, though of course, as I’m sure hundreds will rush to remind me, you can write beautiful code in any language.

Here is a simple example from a well-respected project. Here the return is used in a guard clause. This is good code, for the Java language, in that it puts the guard ...

December 12th, 2017

# To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice?

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

http://www.artima.com/intv/interop.html

Bill Venners: To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice? If everyone fully understands and adheres to the contract of the functions of a particular DLL, shouldn’t updating that DLL in theory not break any code?

Anders Hejlsberg: ...

December 12th, 2017

# Camille Fournier (Rent the Runway) – Consensus Systems for the Skeptical Architect

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

ZooKeeper owns your availability. Worth watching. I thought it was amazing when she said that even 5 years ago, even at Google, most programmers were not thinking in terms of distributed computing. Instead, its been the boom in microservices that has driven the adoption of ...

December 12th, 2017

# Sam Newman’s book Building Microservices

I’m posting here some quotes I like from this book:

Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a grid to create a pool of pricing workers, allowing ...

December 12th, 2017

# Why I prefer dynamic-typing over static-typing: the speed of adapting to change

I am a fan of dynamic typing. So is corporate America. The widespread use of PHP, Ruby, Python and Javascript suggest that dynamic typing is useful. Static typing tends to be used in areas where legal regulations create needs that outweigh programmer productivity. If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

In How ignorant am I, and how do I formally specify that in ...

December 11th, 2017

# Watching the AI bots on a game fight each other is like testing a data analysis model

I was researching something else and I followed a link to the game Wesnoth. I downloaded it and spent some time testing it. The game is entirely built by volunteers. The game is slow and somewhat tedious, but they did a great job with the art and they did a great job enabling a variety of scenarios.

They claim they put some time into the AI engine that lets bots play a character. I was curious about that. I set ...

November 25th, 2017

# That’s supposed to be an umlaut in the title

This sounds like a very serious criticism of WordPress:

Yes, I know that’s supposed to be an umlaut in the title. I just can’t persuade WordPress to do it.

As far as I know, WordPress has handled UTF8 for a long time. Does anyone know what is going on?

Source

November 25th, 2017

# VisualBasic could have saved Microsoft Excel

Companies are rebelling against Excel:

Finance chiefs say the ubiquitous spreadsheet software that revolutionized accounting in the 1980s hasn’t kept up with the demands of contemporary corporate finance units. Errors can bloom because data in Excel is separated from other systems and isn’t automatically updated.

Older versions of Excel don’t allow multiple users to work together in one document, hampering collaboration. There is also a limit to how much data can be pulled into a single document, which can slow down analysis.

“Excel ...

November 2nd, 2017

# Denormalization to diverse apps is the correct path forward for media companies

I am very pleased to see that the New York Times now has this cutting edge architecture, which almost certainly the right way forward at this time:

The denormalized log and Kafka’s Streams API

The Monolog is great for consumers that want a normalized view of the data. For some consumers that is not the case. For instance, in order to index data in Elasticsearch you need a denormalized view of the data, since Elasticsearch does not support many-to-many relationships between ...

October 31st, 2017

# Matthew Vagnoni suggests the blockchain can resolve the issue of patient consent

Back in 2015, when I worked at Rollio, we hired Matthew Vagnoni for 2 weeks to come and teach us about Natural Language Processing. Vagnoni is a fantastic person to work with: he is a true expert in his field, yet he manifests very little ego about his extraordinary skill. He was friendly and calm and also authoritative, as he walked us through the various strategies we might take, when connecting to Salesforce, using NLP.

I only recently realized that ...

October 31st, 2017

# XLSX2CSV in Java or Clojure

This page does not seem to rank very high on Google, which makes it difficult to find a good example of how to use Java to convert an Excel file to CSV. Last year, when I worked on this, I recall it took me many days of searching before I found this example.

Also good:

http://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/poi/ss/examples/ToCSV.java

Source

October 27th, 2017

# A developer asks for help, then gets angry at those who help them

This is pretty wild. Some idiot named “Amo” asks why Supervisord is using 100% of the CPU. A fellow named Paul Calabro then gives what sounds like the correct answer. Instead of saying “Thank you” Amo then threatens to downvote Calabro. Either Amo is vicious or they completely misunderstood the answer. Either way, Amo is an idiot.

If I was Amo, I would delete the question, and thus make it private, because the whole thing is embarrassing. In case ...

October 25th, 2017

# Why would anyone choose Docker over fat binaries?

[ [ In this essay, I use the phrase "fat binary" to refer to a binary that has included all of its dependencies. I am not using it to refer to the whole 32 bit versus 64 bit transition. ] ]

They are blinded by love:

What I like most about Docker is how responsive and quick it is, and the instant repeatability it offers. Spinning up a new container takes literally the same time it takes to run the command ...

October 24th, 2017

# You can set environment variables from the Supervisord config

I often tell clients to consolidate their config files, from all their various apps, inside of a folder inside of /etc (I assume Linux servers).

But an interesting alternative is to set the config in the Supervisord config, and thus consolidate all logging and config info in one place.

[program:django] environment = SITE=domain1, DJANGO_SETTINGS_MODULE=foo.settings.local, DB_USER=foo, DB_PASS=bar command = python manage.py command

One could simply have different entries for development, certain branches, ...

October 24th, 2017

# Infinite scroll is a terrible idea 99% of the time

Pinterest introduced the modern infinite scroll, so women would keep scrolling their home page. For a while, Pinterest was the fastest growing large site in the world. Infinite scroll is useful if you’re simply encouraging people to browse over random items. It is a terrible design idea if you are trying to communicate information. As Edward Tufte says, long lists are fantastic if you are trying to communicate choices or many types of information.

So check out the Web ...

October 15th, 2017

# The resulting passive-aggression kills productivity faster than any methodology or technology decision

These are the truest 6 paragraphs that I’ve ever read about software development ideologies:

Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results.

In his 2003 thesis People and methodologies in ...

October 15th, 2017

# What are the benefits of the fictions that guide software development?

Very interesting and well written:

I’m not going to beat up on any of these paradigms, because what’s the point? If software methodologies didn’t exist we’d have to invent them, because how else would we work together effectively? You need these fictions in order to function at scale. It’s no coincidence that the Agile paradigm has such a quasi-religious hold over a workforce that is immensely fluid and mobile. (If you want to know what I really think about software ...

October 11th, 2017

# NLP challenges: word ordering reverses meaning

Another good example:

When words are used in conjunction with each other, they express an idea that is bigger or more meaningful than each word in isolation. The two clauses I’m not happy I’m working and I’m happy I’m not working contain the sames words, in close proximity, but have quite different meanings.

Source

October 5th, 2017

# AWS does not protect you from devops

Once upon a time, people made fun of Linux GUIs because they would give you a graphical way to edit a configuration file, but you still had to know everything in the configuration file, and what the settings did. People made the reasonable point that slapping a pretty graphical interface on a configuration file did not make it any easier to work with Linux.

Nowadays, the clients I interact with say things like “We want to use AWS because then ...

September 20th, 2017

# Where should a spec be defined relative to a function

This is an interesting conversation on the Clojure mailist, and this question is relevant for more than just Clojure. It touches on the main question of language design.

Didier feels it should be possible to define the spec with the function, and this functionality should be added to the Clojure core, so that this would become idiomatic Clojure:

Gary Trakhman argues that this would make the core “defn” macro too slow, and therefore everything about spec should be left to ...

September 18th, 2017

# How to use Zach Tellman’s Clojure library Manifold

These examples are great, and I wish this was more highly ranked in Google, so I’m linking to it.

Source

September 11th, 2017

# Niche facts that Google can’t find

Google works so well that I’m shocked when it doesn’t work. I recall reading an early RFC, from the early 1970s, when someone proposed allowing variable length bytes. I tried searching Google for this. No luck. Google only shows me the famous RFCs, not the failed experiments.

Source

September 11th, 2017

# Why can’t modern tools match Smalltalk?

I often have the impression that Smalltalk in the late 80s and early 90s hit a peak that has yet to be matched. Great effort was instead put toward languages that are less intelligent, and which limit the computer programmer. Does this have something to do with the deskilling of computer programming, as Stanislav says?

“It amazes me just how blindly complacent programmers have been in the face of the ongoing and very successful deskilling of their profession. ...

September 8th, 2017

# The limitations of decision trees

This article has some nice setups with the visuals:

If you are going to buy a house, you will make a list of pros/cons. That much is common sense. It is also an implicit decision tree. Most of these concepts are common sense. But I am struck, yet again, and how quickly this can seem to be unimaginably complex, once you involve the abstract jargon of math:

And once again, I note the sheer awfulness of math notation. How much faster ...

September 8th, 2017

# Gini Impurity is quick to measure and easy to adapt to

I’m trying to convince a client that we need to match records using a Random Forest approach. They have hit the limits of what can be done with simple string matching. I like basic articles like this, for explaining things clearly, and making it obvious how easy Random Forests can be (especially compared to Neural Nets).

Which is the better split? This is a subjective question. In practice, people use different metrics for evaluating splits. The most commonly used metric ...

September 8th, 2017

# Logs are the future of software

Not sure why it has taken so many decades to become obvious, but clearly a history of all events is better than a relational database for keeping track of data. There is no reason why any company should ever use a relational SQL database. The source of truth should be the log, and one’s consumers should build their own denormalized datasets — and these will often work best in a document store, rather than a relational database.

I am very ...

September 7th, 2017

# Peter Williams on Humane Interfaces

Still caught by the nostalgia I mentioned in the 2nd previous post, I re-read Peter Williams post, linked via the same debate. I recall reading this in 2005.

Martin Fowler has posted a nice article on humane interface design (as opposed to minimal interface design). I am definitely on the side of right and good (read: humane interfaces) in this debate. Nothing takes the fun out of programming faster than having to write a bit of code that you know ...

September 7th, 2017

# The failure of the X Window system paved the road for the rise of the Web

This is an amazing article about the absolute failure of the X Window system. Copyright (C) 1994, so this was written just as the Web was gaining momentum, but before it was obvious that the Web would become the main way that different devices could establish a single interface. Among the many good bits, there is this:

Figuring out where a particular resource value is for a running application is much fun, as resource can have come from any of ...

September 7th, 2017

# The loneliness of advanced skills

I decided to re-read The departure of the hyper-enthusiasts. I’ve linked to that essay many times before. It was formative for me, at a formative moment for the Web, so it was formative in many ways, and it captured something important about that moment, and how things were changing.

I find myself getting nostalgic. That is not interesting in and of itself. To be nostalgic for a formative time, about a formative essay, is ordinary enough.

But I had an ...

September 7th, 2017

# Blue or Not Blue decision trees

This is a point worth remembering:

“Before we get started I need to clarify something. Theoretical decision trees can have two or more branches protruding from a single node. However, this can be computationally expensive so most implementations of decision trees only allow binary splits.”

Source

September 7th, 2017

# If NodeJS is so great, why are the problems so common?

Callback hell. Take a deep breath and ask yourself why this website needs to exist:

Source

August 12th, 2017

# Supervisord has spent 5 years discussing whether or not it should support a timing mechanism?

I’m not sure what to make of this incredible thread in the issue tracker on Github for the Supervisord repo. Someone asked, in 2012, for a way to time when various apps start, so that they could start app A, B and C in order. On April 16 of this year, 2017, someone wrote:

This issue has been open since 2012, does anyone have ownership over it? It’s clearly important to many users; is there any consensus amongst supervisor contributors?

I ...

August 4th, 2017

# A pure GUI for IP/TCP, a technology with a single mandate, with no burden of also offering semantics or structure or hierarchy

Someone on Hacker News said most developers don’t “get” the Web, and I posted this in response:

My history is surprisingly similar to yours, I started in 1999, I used Notepad as my first text editor, and by 2003 I got caught up in the movement towards making markup strict, which I felt was the mark of professionalism. However, by 2006 I had mostly rejected the notion of “strictness”. There were several things that turned me against strictness. One of ...

August 2nd, 2017

# Cultural differences between Clojure and NodeJS

Right now I have to work with NodeJS, because I have a client who uses NodeJS. I am getting used to the asynchronous style of NodeJS. I notice a subtle cultural difference between Clojure and NodeJS. Most of the tutorials for NodeJS assume the person reading is a beginner, someone who may not understand big complex words like “asynchronous”. In the world of Clojure, there is the assumption that the person reading the article has some experience. With Clojure, there ...

July 24th, 2017

# NodeJS remains single threaded

This is a good example of the limits of NodeJS:

async.parallel([ function(callback){ setTimeout(function(){ callback(null, 'one'); }, 200); }, function(callback){ setTimeout(function(){ callback(null, 'two'); ...

July 19th, 2017

# No one should ever use NodeJS, part MMCVXII

I previously wrote about a very surprising bug I discovered in NodeJS/HapiJS.

But today I face a new issue: How to parallelize work in NodeJs? I discovered this fantastic library called paralleljs. It assigns work to child processes. Fantastic idea, very similar to Python. And how do I get required libs into the child process? That is very difficult. From the issues on Github:

to make things a bit more concrete, this snippet would be representative of what I want ...

July 18th, 2017

# Do people love DigitalOne servers?

I’m looking to host my next web site at DigitalOne. If you have experience with them, please let me know.

Source

July 18th, 2017

# When Nginx becomes the bottleneck

This makes me sad. I love Nginx as a reverse proxy, so long as it is invisible and I never have to think about. Realizing that it, too, can be a source of problems really is discouraging.

47,135 connections in TIME_WAIT! Moreover, ss indicates that they are all closed connections. This suggests the server is burning through a large portion of the available port range, which implies that it is allocating a new port for each connection it’s handling. Tweaking ...

July 18th, 2017

# Trying to explicitly enumerate everything that is true is hopeless

This is good:

Let’s take a look at some existing representations. The most famous representation is WordNet. In WordNet, the symbols are groups of words that have the same meaning, called synsets. One synset could be the set consisting of “car” and “automobile.” Each word can be in multiple synsets. For example, “bank” could be in the synset that means river bank and also in the synset that means a place where money is deposited. There are a few kinds ...

July 8th, 2017

# A simple intro to tokenizing with OpenNLP in Clojure

A nice intro:

Finding sentences

Words (tokens) aren’t the only structures that we’re interested in, however. Another interesting and useful grammatical structure is the sentence. In this recipe, we’ll use a process similar to the one we used in the previous recipe, Tokenizing text, in order to create a function that will pull sentences from a string in the same way that tokenize pulled tokens from a string in the last recipe.

We’ll need to include clojure-opennlp in our project.clj file:

(defproject com.ericrochester/text-data ...

July 8th, 2017

# Neural Networks beat Support Vector Machine beat Random Forests

I’ve written about my experience at a startup that used NLP to translate a salesperson’s text message into an entry in Salesforce. At the time, our NLP developer used the Stanford NLP library to try to build a model, but mostly they used a lot of regex and string matching.

I’ve been thinking about how we might have done that project faster and better. I’ve recently been thinking the right approach would have been Random Forests. We were lucky ...

July 8th, 2017

# The struggle to find meaning in words, via word counts in matrixes

Interesting comment:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is the ...

July 8th, 2017

# The problems with DynamoDB

Everything I’ve read suggests to me that companies are drifting towards Cassandra as the best of the NoSQL databases. But some of my clients give me directives such as “Please restrict yourself to AWS technologies”. So I was considering using DynamoDB. And then I read this.

Interesting:

When a customer does an action, it is reasonable to think that this kicks off a number of actions in your system. One action might spawn a number of events to be sent into ...

June 29th, 2017

# The curse of too many dimensions

Damn, this is interesting:

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic optimization.

There are multiple phenomena referred to by this name in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining, and ...

June 29th, 2017

# Morphism is the starting point of category theory

I guess I knew this, but still it is interesting to read:

In algebra, a homomorphism is a structure-preserving map between two algebraic structures of the same type (such as two groups, two rings, or two vector spaces). The word homomorphism comes from the ancient Greek language: ὁμός (homos) meaning “same” and μορφή (morphe) meaning “form” or “shape”.

Homomorphisms of vector spaces are also called linear maps, and their study is the object of linear algebra.

The concept of homomorphism has been ...

June 29th, 2017

# All the weird characters in Clojure explained

This is a very useful guide.

Source

May 31st, 2017

# Cost cutting at British Airways caused the catastrophic failure of its database service

This is obviously a lie:

BA insisted that the computer system outages that grounded hundreds of flights were not linked to ferocious cost-cutting and the outsourcing of work to contractors in its IT department.

There is absolutely no way to get a melt down of this scale unless you’ve just fired your key staff, so you’ve nobody left who knows how to bring the system back online. I’ve worked at very small startups, with just 3 people on staff, and we ...

May 27th, 2017

# Is Clojure dying?

Interesting.

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure ...

May 9th, 2017

# Don’t use PM2 with NodeJS

Interesting:

I don’t use any of the “node” process managers because they don’t solve the problem of starting your process at server boot. I use init scripts, or upstart, or runit or systemd like you’re supposed to for a service. Anyone using forever or pm2 likely doesn’t understand sysops.

Source

May 5th, 2017

# Very long GC pauses when NodeJS is about to die from memory exhaustion

I sometimes see stuff like this in my logs, when my NodeJS app has reached critical:

856870 ms: Mark-sweep 5647.1 (6029.8) -> 5625.8 (6029.8) MB, 10733.9 / 0.0 ms [allocation failure] [GC in old space requested]. 867774 ms: Mark-sweep 5625.8 (6029.8) -> 5625.8 (6030.8) MB, 10903.4 / 0.0 ms [allocation failure] [GC in old space requested]. 878580 ms: Mark-sweep 5625.8 (6030.8) -> 5625.8 (5996.8) MB, 10806.3 / 0.0 ms [last resort gc]. 889353 ms: Mark-sweep ...

April 26th, 2017

# More NLP for Clojure

This looks fantastic:

To do its magic, postagga extracts the phrase structure of your input, and tries to find how do this structure compare to its many semantic rules and if it finds a match, where in this structure shall he extract meaningful information.

Let’s study a simple example. Look at the next sentence:

“Rafik loves apples” That is our “Natural language input”

First step in understanding this sentence is to extract some structure from it so it is easier to interpret. One common way ...

April 26th, 2017

# TCP can fail in many, many ways

Amazing that the world depends on a technology with so many quirks:

So, if we read that data first, and LINGER, are we good to go? Not really. The close() call really does not convey what we are trying to tell the kernel: please close the connection after sending all the data I submitted through write().

Luckily, the system call shutdown() is available, which tells the kernel exactly this. However, it alone is not enough. When shutdown() returns, we still have no indication ...

April 23rd, 2017

# Thinking about Concurrency, Raymond Hettinger, Python core developer

A basic overview of concurrency, most of which I knew, but he had one good line at the end: “If you add enough locks to your concurrent code, it is no longer concurrent. It is completely sequential, you’ve simply made it a lot more complicated than single threaded code.”

Source

April 13th, 2017

# Python has greenlets

I’m sad that Python is doing so well. It gets all the attention that should go to Clojure. They are both great for AI and NLP, but Python doesn’t have as many great ideas for concurrency. Or rather, the language doesn’t, but the eco-system makes up for that with a wealth of interesting ideas — and it is sad to see so much effort go to see making Python work, rather than just starting over with something that has a ...

April 13th, 2017

# Katharine Jarmul describes the bias in Google’s Word2Vec software

Interesting:

I must warn you that parts of this post are disgusting, disturbing and awful. If you are having a rough day, feel free to save for another time. If you are already sick of seeing hateful language, this is likely not a post to read at present. That said, I feel my duty as a former journalist to look at it, expose it, and hope to spark better conversations around how we handle both implicit and explicit bias and prejudice ...

April 11th, 2017

# Why are people ignoring the problems with NodeJS?

Two weeks ago I wrote “A surprising NodeJS failure mode: deterministic code becomes probabilistic under load“. Since then I’ve been reading up on NodeJS and learning more about its substantial failure modes. I’m left feeling very surprised at the success that NodeJS is having.

Just to give you a sense of what I mean, this is how The New Stack sums up the success of NodeJS:

Ready for a Long Term Node Relationship?

In just seven short years, Node.js has gone ...

March 30th, 2017

# A surprising NodeJS failure mode: deterministic code becomes probabilistic under load

I have a client. Let’s say they run some scripts that scrape the web for data, then they run some NLP scripts to pull out the facts they need, then they need to insert that into their MySQL database. When they gather facts, they need to know if the facts are about an organization that is already in the database. So they asked me to create an API that could take their new facts, and try to match those facts ...

March 29th, 2017

# Skip Lists: A Probabilistic Alternative to Balanced Trees

Interesting:

Skip lists are a data structure that can be used in place of balanced trees. Skip lists use probabilistic balancing rather than strictly enforced balancing and as a result the algorithms for insertion and deletion in skip lists are much simpler and significantly faster than equivalent algorithms for balanced trees

A node that has k forward pointers is called a level k node. If every (2i)th node has a pointer 2i nodes ahead, then levels of nodes are distributed in a simple pattern: 50% are level 1, 25% ...

March 29th, 2017

# Novelty seeking multi agent systems

Interesting:

This paper considers novelty-seeking multi-agent systems as a step towards more efficient generation of creative artifacts. We describe a simple multi-agent architecture where agents have limited resources and exercise self-criticism, veto power and voting to collectively regulate which artifacts are selected to the domain i.e., the cultural storage of the system. To overcome their individual resource limitations, agents have a limited access to the artifacts already in the domain which they can use to guide their search for novel artifacts. Creating geometric images called spirographs as a case study, we show ...

March 29th, 2017

# Bushy join trees for snowstorm queries

I am taken aback by the amount of jargon here:

Many workloads for analytical processing in commercial RDBMSs are dominated by snowstorm queries, which are characterized by references to multiple large fact tables and their associated smaller dimension tables. This paper describes a technique for bushy join tree optimization for snowstorm queries in Oracle database system. This technique generates bushy join trees containing subtrees that produce substantially reduced sets of rows and, therefore, their joins with other subtrees are generally much ...

March 29th, 2017

# How to combine novelty search with fitness-based evolution

I need to go back and read this whole article:

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It is still unclear, however, how some key parameters and algorithmic components influence the evolutionary dynamics and performance of novelty search. In this paper, we conduct a comprehensive empirical study focused on novelty search’s algorithmic components. We ...

March 29th, 2017

# Novelty search

Interesting:

Novelty search is a recent algorithm geared toward exploring search spaces without regard to objectives. When the presence of constraints divides a search space into feasible space and infeasible space, interesting implications arise regarding how novelty search explores such spaces. This paper elaborates on the problem of constrained novelty search and proposes two novelty search algorithms which search within both the feasible and the infeasible space. Inspired by the FI-2pop genetic algorithm, both algorithms maintain and evolve two separate populations, ...

March 27th, 2017

# What every object on a network needs

There is no need to rely on the Web for these things:

The problem with operations is that we have to define, a-priori, the semantics of each individual operation. We cannot tell, unless we know beforehand, whether the operation is safe to call multiple times, whether the result of an operation will be the same every time we call it, whether the result can be cached, and if so for how long. Many years of building distributed applications have told ...

March 27th, 2017

# Use pseudo URIs for an id

A long article, but this is interesting:

When facing the problems above, my team at SoundCloud started exploring alternatives that would allow for us to have simple, scalar values that were still rich enough to act as good identifiers across our hundreds of microservices. Reading through decades of industry work on the matter, we found something simple that could help us: Uniform Resource Names, or URNs. URNs were a type of Uniform Resource Identifiers (URIs) that, as opposed to URLs, were ...

March 27th, 2017

# Is NewSQL the next big thing?

I like the fact that Kinesis has an SQL interface:

I can see this being useful for Business Intelligence. But that seems like a niche to me. Both SQL and NoSQL solved big, universal problems.

But maybe the SQL is just window dressing? Maybe the real breakthrough is a distributed system with strong consistency guarantees? It would be a very big deal if someone found a way around the CAP Theorem. But otherwise, NewSQL is just more bad marketing for ...

March 27th, 2017

# Designing a neural net is really hard

Interesting:

2bitencryption says:

Designing a neural network is a thousand times harder than I imagined.

After AlphaGo, I tasked myself with creating a neural network that would use Q-Learning to play Reversi (aka Othello).

At that point, I had already utilized Q-Learning (the tabular version, not using a neural network) for some very simple and mostly proof-of-concept projects, so I understood how it worked. I read up only perceptrons, relu, the benefits/disadvantages of having more/fewer layers, etc.

Then I actually started on the project, thinking ...

March 26th, 2017

# Training a neural net

A great intro:

Training

Training is how we teach a neural network what we want it to learn. It follows a simple five step process:

1.) Create a training data set, which we will call x and load its labels as targets y

2.) Feed the x data forward through the network with the result being predictions y’

3.) Figure out the “loss” of the network, which is the difference between the predictions y’ and the correct targets y

4.) Compute the “gradient” of the loss ...

March 26th, 2017

# Transform real data into a tensor, that is, tensor flow

A good introduction:

Source

March 17th, 2017

# What if we had computer bits that allowed more than two values?

The Red-Black tree is interesting because it can store the color using only bit, and one bit can have one of two values, often represented as either a 0 or a 1. That is the nature of our silicon hardware. But if we had hardware where the basic charge could be one of several values, then it would possible to do multi colored trees. If we had hardware that could hold one of 20 values, it would be possible to ...

March 17th, 2017

# An InputStream requires more memory as it has to buffer the whole file

This is exactly the opposite of what I’ve always understood, and what I’ve read elsewhere:

When opening a workbook, either a .xls HSSFWorkbook, or a .xlsx XSSFWorkbook, the Workbook can be loaded from either a File or an InputStream. Using a File object allows for lower memory consumption, while an InputStream requires more memory as it has to buffer the whole file.

The stream requires more memory? That is crazy. It’s a stream. And what is a File object if not ...

March 8th, 2017

# Joe Armstrong: computer programming is the alteration of values in a key value database

An incredible assertion that would be dismissed if it was coming from someone less talented than Joe Armstrong. It is interesting to think that RAM is a key value store, and so anything that updates RAM is altering the values of a key value store.

He starts by saying that he often ends up with miscellaneous code:

When I’m programming I often get to the point were I say there shoulda function foo/2 in lists.erl but their isn’t. There should be ...

March 1st, 2017

# How to get Nginx to wait for your app

I’ve been struggling with a slow NodeJS app which is behind a reverse proxy handled by Nginx.

I found this bit useful:

Syntax: proxy_max_temp_file_size size; Default: proxy_max_temp_file_size 1024m; Context: http, server, location When buffering of responses from the proxied server is enabled, and the whole response does not fit into the buffers set by the proxy_buffer_size and proxy_buffers directives, a part of the response can be saved to a temporary file. This directive sets the maximum size of the temporary file. The size of data written to ...

February 28th, 2017

# How do we teach self-driving cars to avoid hitting people on bicycles?

This is a good conversation:

ChuckMcM :

“Now Johnny, when you ride your bike you must wear your I-am-a-bike vest and follow these patterns or the cars are likely to kill you.” :-) Summary is that “people riding bikes” (PRB) is a much denser image set than “people in cars” (PIC) and “people walking or jogging” (PWJ), and the PRB objects have a much higher dynamic angular vector capability (they can change direction extremely quickly) combined with a wider dynamic velocity vector ...

February 14th, 2017

# Different NLP approaches

A nice summary:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is ...

February 11th, 2017

# Tony Arcieri’s very good defense of Rails

I just had a project where I had to write an app that could fire several million requests against an API, with data pulled from a database. I wrote the whole thing in NodeJS and HapiJS, and it was an agony. The uncontrolled async of NodeJS tripped me up. If I need to fire 10 million HTTP requests, how do I do that without blowing the stack? As it was, the functions would pile up, and then I’d get a ...

February 7th, 2017

# systemd is no longer interested in the grubby details of reality

Interesting:

Unfortunately, since then systemd developers have shown an unfortunate and increasing streak of idealism. More and more, systemd seems not to be interested in dealing with the world as it actually is, with all of its grubby inconvenient mess; instead it simply assumes some pure and ideal version of things. If the world does not measure up to how it is supposed to be, well, that is not systemd’s problem. Systemd will do the nominally right thing no matter how ...

January 28th, 2017

# NLP as a simple four-step formula: embed, encode, attend, predict

Interesting:

Let’s say you have a great technique for predicting a class ID from a dense vector of real values. You’re convinced you can solve any problem that has that particular input/output “shape”. Separately, you also have a great technique for predicting a single vector from a vector and a matrix. Now you have the solutions to three problems, not two. If you start with a matrix and a vector, and you need a class ID, you can obviously compose the ...

January 27th, 2017

# Paul Phillips denounces Scala

Pacific Northwest Scala 2013 We’re Doing It All Wrong by Paul Phillips For a computer programming tech conference, this talk has a surprising amount of emotional rawness. Phillips is denouncing Scala both on a personal level and on a technical level.

Source

January 27th, 2017

# One team that gave up on Scala

It’s interesting how Scala was initially seen as “A Better Java” but now is seen as having been to experimental in its approach.

I remember when I first saw the potential issues of scaling Scala at Gravity back in 2009/10ish. It was close to the end of the day when we had a major issue reported in production that was affecting our large customers. Several of us started investigating and were able to track the source of the issue. The ...

January 27th, 2017

# Scale programmers do not trust Martin Odersky

This whole conversation is incredible. Here is a forum full of Scala programmers, and they seem to be talking against Odersky’s proposal. And with some anger. I’ve never seen a Rubyist talk about Matz like that. I’ve never heard a Clojurist talk about Rich Hickey that way. Java programmers are, at most, muted in their criticism of Gosling, with the overall recognition that “mistakes were made” but made in good faith. I suppose you could argue that Pythonists are split ...

January 27th, 2017

# The failure of hierarchy based on category

This is good:

The Hierarchy Problem

Every time I start at a new company, I struggle with the problem when I’m creating a place to put my Company Documents, e.g. the Employee Handbook.

Do I create a folder called Documents and then create a folder called Company in that?

Or do I create a folder called Company and then create a folder called Documents in that?

Both work. But which is right? Which is best?

The idea of Categorical Hierarchies was that there were Base Classes ...

January 27th, 2017

# Logicians hate randomness

My strongest subject was always logic, even though I was terrible at most other forms of math. And though I have doubts about randomness, I certainly love inconsistency, which can be approached in a logical way. Every computer programmer who writes concurrent code has to deal with inconsistent data, and it would be great if logicians thought more deeply about how to formalize this.

With Gödel, at first there was a lot of shock. As a student in the late ...

January 24th, 2017

# Movio.co decides to switch from Scala to Go, and they reference my essay

Since they cite me (and my essay from 2014) as part of their decision making process, I want to throw in 2 cents here. They ended up deciding on Go, whereas I have ended up preferring Clojure, yet I agree with a lot of what they say, so I’ll try to clarify why I ended up with a different decision than what they made.

I understand what they mean when they write:

I think the first time I appreciated the positive ...

January 18th, 2017

# Modern programming depends on DSLs, and Lisp is the best as DSLs

Anders Hovmöller seems to be stuck with a mentality shaped by the C++ language, and so he doesn’t get the important of DSLs.

In C++ land it’s well known that you must keep to a subset of the language in a code base to keep sane. It’s also known that every time you increase the size of the subset you are making the situation worse even if it solves a small problem at hand.

In Lisp land on the other hand, all ...

January 8th, 2017

# This year’s goal: move away from WordPress

Like a lot of people, I first used WordPress because it was quick and easy to get going, and because my designer knew it very well. But I dislike the dependence on MySQL, I dislike its slowness, and I dislike the security problems that I’ve had (this site has been hacked twice).

For my modest goals, I only need a static site, so I plan to switch to Cryogen. I was interested by this article:

For a while I ...

January 8th, 2017

# Different people have different strengths

I love this :

I’ve had similar self-doubt in the past, I considered myself fast but sloppy. I changed my opinion after a workshop on different team roles at my previous job. A lot of it was boring workshop fluff, but I loved the core message: that many personality traits aren’t purely positive or negative. Perfectionists are nitpickers. Fast developers are sloppy. Experienced ones overthink stuff. Bleeding-edge evangelists ruin long-term stability etc. There are two sides to every coin. We tend to ...

January 8th, 2017

# Can event logs, as a software architecture, really work?

This is an interesting bit of criticism of this architecture, an architecture which certainly has gotten a lot of attention over the last 4 years:

I have worked on, or cleaned up, 4 different CQRS/ES projects. They have all failed. Each time the people leading the project and championing the architecture were smart, capable, technically adept folks, but they couldn’t make it work.

There’s more than one flavor of this particular arch, but Event Sourcing in general is simply not very ...

December 15th, 2016

# Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

I posted this question to StackOverflow and I would appreciate any help figuring out what the memory issue is:

Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

Source

December 14th, 2016

# Ugly engineering

This sounds hideous:

I used to work on the DynamoDB team. Throwaway account because my normal account can be tied back to my real name.

“Each hash key resolves to a number of possible servers the data can be on. Data is replicated across several of these servers. For redundancy. The hash key determines which shard to use. On individual machines, each set of data is stored by a compound key of hash key and sort key (if there is a sort ...

December 13th, 2016

# Beautiful engineering

Wow. Sometimes I witness some engineering that leaves me feeling joy and inspiration. I felt that way about the Golden Gate Bridge. But sometimes I feel that way about code. For instance, right now, I just learned that the defun macro for Clojure allows this function for calculating a fibonacci number.

A fibonacci function: (defun fib ([0] 0) ([1] 1) ([n] (+ (fib (- n 1)) (fib (- n 2))))) Output: (fib 10) ;; 55

That ...

December 7th, 2016

# The crazy process of trying to release a Salesforce app

Right now I’m helping a client release an app for Salesforce in the Salesforce App Exchange. The process for releasing an app is completely insane and very poorly documented (or rather, there is an abundance of documentation for various parts of the process, but there is no summary of the process). I was lucky to stumble upon this post, which offers the nearest thing to a summary of the process:

1) Create a Dev Org

2) Become a Salesforce.com Partner

3) Create Test ...

November 26th, 2016

# Clojure for XML

This is a fantastic overview of different approaches:

Zippers are probably the easiest way to manage xml – once you grok them.

Zippers are a strange beast. Wikipedia describes them as:

A technique of representing an aggregate data structure so that it is convenient for writing programs that traverse the structure arbitrarily and update it’s contents…

I like to think of a zipper as a kind of pointer to part of a tree – at any time if you have a tree of nodes ...

November 26th, 2016

# How far can code be optimized

This is funny and interesting:

When you’re competing with another player, they will probably find a way to beat your score by just a few points. Let’s say my score is 340 and a friend beats me with a score of 335. (lower is better. The score is just the number of executed instructions) What follows is a bunch of head-scratching about how you could possibly get any more cycles out of the algorithm. After an hour of staring and trying ...

November 14th, 2016

# NLP with Clojure and OpenNLP

This is a fantastic introduction:

Natural Language Processing (NPL) opens the door to the possibility of turning otherwise inert text into meaningful or, more interestingly, actionable information. It is the latter that I am interested in and what this installment will focus on. I will explore the basics of NLP using the OpenNLP library and Clojure to convert a sentence into a useful structure to store or act on. More specifically, my goal is to take simple sentences that indicate the ...

November 13th, 2016

# The best interview technique: pay people $200 to attempt some task over the weekend I like this idea: In regards to what works the best, I found that these 2 ideas work the best when combined. PAID Sample project assignment (err on the side of paying fairly — say$100+/hour for estimated completion time — if the problem should require 2 hours to complete, offer $200) Bring the candidate in and discuss the solution. Let the candidate talk about their design decisions, challenge them as you would any team member and let them provide their reasoning. Paying candidates to work on a ... Read More Source November 5th, 2016 No Comments # The perfect Javascript file uploader? This looks amazing: The default behavior of Fine Uploader is to immediately attempt to upload files as they are selected. One option allows you to simply queue all files, and then start uploading at a later time by calling uploadStoredFiles() on your Fine Uploader instance. You can also easily allow your users to edit the names of each submitted file before uploading. This Fine Uploader instance below also demonstrates the edit file name feature, which allows you to edit the name of ... Read More Source October 31st, 2016 No Comments # Stripe almost got burned by Bitcoin It’s amazing how many disasters seem to happen whenever companies try to use crypto-currency: For a variety of reasons, it is sometimes necessary to refund bitcoin transactions: For example, a customer cancelling their order; accidentally sending in the wrong number of bitcoins; or even sending in the correct number of bitcoins, but not within the requisite time window, resulting in their value being lower than necessary. Consequently, Stripe allows for bitcoin transactions to be refunded — with the caveat that, for ... Read More Source October 27th, 2016 No Comments # Software doesn’t need options At least for the consumer. An API for a developer is something else completely. Great software is: Opinionated Options are a cop out. It’s the designer’s job to figure out the best way to use the product, and delegating that choice to the user is pure laziness. Hive only resorts to options as a last resort. Source October 3rd, 2016 No Comments # How process becomes bloated and bureaucratic over time This is good: Checkout/Ordering team: “Your Payment Processing release broke Checkout in production.” Payment Processing team: “Didn’t know this would affect Checkout. Had no time to look into it because we were too busy working on the new payment functionality in the iOS app.” Mobile App Product Visionary: “I want my own dedicated team. This happened because Payment Processing is spread too thin to work on the app and the backend at the same time.” Development Manager: “Org chart change – let’s have all ... Read More Source September 18th, 2016 No Comments # Rapidly Exploring Random Trees Interesting. It says the RRT proceeds by breaking the Voronoi spaces into smaller Voronoi spaces. The algorithm has a bias towards unexplored territory, unlike a purely random algorithm, which tends to loop back to explored space. I didn’t dig into the algorithm, but I can imagine how this would be useful in some situations. Source September 18th, 2016 No Comments # As a consolidation client for Kinesis, ZeroMQ is much better than Kafka This is fantastic: With the problem of real-time log consolidation solved with ZeroMQ, supporting durable storage of logs for later post-processing becomes a smaller and a much more manageable challenge. For example, off the shelve solutions like Logstash are capable of capturing data from ZeroMQ and publishing it to a variety of destinations. In our particular situation at Auth0, we are already using AWS Kinesis, ElasticSearch, and Kibana as a log processing pipeline in other parts of our operations. As such we ... Read More Source September 18th, 2016 No Comments # I always learn what I need to know about a year too late At Celolot I wrote a bunch of Clojure apps that talked to each other over Redis thanks to Carmine/Nippy. We passed around maps, and each app checked a field in the map to see if the map was a message that it was suppose to respond to. But I bet this is a thousand times faster: ZeroMQ does not have a first class notion of a topic, yet it does have a first class concept of a subscription filter. Subscription filters ... Read More Source September 18th, 2016 No Comments # Camille Fournier on ephemeral nodes in etcd I missed this the first time. In etcd a service needs to constantly update the fact that it is alive. If it dies, its info disappears. So other services can contact etcd and ask “Who is alive right now?” ZooKeeper allows stateful connections. Thick clients. But what happens when you try to use ZooKeeper from a language that does not allow threads? Do you block on the only thread that you have? For pub-sub she recommends RabbitMQ or ZeroMQ. Source September 18th, 2016 No Comments # Why would I ever want to use Docker? I’m trying to be open minded about Docker, but it is a struggle to grok why this is good. Do I want to bind all dependencies to my app? Great, then I’ll build uberjars that combine everything into 1 binary. But how will I work with Ruby On Rails? That’s easy: it is time to move away from Ruby On Rails. If Ruby can only be kept alive by using Docker, then we should give up on Ruby. How will ... Read More Source September 18th, 2016 No Comments # Unary Numeral System Interesting: Addition and subtraction are particularly simple in the unary system, as they involve little more than string concatenation. The Hamming weight or population count operation that counts the number of nonzero bits in a sequence of binary values may also be interpreted as a conversion from unary to binary numbers. However, multiplication is more cumbersome and has often been used as a test case for the design of Turing machines. Compared to standard positional numeral systems, the unary system is inconvenient ... Read More Source September 18th, 2016 No Comments # It took me 10 years to write that 5 line app There is the story where Picasso paints a quick sketch and wants a huge amount of money for it, and the art collector says “It only took you 15 minutes to make that sketch!” and Picasso says “I spent 60 years learning how to do that in 15 minutes.” Maybe something like this applies when it comes to microservices. When I advocate for microservices, I bring 17 years of experience to the conversation. The first 7 of those years meant ... Read More Source September 18th, 2016 No Comments # For the last 10,000 years, the banking industry has relied on eventual consistency Why do computer programmers feel smug offering the following example, when it is so clearly wrong? Transactions, to a database, are important, because banks must keep track of money. Suppose a person were to move$100 from their Savings account to their Checking account. Suppose $100 is added to checking, and just then the electricity dies, and the computers die, before$100 can be subtracted from their Savings account. The person now has an extra $100, which they should not ... Read More Source September 17th, 2016 No Comments # I keep really good company This article by Yegor Bugayenko cites the following people: Edsger W. Dijkstra Alan Kay Paul Graham Richard Mansfield Eric Raymond Jeff Atwood Linus Torvalds Oscar Nierstrasz Rich Hickey Eric Allman Joe Armstrong Rob Pike John Barker Lawrence Krubner (me) Asaf Shelly I must be doing something right to get cited alongside Edsger W. Dijkstra, Alan Kay, Paul Graham, Linus Torvalds, and Rich Hickey. Source September 17th, 2016 No Comments # When is consistency needed in a database? This is a great comment : Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. “Low level” operations as part of settlement may us transactions, but the bank system is “designed” (more like it has grown by accretion) to function almost entirely by settlement and reconciliation rather than holding onto any notion of consistency. The real world rarely involves ... Read More Source September 11th, 2016 No Comments # Why did Salesforce use markup tags for VisualForce? By now, I think we can agree, using HTML for the web was a mistake. But Sir Tim Berners-Lee could not have known that back in 1989. He was thinking that he could create a Semantic Web, and for that, the use of a markup language was defensible. Only in retrospect do we see that HTML was mostly used as a GUI for TCP/IP. Eventually Sir Tim Berners-Lee conceded the point, and so he went off and created RDF. But ... Read More Source September 5th, 2016 No Comments # Comparing data sets that are imbalanced Interesting: Research on imbalanced classes often considers imbalanced to mean a minority class of 10% to 20%. In reality, datasets can get far more imbalanced than this. Here are some examples: 1.) About 2% of credit card accounts are defrauded per year1. (Most fraud detection domains are heavily imbalanced.) 2.) Medical screening for a condition is usually performed on a large population of people without the condition, to detect a small minority with it (e.g., HIV prevalence in the USA is ~0.4%). 3.) Disk ... Read More Source September 5th, 2016 No Comments # Base 10 numbers need log(10) digits to be described I’m sure I knew this example once, but I’d completely forgotten it, and it is so perfectly obvious when we talk to non-technical people and they ask for an example of what logarithmic growth looks like: In mathematics, logarithmic growth describes a phenomenon whose size or cost can be described as a logarithm function of some input. e.g. y = C log (x). Note that any logarithm base can be used, since one can be converted to another by multiplying ... Read More Source September 5th, 2016 No Comments # Accessing any single element in an array takes constant time as only one operation has to be performed to locate it. This seems like it could be used as a trick question that would trip me up during a job interview: An algorithm is said to be constant time (also written as O(1) time) if the value of T(n) is bounded by a value that does not depend on the size of the input. For example, accessing any single element in an array takes constant time as only one operation has to be performed to locate it. However, finding the minimal ... Read More Source September 5th, 2016 No Comments # Beware Big O Notation in higher level languages This essay has a nice bit of details about a general point that needs to be made: Here we get the result that will be counterintuitive to many. No matter how large n gets, the Array List still performs better overall. In order for performance to get worse, the ratio of inserts to iterations has to change, not just the length of the collection. Note that isn’t an actual failure of Big O analysis, it is merely a common human failure ... Read More Source September 5th, 2016 No Comments # The gamification of Slack makes it addictive but also makes it a drain Very interesting: Everything bad about Slack is fundamentally Slack’s fault. Slack’s sane default for a new user is to play a sound and send a desktop notification every time anything happens anywhere. There is no way to simply turn off the screaming red circle on the dock icon. There is no way to simply fold away the sidebar so you can focus on whatever it is you are trying to do. These UI nitpicks help us understand Slack’s conception of the ... Read More Source September 5th, 2016 No Comments # It’s not even XML! Dave Winer said he is deeply depressed. Sometimes he seems frightened by the march of change. I suppose that is an aspect of depression. His initial fear of JSON seems like an expression of fear: I’ve been hearing, off in the distance, about something called JSON, that proposes to solve a problem that was neatly solved by XML-RPC in 1998, the encoding of arrays and structs in a format that could easily be processed by all programming languages. The advantage ... Read More Source September 5th, 2016 No Comments # Finding novelty via friends is a metric not yet captured by recommendation engines This is a good point: The traditional regime of recommendation systems has been obsessed with (1). What’s the uplift of recommendation algorithm A vs recommendation algorithm B? Which is driving more click-thrus and conversions? There’s something fundamentally broken in the this way of thinking though. I don’t care what the computer says. I care about what my friends say. The meaningful music I’ve discovered over the last 10 years has been music liked by someone I respect. Friends. Other musicians I ... Read More Source September 5th, 2016 No Comments # Detecting voting rings with HyperLogLog While this seems like a clever trick, I typically want a lot of metrics regarding voting, so throwing away the metadata doesn’t seem like an option to me. As an aggregate tool whose only purpose is finding voting rings, maybe this useful maybe? But consider what would happen if we created a HyperLogLog counter for every user on Reddit, and any time that a user receives an upvote, we update the corresponding HyperLogLog counter with the id of the user ... Read More Source September 5th, 2016 No Comments # Great browser software in 1996 These technologies sound so amazing, I wonder what happened to them? Atlas is the precursor to the next step in Navigator’s evolution, Version 3.0. It’s “alpha” code; in other words, that celebrated tower in Pisa is more stable. When it works, Atlas promises to deliver: VRML viewing. VRML (Virtual Reality Modeling Language) is one of the competing standards for expressing three-dimensional information in compact form. VRML documents can be static or interactive. For example, you could create a Web page that ... Read More Source September 5th, 2016 No Comments # The Big O cheat sheet I just discovered the Big O cheat sheet and I think this is very useful: Source September 4th, 2016 No Comments # Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring I just realized that I’ve been confusing Faraday and Maxwell. It’s Maxwell who did the 4 equations. Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring, and found that upon passing a current through one coil a momentary current was induced in the other coil.[2] This phenomenon is now known as mutual induction.[44] The iron ring-coil apparatus is still on display at the Royal Institution. In subsequent experiments, he found that if ... Read More Source August 19th, 2016 No Comments # HAProxy for MySQL Three years ago, I was intrigued to read that Twitter had implemented dynamic network topologies. Something similar is becoming more common, thanks to the use of Nginx/HAProxy. One can use this for many things, such as connecting to a pool of MySQL servers. However, there are some downsides worth remembering: You have to be careful to tune your xinetd script to increase the cps and per_source limits accordingly to the load that your pool of servers generate with this proxy forwarding. Assuming that ... Read More Source August 19th, 2016 No Comments # Clever uses of a custom TCP stack Interesting: Fun anecdote, at Blekko we had people who tried to scrape the search engine by fetching all 300 pages of results. They would do that with some script or code and it would be clear they weren’t human because they would ask for each page right after the other. We sent them to a process that Greg wrote on a machine that did most of the TCP handshake and then went away. As a result the scrapers script would hang ... Read More Source August 16th, 2016 No Comments # When algorithms betray you Interesting: You decide to share an article about Brexit from “The Atlantic”, which will somehow shed light to all your friends as to why it happened. The article is 1,000 words long — you only read half of it, but that’s good enough. It captures all the arguments you’ve been wanting to make for the past two months to your friends. Will this be the Facebook post that finally spurns your friends into action? You realize your Facebook friends all agree with your ... Read More Source August 11th, 2016 No Comments # Only good leadership can create good software This is very true, and it applies to more than software: Thirteen years ago, Eric Raymond’s book The Cathedral and the Bazaar (O’Reilly Media, 2001) redefined our vocabulary and all but promised an end to the waterfall model and big software companies, thanks to the new grass-roots open source software development movement. I found the book thought provoking, but it did not convince me. On the other hand, being deeply involved in open source, I couldn’t help but think that it ... Read More Source August 11th, 2016 No Comments # Clever hacks whereby sites can see aspects of your browser history Very interesting hacks: Up until mid-2010, any rogue website could get a good sense of your browsing habits by specifying a distinctive :visited CSS pseudo-class for any links on the page, rendering thousands of interesting URLs off-screen, and then calling the getComputedStyle API to figure out which pages appear in your browser’s history. After some deliberation, browser vendors have closed this loophole by disallowing almost all attributes in :visited selectors, spare for the fairly indispensable ability to alter foreground and background ... Read More Source August 8th, 2016 No Comments # The control mania of Scrum This is very good: Scrum inhibits deep understanding and innovation This is actually my biggest gripe about Scrum. As mentioned above, in Scrum, the gods of story points per sprint reign supreme. For anything that doesn’t bring in points, you need to get the permission of the product owner or scrum master or someone who has a say over them. Refactoring, reading code, researching a topic in detail are all seen as “not working on actual story points, which is what you ... Read More Source August 8th, 2016 No Comments # The daily Scrum meeting is a pointless ritual I love this: The daily standup deserves a blog post of its own. This religious ritual has become a staple of every team in the world. Ten minutes of staring into the void, talking about what you did while no one else listens, because they were in the middle of something five minutes ago and will go back to it in another five minutes, and waiting for everyone else to finish. I know this sounds cynical, but it is the ... Read More Source August 8th, 2016 No Comments # If you measure programmers by points then they will optimize for points The worst thing with Agile/Scrum is that you can hit all your 2 week targets and yet a year later have software that no one wants to use, so to the extent that Agile is suppose to reduce risk, it fails completely. But why does it fail? This is good: No matter how you define story points, the real issue with them doesn’t go away. The main purpose of points is making planning more reliable, and providing a temporal perspective ... Read More Source August 8th, 2016 No Comments # Problems with the Scrum process This is a great conversation: cageface says: But the author’s criticisms of the incentives of Scrum are on point I think. Because the stories are always articulated in terms of user facing features they encourage developers to hack things together in the most expedient way possible and completely fail to capture the need to address cross cutting concerns, serious consideration of architecture, and refactoring. This is how you can get two years into a project and have managers and clients that think that ... Read More Source August 3rd, 2016 No Comments # The JustWorks job interview One of the most popular posts I’ve written is “Embarrassing code I wrote under stress at a job interview“. People get a laugh out of the dumb things I do doing job interviews. Here is another such post. I did a job interview at JustWorks. They asked me to write the code to solve this problem: The cost of a stock on each day is given in an array, find the max profit that you can make by buying once ... Read More Source August 3rd, 2016 No Comments # Is Docker ready for production? I prefer uberjars to Docker, and I prefer fat binaries, such as those allowed by Go, over Docker. And still there is the question “Is Docker ready?” Senex says: I’ve been tracking the beta for a while. I’m confused about this announcement. These issues still seem unresolved? (1) docker can peg the CPU until it’s restarted (2) pinata was removed, so it can’t be configured from CLI scripts (3) it’s not possible to establish an ip-level route from the host to a container, which ... Read More Source July 27th, 2016 No Comments # How to package dependencies in Golang Very interesting, as I’m a fan of the idea of “vendoring”: An attempt to fix packaging in Go: Manul is a vendoring utility for Go programs. What’s the reason for yet another utility? Because all other vendor utilities suffer from the following: Some wrap the go binary and spoof the GOPATH env variable. You will have a non-go-gettable project which needs additional software in order to compile and run; Some copy the source code of dependencies into the vendor directory: It will be nearly impossible to find ... Read More Source July 26th, 2016 No Comments # Google Maps does not name Nachitschewan? I can not find a zoom level where Google Maps names Nachitschewan. Is this a political thing? Source July 25th, 2016 No Comments # The advantage of a weight-balanced B-tree A very interesting data structure of which I know nothing: Source July 22nd, 2016 No Comments # A string of characters is best parsed using a finite state machine I already knew this, but I don’t think I knew the extent of what was sacrificed to give Perl backreferences s: Notice that Perl requires over sixty seconds to match a 29-character string. The other approach, labeled Thompson NFA for reasons that will be explained later, requires twenty microseconds to match the string. That’s not a typo. The Perl graph plots time in seconds, while the Thompson NFA graph plots time in microseconds: the Thompson NFA implementation is a million ... Read More Source July 21st, 2016 No Comments # The problems with Clojure The conflict between Object Oriented Programming and Functional programming comes up a lot with Clojure, because to do any serious work you have to eventually use some Java, which takes you back into the world of Object Oriented Programming. This sums up my feelings: The thing I like most about Elixir is the low friction between it and it’s host language, Erlang. Erlang is a functional language right from the start, and the BEAM is designed to run a functional ... Read More Source July 21st, 2016 No Comments # The RegX that killed Stack Overflow A great post-mortem of a crash at Stack Overflow: The regular expression was: ^[\s\u200c]+|[\s\u200c]+$ Which is intended to trim unicode space from start and end of a line. A simplified version of the Regex that exposes the same issue would be \s+$which to a human looks easy (“all the spaces at the end of the string”), but which means quite some work for a simple backtracking Regex engine. The malformed post contained roughly 20,000 consecutive characters of whitespace on a ... Read More Source July 21st, 2016 No Comments # The difference between Kafka and Kestrel For my purposes, for the next 6 months, it seems that Kestrel will be all I need. I’d have to be very successful before I would need anything as complex as Kafka. The biggest conceptual difference is that Kestrel is a simple stand alone queue where as Kafka is a full fledged queuing system. Kestrel runs on a single machine and has no concept of clustering or failover or any other features you might expect in a queuing system. Instead the clients ... Read More Source July 21st, 2016 No Comments # RabbitMQ is difficult, Kafka is awesome, Kestrel is beautifully simple A great review of these 3 queues: Not mentioned below is that RabbitMQ works hard to guarantee delivery of a message, so it is slow, but that is because it is in some ways doing more than Kafka. RabbitMQ: I created 4 queues, wrote a ruby client and started inserting messages. I got a publishing rate of about 20k/s using multiple threads but I got a few stalls caused by the vm_memory_high_watermark, from my understanding during those stalls it writing to disk. ... Read More Source July 21st, 2016 No Comments # Kestrel is as simple as Redis but Kestrel is an actual queue I feel stupid that I didn’t look at Kestrel sooner. I feel especially stupid that I’ve been using Redis when I didn’t actually need it. Redis has a beautifully flexible API which makes it tempting to use Redis for everything, but Redis is really a cache that focuses on speed above all else. Everything in Redis has to fit in memory, and Redis will drop anything that can’t fit in memory. Also, the stuff in Redis can be mutated, ... Read More Source July 21st, 2016 No Comments # Only use Amazon SQS if you need a high-latency high-concurrency service Also interesting, don’t use Amazon SQS unless your needs fit this very specific model where you can deal with the latency and make up for it by being highly concurrent. I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is ... Read More Source July 21st, 2016 No Comments # For Apache Storm, use Kestrel There is a lot in this thread that is interesting. This is coming from Nathan Marz, the guy who invented Storm. He says he uses Kestrel, so that is a big endorsement. You want to make sure that your spout source can support the out of order acking that Storm requires for guaranteed message processing. We use Kestrel because it has this property and is the simplest. RabbitMQ is another good one to consider. Source July 20th, 2016 No Comments # The problem with RabbitMQ Seems like RabbitMQ is good but not perfect: Much like you’d chuck memcached on each of your web servers and access them in a ring, Darner can occupy a small niche on each box in your fleet’s resources. Tens of MB of RAM and negligible CPU opens up hundreds of gigabytes of queue spool per node. As queue size grows, memory usage remains constant. Contrast this with Redis, which is speedy but limited in queue size to what will fit ... Read More Source July 20th, 2016 No Comments # ZeroMQ is the opposite of a queue Reading this again, several years after the first time I read it, and now it occurs to me: I would use ZeroMQ only in exactly those situations where I would not use a queue. If I only need a queue, I can use Kestrel. But if I need some messaging pattern that is not supported by any queue, then I would use ZeroMQ. That is, if I had a truly unique situation that needed a unique pattern, then ZeroMQ would ... Read More Source July 20th, 2016 No Comments # Everything you need to know about queues This looks like an amazing resource and I’m sure I’ll go back and read more soon. Source July 20th, 2016 No Comments # The New York Times uses RabbitMQ RabbitMQ is written in Erlang, so I was inclined to think well of it, though I heard criticism of it. And then the New York Times used it. A very surprising vote of confidence in RabbitMQ: This architecture – Fabrik – has dozens of RabbitMQ instances spread across 6 AWS zones in Oregon and Dublin. The instances are organized into “wholesale” and “retail” layers. Connection to clients is via websockets/sockjs. Upon launch today, the system autoscaled to ~500,000 users. Connection times ... Read More Source July 20th, 2016 No Comments # Gauges uses Kestrel which is based on work from Blaine Cook Why does Kafka get all the attention if Kestrel is so reliable? I assume this is because Kafka can do so much more, though of course the devops work of Kafka can be frightening. I didn’t know about the Blaine Cook connection (of Twitter fame): Before I get too far a long with this fairy tail, let’s talk about Kestrel — what is it and why did I pick it? Kestrel is a simple, distributed message queue, based on Blaine Cook’s ... Read More Source July 20th, 2016 No Comments # Kestrel versus Resque Interesting note about these queues, which certainly convinces me to use Kestrel: There are a lot of things that could be done with either Kestrel or Resque. Because Resque is backed by Redis, you have to remember that all of the messages waiting to be processed have to be able to fit in the RAM of the Redis server, with Kestrel you could queue millions or billions of messages and then start to pull them off. The biggest difference between the ... Read More Source July 20th, 2016 No Comments # How bad can MongoDB be with Dirty Reads? Very worrisome, though no sane person would use MongoDB to track something involving money. That’s simply not what MongoDB is for. How bad are dirty reads? Read uncommitted allows all kinds of terrible anomalies we probably don’t want as MongoDB users. For instance, suppose we have a user registration service keyed by a unique username. Now imagine a partition occurs, and two users–Alice and Bob–try to claim the same username–one on each side of the partition. Alice’s request is routed to the ... Read More Source July 20th, 2016 No Comments # How to return a disallowed field in GraphQL Best Practices for GraphQL are still in a very immature stage, but these two ideas both have some merit: First, return null for the requested field. This seems to work well in cases where there is no real harm in asking for a particular set of data and no real harm in denying it. A good example would be asking for the email of a user where the backend only provides the user’s email to that user themselves. If I request my ... Read More Source July 20th, 2016 No Comments # Why has Google released gRPC This is interesting: Google has been using a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across our data centers for over a decade. Our internal systems have long embraced the microservice architecture gaining popularity today. Having a uniform, cross-platform RPC infrastructure has allowed for the rollout of fleet-wide improvements in efficiency, security, reliability and behavioral analysis critical to supporting the incredible growth seen in that period. Stubby has many great features – ... Read More Source July 13th, 2016 No Comments # Why I hate Ruby On Rails I’m trying to help a friend with a very small Ruby On Rails project, where she needs a few elements of the interface tweaked. But this is Ruby On Rails, so of course I’m losing 3 hours trying to get setup. I’ve been drawn into the endless swamp of upgrading various libraries, which seems to be an automatic part of dealing with Ruby On Rails. If this was Clojure, or NodeJS, setup would have taken 5 minutes and I would ... Read More Source July 13th, 2016 No Comments # Mixins are harmful — the Javascript edition Over the years there have been arguments against mixins, but the dominance of Object Oriented Programming meant that mixins still survived and had theoretical justification. Now, the spread of the Functional Paradigm has put renewed pressure on mixins. Dan Abramov at Facebook just came out with an essay against mixins: “How do I share the code between several components?” is one of the first questions that people ask when they learn React. Our answer has always been to use ... Read More Source July 10th, 2016 No Comments # Edges is just a wrapper around the real results that adds extra data used for slicing the result cursor I think the “edges” and “nodes” terminology is one of the most confusing things about React / Relay / GraphQL, at least at first: Relay connection defines a relationship an an object that has two root fields: edges and pageInfo. Edges is just a wrapper around the real results that adds extra data used for slicing the results (cursors), PageInfo has metadata related to the current page. This is how it looks like: connectionName { edges { ... Read More Source July 10th, 2016 No Comments # Make the Web compatible? It must be 1999 again because someone is writing about browser incompatibilities: Users won’t switch browsers, they’ll switch sites You might think that users will switch browsers to use your site. But many won’t or can’t. Users have no patience for things that don’t work, and they’ll just go to a competitor’s site instead. Failing at a critical point could turn a potential user away forever. According to Akamai, 32% of users who encounter a problem on your site are less likely to make ... Read More Source June 30th, 2016 No Comments # The Agile process of software development is often perverted by sick politics For those of you who don’t want to read this whole essay, here is the summary: The word “agile” has a plain meaning in standard English, and that meaning was considered something positive by software developers, so much so that the most successful new development process of the last 30 years calls itself “Agile”. However, at many of the companies that I have worked, actual agility is suppressed because of various political factors. Fear wins out over trust. Instead of actual ... Read More Source June 29th, 2016 No Comments # How much time and energy should a computer programmer spend learning a proprietary server system? By itself, this is an anecodote about Google’s particular system: (but see my point at bottom) Is there a lesson in this? Well, if I were in the business of programming Google App Engine, a few days’ effort up front to get it going might seem not to be a big deal. However, when I visit a team who have just added a new person, I invariably find that person struggling to set up their workstation. There’s usually someone in the room ... Read More Source June 29th, 2016 No Comments # Is Phoenix going to take over the tech industry? If not, why? This has been one of the big puzzles of the last 30 years: if some technologies are clearly superior, then why don’t they take over? Apparently because they lack an element that appeals to management in big corporations? That’s always been the argument against Lisp: it is great for the individual master craftsman, but it doesn’t work in a big corporation full of badly paid, mediocre programmers. Erlang has been the most safe, resilient technology out there ... Read More Source June 23rd, 2016 No Comments # Pagination with React / Relay / GraphQL This is an awesome article: Relay proposes a standard to define a has-many relationship for a GraphQL field. This standard defines a common structure that allows Relay to paginate and filter the results in an efficient way by using cursors, which I’ll explain in a bit. This is the definition of a Relay connection (from the Relay connection specs): Relay’s support for pagination relies on the GraphQL server exposing connections in a standardized way. In the query, the connection model provides a standard ... Read More Source June 15th, 2016 No Comments # Demanding ssh key login to a server is safer than allowing passwords This is good and true: Require ssh key authentication We tend to avoid passwords for logging into servers. There was a lot of discussion around this after Bryan’s original guide came out, but I tend to fall into this camp as well. Here are a few notes on this: ssh keys are better than passwords only because they contain and require more information. Passwords can be brute forced. Guessing a public key is so essentially impossible that they can be considered perfectly secure What ... Read More Source June 14th, 2016 No Comments # Object/Relational Mapping is rarely worth it I think this can be stated simply: for your basic operations (CREATE, UPDATE, READ, DELETE) then an ORM offers some convenience. But ORMs break down as soon as you have a complex query. And yet, if you start using an ORM because it helps you with the simple queries, it will seem natural to you to extend it and try to use it for the complex queries. This is a strained analogy to the Vietnam war: Object systems are typically characterized ... Read More Source June 11th, 2016 4 Comments # AWS is inappropriate for small startups because its complexity demands a specialist Sean Hull references a conversation that he and I had in Slack. I would like to expand on the argument that I made then. You might want to read his essay first, where he makes these points about AWS: 1. Over 70 services offered 2. Still complex to build high availability 3. Need a dedicated devops 4. Orchestration involves many moving parts 5. Troubleshooting failed deployments is difficult At the time of our conversation I had crashed an AWS instance and we were having trouble fixing ... Read More Source June 11th, 2016 No Comments # GraphQL is the best thing about React / Relay / GraphQL React / Relay / GraphQL: The big breakthrough here is obviously GraphQL. Not React or Relay, since there are many competing implementations that do similar things, but GraphQL. Even though there are graph query languages out there (Gremlin, etc) they were not suited to querying JSON over the wire. GraphQL is ideal. For my next project, I hope to do a Clojure implementation of most of GraphQL, because I think it can be married to Om.Next in a very powerful way. Actually, ... Read More Source June 10th, 2016 No Comments # Krubner, you saved my day Aw man, after all the mistakes I make, it is nice to know I help people sometimes: Source June 10th, 2016 No Comments # Sometimes I make a bad decision It’s very, very, very rare, almost unheard of, but sometimes I make a mistake. I was helping at Open Road this spring. Sean Hull was also helping. Open Road runs a bunch of WordPress sites, such as The Line Up. All of the images for these sites were stored on a central NFS server. All of the machines were starving for hard disk space. The previous Director Of Technology had made many, many bad decisions, including the decision to save ... Read More Source June 7th, 2016 No Comments # The Belief that Tests are Smarter than Code Telegraphs Latent Fear or a Bad Process This part is really good: Programmers have a tacit belief that they can think more clearly (or guess better) when writing tests when writing code, or that somehow there is more information in a test than in code. That is just formal nonsense. The psychological perspective is instructive here, and it’s important because that — rather than any computational property — most drives developer behaviour. If your coders have more lines of unit tests than of code, it probably means one of several things. They may be ... Read More Source June 7th, 2016 No Comments # The problem with unit tests Interesting: Unit testing is of course not just an issue in object-oriented programming, but the combination of object-orientation, agile software development, and a rise in tools and computing power has made it de rigueur. As a consultant I often get questions about unit testing, including this real one from a recent client of mine, Richard Jacobs at Sogeti (Sogeti Nederland B.V.): My second question is about unit tests. If I remember correctly you said that unit tests are waste. First, I was surprised by that. Today however, my ... Read More Source June 6th, 2016 No Comments # Ruby On Rails is obsolete This is a strange conversation to be having in the year 2016. Ruby On Rails changed the technology industry in 2004, but it can not claim to be winning in 2016. It still lacks a story for concurrency. It is being undermined by 2 forces: 1.) the need for concurrency, which is offered by other eco-systems, such as those that run the JVM. A language like Clojure offers the high level of meta-programming that Rubyists love, but with the vast arsenal ... Read More Source June 2nd, 2016 No Comments # Sometimes peer review delays progress for many decades This is what real leadership looks like: Morris intentionally consumes H. pylori. Like Marshall, he becomes ill, but unlike Marshall, he is not completely cured by antibiotics. The infection will remain with him for three years. I wish he had been given a medal for his self-sacrifice. He was attempting to save millions from suffering. Interesting, though very disappointing: 1881 Klebs notes the presence of bacteria-like organisms in the lumen of the gastric glands. 1889 Walery Jaworski describes spiral organisms in sediment washings ... Read More Source May 22nd, 2016 No Comments # Why does Rails become awful? Interesting: Let me tell you a story about what can happen in a convoluted Rails codebase. Once, I joined an existing project. It was a huuuuge app which was running an on-line shopping community website. Complicated sales model, complicated promotions, complicated product setups, coupons, user groups, messages – it had it all. I joined them to help ship a few new features. One of my early tasks was to…add a link to something on some page. It took me few days ... Read More Source May 15th, 2016 No Comments # The dumbest name for a good architecture is “serverless” I strongly agree that the idea is good, but the name “serverless” is really stupid: So… this is backend infrastructure (i.e. it runs on a server) to let your team deploy “severless” apps… So they’re serverless. Except the server you’re running it on. And the lamba/style code they wrote and uploaded to it. But serverless, because no ops staff required. Except the ones who installed and maintain this. This is like a snake eating its own tail and wondering what hurts. Edit: despite my sarcasm, ... Read More Source May 10th, 2016 No Comments # The sad, slow way a system of cron scripts becomes ugly Do you have a chore that needs to run in the background, maybe once a day, or once an hour? Cron scripts will save you! They are the most amazingly amazing thing God has invented since Adam and Eve! For sheer wonderfulality they have no equal among the products of mortal fallen flesh! At least at first. The simple cron script is wonderfully direct and efficient. But the first can lead to a second. The second can lead to a third. ... Read More Source April 10th, 2016 No Comments # What do AWS services actually do? I love this: Data Pipeline Should have been called Amazon ETL Glacier Should have been called Really slow Amazon S3 Kinesis Should have been called Amazon High Throughput It’s like Kafka I have used this but I did not know the name: Snowball Should have been called AWS Big Old Portable Storage Use this to Get a bunch of hard drives you can attach to your network to make getting large amounts (Terabytes of Data) into and out of AWS It’s like Shipping a Network Attached Storage device to AWS These are useful names that actually describe what ... Read More Source March 27th, 2016 No Comments # The fanaticism of “test driven development” is slowly fading This was one of those trends that went too far and I’m glad to see people talking about reasonable limits: If you’re part of the Ruby On Rails community for a long time, you’ve probably read tons of articles about testing Rails application (less these days, though). Although there always have been diverging opinions on the matter, it seems the common wisdom was to say that you had to test everything: models, controllers, views and full-stack tests. Oh, and you had ... Read More Source March 26th, 2016 No Comments # RESTful APIs are dead, long live GraphQL We can argue whether RESTful APIs ever existed, since very few ever implemented HATEOAS. Ruby On Rails claimed it offered RESTful APIs, but it never offered HATEOAS. After 10 years of failure, the world is looking for something new. Interesting: Imagine we have a simple application that fetches a list of stories, and some details about each one. Here’s how that might look in resource-oriented REST: // Fetch the list of story IDs but not their details: rest.get('/stories').then(stories => // This ... Read More Source March 26th, 2016 No Comments # Picking React isn’t a technology decision, it’s a business decision Interesting: The clear winner right now, is React. Components all the way down makes your application much easier to reason about. The learning curve is very flat. The important APIs would fit on one page. JSX is awesome. You get all the power of JavaScript and its tooling when writing your markup. It is the natural match for Flux and Redux (more on that later). The React community is amazing, and produced many best of breed tools such as Redux (also more on that later). Writing high ... Read More Source March 26th, 2016 No Comments # Pay discrimination in tech This sounds bad: Women who write the software that runs on mainframe computers earn on average 72 cents per dollar earned by their male counterparts, according to research conducted by Glassdoor Inc., the online job information firm. That pay gap exists even after controlling for age, education, experience, job title, employer and location. This suggests that the gap widens with age: However, not all tech jobs pay men and women so differently. Among mobile developers, there is just a 2.9% gap between ... Read More Source March 26th, 2016 No Comments # Left-Pad as a service Everyone is laughing over this. An 11 line function as its own NPM module? I am unsure how I feel. The trend has been to smaller and smaller pieces of functionality. I don’t find this outrageous. Still, the parody is funny: c4n4rd is game: This is really exciting!!! I was a bit disappointed that the right-pad will be out only in 2017. I am looking forward to that release because there is a high demand for it now. What kind of ... Read More Source March 26th, 2016 No Comments # null Testing WordPress to see if a title of “Null” works. Source March 18th, 2016 No Comments # The end of the test-first mania It worries me that the tech industry is driven by such intense manias, which have cost me several jobs. For a long time you were not allowed to question Object Oriented Programming. I still go to job interviews and I’m asked the basics of Object Oriented Programming, and it is clear I have to say nice things about Object Oriented Programming or I won’t get the job. Thankfully none of these people ever check Wikipedia, where I am cited as ... Read More Source March 17th, 2016 No Comments # Onyx tested by Jespen is genius tested by genius Good lord, this makes me happy. I think Michael Drogalis is a very smart guy and everyone knows that Kyle Kingsbury is a very, very smart guy. Kingsbury’s work on Jespen is the finest work that anyone has ever done on the problems of distributed data. Onyx is exciting as an a Clojure answer to Apache Storm. Sad to say, Storm is written in Scala. The idea of testing Onxy with Jespen is one of the finest ideas ... Read More Source March 14th, 2016 No Comments # Unix processes have gotten bigger and slower over the years Interesting: The obvious question thus is: Why state machines? Why not processes or threads? And the obvious answer is: Performance. When UNIX was still young, scheduling was supposed to be done by the OS on per-process basis. When implementing a network server, for example, you were supposed to fork a new instance of the process for each TCP connection and rely on the OS scheduler to switch between the processes. I guess it made sense from performance point of view back then. All ... Read More Source March 10th, 2016 No Comments # Jakub Holý on copying :pre and :post conditions Obviously I have read this article before (I offered response and was quoted before) but I’ve gone back to read it again and this jumped out at me: Do you repeat the same checks again and again? Then you could either copy them using with-meta (they end-up in metadata anyway) or reuse the explicitly: (defn with-valid-car [f] (fn [car] {:pre [:make :model :year]} (f car))) (def count-price (with-valid-car (fn [car] (do-something car)))) ;; or make & use a macro to make it nicer That ... Read More Source February 15th, 2016 No Comments # Sick politics is the driving force of useless ceremony This is part 7 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the driving force of useless ceremony 8.) Functional programming is not the same as static data-type checking Interlude 9.) Inheritance has nothing to do with Objects 10.) Is there a syntax for immutability? 11.) Immutability enables concurrency 12.) Quincy’s ... Read More Source February 13th, 2016 No Comments # Immutability changes everything (Acknowledgements: I offer a huge “Thank you” to Natalie Sidner for the tremendous editing she did on the rough draft of this post. To the extent that this article is readable, it is thanks to her. Any mistakes are entirely my fault, and I probably added them after she was done editing. If you need to hire a good editor, contact Natalie Sidner at “nataliesidner at gmail dot com”. Also, I thank Blanche Krubner for reviewing this work. As Mrs Krubner studied ... Read More Source February 13th, 2016 No Comments # Functional programming is not the same as static data-type checking (Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!) This is part 8 of a ... Read More Source February 10th, 2016 No Comments # Interlude I am mostly done writing the second half of this series, however, it needs to be edited, some code needs to be written, all of the code needs to be checked, and I need to work with Leah McCloskey to develop further cartoons for illustrating the points made in this series. Also, I have a full time job. So it will likely be 2 or 3 months before I can publish the second half of this series. The series is ... Read More Source February 10th, 2016 No Comments # Get rid of all Dependency Injection This is part 6 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the driving force of useless ceremony 8.) Functional programming is not the same as static data-type checking Interlude 9.) Inheritance has nothing to do with objects 10.) Is there a syntax for immutability? 11.) Immutability enables concurrency 12.) Quincy’s ... Read More Source February 10th, 2016 No Comments # Mutable iterators are the work of the Devil The assignment from Hell Assume you die, and then you wake up in a beautiful place, floating on a cloud, and you hear a trumpet in the distance, and you hear beautiful singing. Then a voice from up on high says to you, “I would like you to count all of the positive integers. I will give you all eternity to do it.” You assume you are now in heaven, and God is giving you your first heaven-based assignment. Can you ... Read More Source February 10th, 2016 No Comments # Clojure has mutable state Warning: This post is an intro to mutable state in Clojure. If you already know Clojure, you can skip this. If you have no interest in learning Clojure, you can skip this. This is part 3 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the ... Read More Source February 10th, 2016 2 Comments # Why I hate all articles about design patterns Sometimes play is a lie Imagine two children, both nine years old, decide to play tea. They rummage around in the closets of their respective parents. They dress up as they’ve seen their parents dress up. The one goes over to the other’s house. “Mr. Worthington, it is so nice to see you,” says the girl who is acting as host. “Would you like some tea?” “Why, yes, Mrs. Kentworth, I would be delighted to have some tea.” The host brings an empty cup ... Read More Source February 10th, 2016 No Comments # A parable about concurrency (demonstrated with comical cartoons) (Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!) This is part 1 of a ... Read More Source February 9th, 2016 No Comments # Why does Erlang allow so many processes to crash and how does promote reliability? Why does Erlang encourage crashes? Back-burning and controlled burns are a real world example of fighting fire with fire. In Saguenay–Lac-Saint-Jean, the region I come from, blueberry fields are routinely burnt down in a controlled manner to help encourage and renew their growth. To prevent forest fires, it is fairly frequent to see unhealthy parts of a forest cleaned up with fire, so that it can be done under proper supervision and control. The main objective there is to remove ... Read More Source February 6th, 2016 No Comments # Hindley-Milner and the lambda calculus A nice attempt to explain the crazy syntax of the math that formalizes the idea that the type of an expression can be deduced from the expression itself: Okay, so we want to talk about expressions. Arbitrary expressions. In an arbitrary language. And we want to talk about inferring types of these expressions. And we want to figure out rules for how we can infer types. And then we’re going to want to make an algorithm that uses these rules to ... Read More Source February 3rd, 2016 No Comments # Beware of Ruby libraries that generate way too many objects This is several years old and Ruby garbage collection has gotten better, but still, the point about certain libraries being excessive remains valid. Be aware that you are allocating objects, for instance something as simple as 100.times{ ‘foo’ } allocates 100 string objects (strings are mutable and therefore each version requires its own memory allocation). Make sure to evaluate the libraries you use, for instance switching a Sinatra XML rendering action from Builder to Nokogiri XML Builder saved us about ... Read More Source February 3rd, 2016 No Comments # Pat Shaughnessy dissects how much work Ruby has to do to give you a string Easy is difficult, and this is a great look at how much work Ruby has to do so that you, the software developer, can change your mind about what kind of string you want: The standard and most common way for Ruby to save string data is in the “heap.” The heap is a core concept of the C language: it’s a large pool of memory that C programmers can allocate from and use via a call to the malloc ... Read More Source February 3rd, 2016 No Comments # Software development is a complex system of multiple poorly understood feedback loops and interactions I wouldn’t use exactly the same words that this article uses, but I agree with the gist of this part, especially where small, fast-moving startups are involved : Finely grained management of software developers is compelling to a business. Any organization craves control. We want to know what we are getting in return for those expensive developer salaries. We want to be able to accurately estimate the time taken to deliver a system in order to do an effective cost-benefit analysis ... Read More Source February 2nd, 2016 No Comments # Leonardo Borges’s Imminent for futures and promises Very interesting: For the impatient, I’ve included a couple of examples below. I’ve chosen to translate the examples presented by Ben Christensen – of RxJava – in this gist. Albeit them being in Java, they highlight perfectly the problem with blocking futures. Here’s their Clojure equivalent: ;; ;; Example 1, 2 & 3 are handled by the approach below ;; Original examples: https://gist.github.com/benjchristensen/4671081#file-futuresb-java-L13 ;; (defn example-1 [] (let [f1 (remote-service-a) f2 ... Read More Source February 2nd, 2016 No Comments # Why the tech community rejected XML Interesting: Java was a very limited language, and it was extremely verbose. There were type declarations and coersions everywhere. Almost all looping was managed with for loops (there were no higher order functions, and recursion wasn’t tail cail optimized… also there was no loop-recur macro…). So XML wasn’t just there to be data, it was also code. Everyone wrote little mini-languages into XML, because some things were so damn painful to express in Java that it was easier to just implement ... Read More Source February 2nd, 2016 No Comments # Clojure community problems Interesting: What an amazing language. Relatively frequent, consistently stable releases. A pleasure to use. A friendly, smart community. I feel very lucky to be a Clojure user! Thank you for all of your hard work. The Clojure contrib process frustrates me more than any technical or community aspect of the language. Clojure gets a lot right, but as has been repeatedly discussed the pace of evolution and the maintainership’s dim view of 3rd party non-bugfix work flatly leads to worthy but minor ... Read More Source January 31st, 2016 No Comments # We want loose-coupling and high cohesion This seems like a great rule of thumb for microservices: “We want to avoid dumb, anemic services that are little more than CRUD wrappers” But it doesn’t cover the old territory which, if you were using Ruby or PHP, you would cover with a cron script. I suppose all the cron scripts must become functions that live inside the “service” which deals with a given part of the datastore. But that is not how my friends talk about “microservices”. Page 58, ... Read More Source January 31st, 2016 No Comments # A message hospital (or dead letter queue), where messages got sent if they failed I like this book very much. I also like the idea of a “message hospital”. Page 57, Building Microservices, Sam Newman: Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a ... Read More Source January 31st, 2016 No Comments # Jon Williams, Fractional CTO: Include business teams in the Agile process Jon Williams offers a smart idea about getting the business people and the tech team to share as much information as possible, and even have the business people join the scum sessions. The best implementation of the Agile process that I ever participated in was at ShermansTravel.com, back in 2011/2012. We had fairly good communication between the tech team and the folks running the actual parts of the business: editorial, advertising, marketing and more. A lot of the success of ... Read More Source January 31st, 2016 No Comments # Joe Armstrong figured out the right way to do everything, and nobody cared I am puzzled why good ideas so rarely win out in the tech world. I do know the old saying “Most industries have a top player with the best marketing and the second best technology, with a second place player that has the best technology and mediocre marketing.” Back in the 1990s it was common to apply that slogan to Microsoft and Apple (when Apple had mediocre marketing! Such long forgotten days!). Still, the lack of innovation in this industry ... Read More Source January 30th, 2016 No Comments # When technology goes on strike Interesting: Amazing but true When I’d finished this article, I wanted to spell check the content. emacs-ispell mode decided to go on strike. It could not find aspell, the program that I use for spelling checking. My emacs spell checker has worked faithfully on this machine for several years. And just when I complain that I spend half my life fixing things that shouldn’t be broken the emacs spell checker decides to break. I don’t believe in malicious Gods, nor that the laws ... Read More Source January 24th, 2016 No Comments # Groping in the dark as a method to discover module boundaries in microservices I’m not sure that we will ever have a better way (but then, we can never pretend to be like other types of engineers, can we?): Having clients talk to the application service instead of an ORM or the data backends directly lets you forget that there is a MASSIVE PROBLEM with ORM frameworks (for anything more interesting than, say, a blog website framework), because you will still have to write a translation between your application representation and the database representation. ... Read More Source January 24th, 2016 No Comments # Why I love Erlang And why I dislike Scala. I would bet that Scala was designed in the opposite fashion: Everything was very problem oriented and we did not have as goals that Erlang should be a functional language or that we should implement the actor model. We knew nothing of the actor model until later when we heard that Erlang implements it. :-) … Again our goal was to solve the problem, not design a language with a predefined set of primitives. Source January 22nd, 2016 No Comments # HTML is the failed GUI for TCP/IP I posted this to Hacker News. I was surprised that someone did not immediately understand what I meant when I referred to HTML as the GUI of TCP/IP. Someone had surveyed 80 frontend designers and found they lacked basic knowledge of HTML and CSS. I responded: The contrarian argument is that this signifies an important truth, that HTML never worked the way it was suppose to. In the same way that we might argue that a misunderstood product is the fault ... Read More Source January 20th, 2016 No Comments # An Amazon DSL in Clojure Good lord. This is an impressive list of supported Amazon APIs: Api Gateway Autoscaling CloudFormation CloudFront CloudSearch CloudSearchV2 CloudWatch CodeCommit CodeDeploy CodePipeline Config DataPipeline DeviceFarm DirectConnect Directory DynamoDBV2 EC2 EC2 Container Registry ECS ElastiCache ElasticBeanstalk ElasticFileSystem ElasticLoadBalancing ElasticMapReduce ElasticTranscoder Glacier IdentityManagement Kinesis KinesisFirehose KMS Logs Lambda MachineLearning OpsWorks RDS Redshift Route53 S3 SimpleDB SimpleEmail SimpleWorkflow SNS SQS StorageGateway Source January 20th, 2016 No Comments # A 3D DSL in Clojure Very interesting, and a nice example of how much Clojure allows the creation of new languages: My first step in the HOLO design process was the creation of a sufficiently flexible GP playground for my later experiments and to evolve path-finding agents to create a (typographic) form as an initial design idea. Related to this, though a year prior, I created the Mophogen DSL, partially done as a component of my commission for the Barbican / Google DevArt exhibition. Morphogen ... Read More Source January 16th, 2016 No Comments # Microservices first Stefan Tilkov adds clarity to an argument that I’ve been making since 2013: If you are actually able to build a well-structured monolith, you probably don’t need microservices in the first place. Which is OK! I definitely agree with Martin: You shouldn’t introduce the complexity of additional distribution into your system if you don’t have a very good reason for doing so. (So what would be a good reason? There are many, but to me the most important one is to ... Read More Source January 16th, 2016 No Comments # I give good tech advice to startups, and different good advice to Enterprises I had a job interview recently that went well. But then, the CTO read something that I had written a year and a half ago. Then the CTO said to me “We can’t hire you. We are a tiny startup. We are facing some serious deadlines. We need to push a product out the door.” I said, “I want to help you push a product out the door. I can work hard and help you guys move fast.” He said, ... Read More Source January 16th, 2016 No Comments # The cost of using patterns is that I have to give up the illusion that I am infinitely creative This is true of all creative fields: The cost of using patterns is that I have to give up the illusion that I am infinitely creative. I don’t invent a new programming language with optimizing compiler and novel operating system and complete programming environment for every line of code. To make best use of my three billion seconds, I should ignore most of my options and focus on a few degrees of freedom that really matter right now. Source January 16th, 2016 No Comments # Paralyzed by too much thinking I like this: When I started programming again, I vowed not to type a single character unless I knew what pattern I was applying as I did so. The result was incredibly frustrating. I want a class called “Stack”, but why “Stack” and not something else. Then I would go and write the patterns for naming classes and then I could type “Stack”. Then I would want to make its first method public, but why public? At first, it was like ... Read More Source January 14th, 2016 No Comments # The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise Interesting What is the black king in chess? This is a strange question, and the most satisfactory way to deal with it seems to be to sidestep it slightly. What more can one do than point to a chessboard and explain the rules of the game, perhaps paying particular attention to the black king as one does so? What matters about the black king is not its existence, or its intrinsic nature, but the role that it plays in the game. The ... Read More Source January 13th, 2016 No Comments # Invoxia will allow Alexa to figure out who is speaking I will be pleased if this becomes more than vaporware: Invoxia, one of the recipients of an Alexa Fund investment from Amazon last September, announced this week at CES that it’s the first third-party hardware maker to incorporate all the power of Alexa into a product other than the Echo. The company’s Triby, as it’s called, is a colorful, magnet-backed Bluetooth speaker resembling an old-school radio. It’s designed to let family members, including young children, make internet-based phone calls with ... Read More Source January 7th, 2016 No Comments # CQRS Read Models Interesting: We’ve been using this mechanism with great success during development. It let us freely modify the schema by only changing the Java classes and never worrying about table definitions. Thanks to combination with CQRS, we could even maintain long-running demo or pilot customer instances. Data has always been safe in the event store. We could develop the read model schema incrementally and have the changes automatically deployed to a running instance, without data loss or manually writing SQL migration scripts. Obviously ... Read More Source January 6th, 2016 No Comments # No one realizes that there is an app store for the Amazon Echo? More from Jose Jaquinta: I ran into some people during various New Year’s parties who had Echos, but had no clue there were skills for them. I laboriously explained how to launch the companion app, find the skills tab, and enable the skill. It would a lot simpler if a user could enable/disable skills without having to go through the companion app. For example: Alexa, enable Demotivate Me I realize this is a request for The Echo, and not the Alexa Skills Kit. However, ... Read More Source January 6th, 2016 No Comments # The Amazon Alexa certification process has gone out of control This is sadly accurate: The certification process has gone out of control. This is one example of the… I’m struggling to keep to professional language… questionable judgement… show by the certification team. I’ve had two skills come back with a similar objection. I want to ask them if they have even used Alexa regularly? It gets REALLY tedious to have it endlessly repeat things. It’s like, “Yes, I know what I can do next. Please let me get on with ... Read More Source January 1st, 2016 No Comments # Mark Lee Smith fights functional programming in 2011 I find it interesting to go back and look at the sites where was learning about Functional Programming back in 2011. It’s interesting to read those who were on the losing side of history, fighting against Functional Programming: If you’re writing 2000-line classes and 500-line methods then you’re in no way qualified to be speaking about object-oriented programming, let alone comparing it with the latest FAD ideology. Christian Toivola gave what has become the standard answer: @Mark Lee Smith: With the ... Read More Source January 1st, 2016 No Comments # When did the Internet start? Interesting: TCP version 1 was designed in 1973. This was documented through RFC 675. TCP version 2 was documented in March 1977. In August 1977, Jon Postel realized they were going the wrong direction with the protocol. “We are screwing up in our design of internet protocols by violating the principle of layering. Specifically we are trying to use TCP to do two things: serve as a host level end to end protocol, and to serve as an internet packaging and ... Read More Source December 31st, 2015 No Comments # RFC 128: We are contemplating allowing sender and receiver to specify different byte sizes Wow. Can you imagine how that would have turned out? On the bright side, Unicode would not be necessary, since a company in China could specify a byte size large enough to represent any Chinese characters, and likewise for Iran, Korea, Hindi, etc. . Source December 31st, 2015 No Comments # NLP versus Conversational AI Obviously, I have written a lot about this subject during the year. See my post “What happens when the Board Of Directors begins to panic“. But this is getting big: To date, commercial NLP research has been focused on understanding the many ways people say commands like, “Set my alarm for 7 am tomorrow” or “Find a good pizza place nearby.” To unlock the power of conversational computing, new technology is required that extracts semantics across multi-turn natural language ... Read More Source December 31st, 2015 No Comments # Allowing an object to exist in an invalid state is an Anti-Pattern This post uses the language of Object Oriented Programming, but its point would also be true to all forms of programming. In Clojure, if you have HashMaps that you then convert to JSON and save to MongoDB, then why would you ever allow the HashMap to exist in an invalid state? Real immutability might imply that the HashMap is created with everything that it will eventually need. This is an interesting idea. Why do we validate data? Typically, to make ... Read More Source December 31st, 2015 No Comments # Websockets are an all or nothing proposition Interesting: I’m also a heavy websocket user, and agree with most points. I have previously used websockets on top of a traditional web app, and have been disappointed with the results. My opinion now is that websockets are an all-or-nothing proposition. And I have gone all-in. My latest project has: - websockets-only api (except a some image uploads and oauth/login) - https-only - single page app, central store with observable data, using a single connection servicing the whole application - using SignalR (supports fallbacks for IE9 ... Read More Source December 31st, 2015 No Comments # Any growing Rails app runs into the problem of fat models This is a good comment: A lot of things in Rails are are anti-patterns in large code bases, but pragmatic in small ones. ActiveRecord itself is a prime example: when you start an app, putting your business logic directly in ActiveRecord objects works pretty well in most cases, but later on as the models proliferate and grow, you realize that some of them contain business logic which is far too complex to warrant being conflated with persistence concerns. The result ... Read More Source December 18th, 2015 No Comments # How to find a major security breach at Facebook Facebook offers a bounty for bugs, but Facebook did not play fair on this one. Instead, Facebook immediately reached out to the hacker’s employer and accused the hacker of unethical behavior. The Facebook CSO later offered the cheap excuse that the hacker might have been working in an official capacity for the employer. The more likely story is that Facebook was embarrassed and wanted this story to be remain hidden. Interesting: With the newly obtained AWS key, I browsed several ... Read More Source December 17th, 2015 No Comments # How to test the Salesvoice skill for the Amazon Echo 1.) if you have access to a Salesforce account, use that. If there is an admin at your company who manages the Salesforce account, talk to them. Otherwise, you can sign up for a free 30 day trial on Salesforce. Be sure to go for the Enterprise Edition. 2.) You must enable the SalesVoice Connected App 3.) Relax IP restrictions, or whitelist the IP from which your Amazon Echo will be in use 4.) You will need to enter 4 pieces of ... Read More Source December 15th, 2015 No Comments # The importance of the Module class to NodeJs Interesting: The Module type found in module.js has two main roles inside of Node.js. First, it provides a foundation for all Node.js modules to build off of. Each file is given a new instance of this base module on load, which persists even after the file has run. This is why we are able attach properties to module.exports and return them later as needed. The module’s second big job is to handle Node’s module loading mechanism. The stand-alone require function that we ... Read More Source December 15th, 2015 No Comments # Require and Export in NodeJs Interesting: Finally, the last thing to consider is what happens when you directly export a function: var powerLevel = function(level) { return level > 9000 ? “it’s over 9000!!!” : level; }; module.exports = powerLevel; When you require the above file, the returned value is the actual function. This means that you can do: require(‘./powerlevel’)(9050); Which is really just a condensed version of: var powerLevel = require(‘./powerlevel’) powerLevel(9050); Source December 14th, 2015 No Comments # How to build conversations via the Amazon Echo If you read my post about the startup I was this year, then you know the toughest challenge we faced was the building the finite state machine that could handle conversations. So I read this about the Amazon Echo with great interest: My task would have ended here if creating an event would only require a date and time. But to be useful, I would like to include a duration, a topic, perhaps even a location. To be even more ... Read More Source December 13th, 2015 No Comments # Using a glottal stop to force the Amazon Echo to correctly pronounce “tw” This an impressive bit of phoneme hacking, by Joseph Jaquinta: It’s also occasionally off on some words. A particular problem for me was Alexa’s inability to correctly pronounce “tweets”. It can say “tweet” just fine, but “tweets” comes out as “wheets”. Since one of upcoming skills, Tweet Poll, centers around producing election statistics based on tweets, it uses the word “tweets” a lot. So I was pretty excited when I saw that the new update included features for giving more fine control ... Read More Source December 13th, 2015 No Comments # Dialogue designers replace graphic designers when creating voice interfaces Over the last 30 years we’ve gotten used to the Graphical User Interfaces in software that appears on a computer screen. And most of us (who work in the tech industry) have had the experience of working with a person who, depending on their skills, will be given a title such as “graphic designer” or “user experience designer” or sometimes “product designer”. Working with the Amazon Echo, myself and my partner have come to realize that we still need a ... Read More Source December 13th, 2015 No Comments # How can I connect the Amazon Echo to wifi in an Enterprise with secure wifi? I posted this question on the TalkEcho forum. Any suggestions are welcome. One big problem we face (developing Enterprise apps for the Amazon Echo) is when we do a demo at a large company. Then we are typically facing the WPA2 Enterprise version of WiFi, which the Echo does not seem to support. When I look here: https://www.amazon.com/gp/help/customer/display.html?nodeId=201549640 I see it says: “Amazon Echo connects to dual-band Wi-Fi (2.4Ghz / 5Ghz) networks that use the 802.11 a/b/g/n standard. It does ... Read More Source December 13th, 2015 No Comments # The old debates about Clojure This was written in 2009 and I linked to it in 2011, and I just stumbled upon it again. It is interesting to remember the old fights about Clojure, the struggle for respectability that Clojure faced in those old days. For the most part, that particular fight is no longer facing Clojure. Source December 12th, 2015 No Comments # The Amazon Echo Blogosphere I will be updating this blog post as time goes by. I hope to build a complete catalog of blogs that are primarily focused on the Amazon Echo. Here is what I found after 30 minutes of searching: LoveMyEcho is one of the few blogs that seems devoted to the Amazon Echo Joseph Jaquinta has a blog on LinkedIn devoted to the Amazon Echo Engadget has a category for Amazon that sometimes covers the Echo Gizmodo has a category specifically for the ... Read More Source December 12th, 2015 No Comments # Custom commands for the Amazon Echo Interesting: Telling Amazon’s Alexa to play some music or add an item to your shopping list is pretty cool, but there comes a point where you start to feel the limits of the Echo’s pre-programmed voice commands. It doesn’t have to be that way anymore: Amazon and the automation wizards at IFTTT just announced a new feature that allows you to create custom voice phrases to make Alexa do whatever you want. Unfortunately, there’s a catch: every command has to ... Read More Source December 12th, 2015 No Comments # Ellery Coffman demonstrates home automation via Alexa Ellery Coffman gets Alexa to start a movie on the TV, then stop it, and then play an album on the record player, and then stop it. This is impressive in all ways, except I was surprised at how slow it was. I’ve got an Alexa app running on a Rackspace server, which pulls data from Salesforce, and the response is much faster, despite the fact that there are 2 API calls involved, and despite the fact that Salesforce ... Read More Source December 4th, 2015 No Comments # The advantages of a Natural Language Processing interface “admin” of Daily Hacker News is critical of my ideas regarding a Natural Language Processing interface for software. Some of their points conflate Natural Language Processing with the text version of Natural Language Processing, and right now I’m mostly working on Natural Language Processing via voice interfaces, so I’m going to reply with voice examples, so as to remind everyone that Natural Language Processing is a broad topic. “admin” starts with this point: Let’s go back to Lawrence’s example. Posit, ... Read More Source December 2nd, 2015 No Comments # People want to be able to talk to their computer When GUI interfaces were new in the 80s we were told they were better than command line interfaces because they offered “discoverability”. But it turned out they didn’t offer nearly enough discoverability, so in the 90s “Wizards” became big things, especially in the Microsoft world. A Wizard provided the context that a GUI by itself did not. A Wizard would tell a user “Do this, now do this, now do this, and then do this.” But Wizards were visually complex. ... Read More Source December 2nd, 2015 No Comments # Clojure is easier than Scala Interesting: After about a week of experimentation, we hit a sticking point. While we had a large number of “normal” CRUD operations, all of our queries on behalf of the end user, were generated dynamically. Most of the Scala database ecosystem is geared towards extending Scala’s type system to include database query type checking. Overall, our enthusiasm for Scala didn’t survive the first attempt to port our query language system. Clojure for the win It was at this juncture where we re-evaluated Clojure, ... Read More Source November 28th, 2015 No Comments # Christine Dodrill on the Universal Design She has her own terminology: State – What is true now? What was true? What happened in the past? What is the persistent view of the world? Events – What is being changed? How will it be routed? Policy – Can a given event be promoted into a series of actions? Actions – What is the outcome of the policy? Mechanism – How should an event be taken in and an action put out? Policy seems a bit confusing, but sometimes there is a reward if ... Read More Source November 28th, 2015 No Comments # Separate AWS accounts for development, staging, and production Interesting: Instead of using regions or tags to separate different staging and prod instances, we switched over totally separate AWS accounts. We need to ensure that our provisioning scripts wouldn’t affect our currently running services, and using fresh accounts meant that we had a blank slate to start with. The ops account serves as the jump point and centralized login. Everyone in the organization can have a IAM account for it. The other environments have a set of IAM roles to switch ... Read More Source November 28th, 2015 No Comments # I am one voice of many, who question Object Oriented Programming I am just one of many voices questioning Object Oriented Programming and promoting the Functional paradigm as an alternative. I pleased to think my big essay, which surveyed quite a bit of what had been written before, is still able to occasionally encourage people to re-think their support for Object Oriented Programming. So I see Matt’s Code Cave write: Why functional programming and why Clojure? I’ve been interested in the idea behind functional programming for a while now – being used ... Read More Source November 21st, 2015 No Comments # The States and Fates of Javascript Interesting resource regarding Javascript Promises: States Promises have three possible mutually exclusive states: fulfilled, rejected, and pending. A promise is fulfilled if promise.then(f) will call f “as soon as possible.” A promise is rejected if promise.then(undefined, r) will call r “as soon as possible.” A promise is pending if it is neither fulfilled nor rejected. We say that a promise is settled if it is not pending, i.e. if it is either fulfilled or rejected. Being settled is not a state, just a linguistic convenience. Source November 21st, 2015 No Comments # Why modern Javascript development sucks The package management tools are broken: Source November 21st, 2015 No Comments # How do you get latitude with Javascript Here is something I did not know about:$cordovaGeolocation.getCurrentPosition always returning “Position retrieval timed out.” in ios

That’s an error, but it made me aware that you can try to get location via Javascript. I think that is amazing. Why even bother with a native app? And I say that as someone who just spent 6 months at a starup that was committed to doing a native app, in part for native features, including the ability to get latitude and longitude.

Source

November 20th, 2015

# Trailblazer has good ideas hidden by Object Oriented Programming

Trailblazer for Ruby On Rails has some very good ideas in it though they are somewhat buried by the Object Oriented programming garbage:

To give an example of how operation works, let’s say you click on a delete button. What happens when you do that is you go to the control that knows a little bit of this action because it does authentication—if it’s valid it does one thing and if it’s not it does another thing. Instead of delegating ...

November 19th, 2015

# Nested relations are the death of Object Oriented Programming

Last winter made fun of a Python team for wasting time trying to figure out how to handle deeply nested relations. They were slowly groping their way toward a graph database solution, which seems to be the route that most end up taking, given enough years and enough pain. This seems to be an area where Object Oriented Programming clearly fails.

And yet, many smart folk continue to try. Ruby’s Reform:

Syncing Back

After validation, you have two choices: either ...

November 19th, 2015

# Rails derails

A super interesting debate about the move away from Rails.

We seem to be on the same side when it comes to quality metrics.

Source

October 10th, 2015

# A Java Hashmap is not a Clojure collection

I did not know this till just now. I have a co-worker who wrote a library in Java. From my Clojure code, I call this Java function:

public HashMap<String,HashMap<String,Integer>> init(String debrief,String companyName, ArrayList<String> contacts, ArrayList<String> accounts, ArrayList<String> requiredFields) { try { return transformer.transform(debrief, companyName, tecClassifier, rollioClassifier, caseClassifier, caselessClassifier, customClassifier, pipeline, parser, props, firstNames, lastNames, accounts, contacts, requiredFields); ...

October 10th, 2015

# The TCP checksum is weak, and the Ethernet checksum will accept corrupt TCP that passed the TCP checksum

Interesting:

At Twitter, a team had a unusual failure where corrupt data ended up in memcache. The root cause appears to have been a switch that was corrupting packets. Most packets were being dropped and the throughput was much lower than normal, but some were still making it through. The hypothesis is that occasionally the corrupt packets had valid TCP and Ethernet checksums. One “lucky” packet stored corrupt data in memcache. Even after the switch was replaced, the errors continued until ...

October 10th, 2015

# When is computer programming easy?

Interesting:

While it is often thought that “real” programmers like real programming environments (my husband, for instance, is perfectly happy in MF Assembler, which is pretty grim), most programmers seem to become pretty happy moving up to graphical user interfaces and visual environments where they choose from a list of “correct” choices. I don’t think it is only users that can be more productive when they are better supported.

I believe there is a direct relationship between the ease of use of ...

October 6th, 2015

# How easy is it to write immutable Javascript?

If you reinvent the language as a new language, then you can have immutable Javascript. On the frontend you have no choice, but on the backend? Why not use a language that offers what you need upfront, rather than forcing you to work for it? Interesting:

The main issue I’ve had using immutablejs with Redux is debugging. Whereas previously I could simply mouse-over a data structure when I hit a breakpoint (or crash), I now have to do a REPL ...

October 4th, 2015

# Erlang – OTP – Cowboy are the cutting edge

At the current time, the only thing out there, in the tech industry, that might cause me to move a way from Clojure, is something build around the Erlang VM . In those situations where performance and massive concurrency are needed, something like Erlang/Cowboy or Elixir/Phoenix need to be looked at.

I load tested (using wrk) nginx serving a two line, static HTML file, against a basic Cowboy service that parsed parameters, did an ETS lookup, and rendered several hundred ...

October 4th, 2015

# Clojurescript as the frontend to Erlang

Radically different, yet becoming more common:

We use vanilla SmartOS, so that there is no dependency on FiFo for your running VM’s. You could just switch FiFo off and all your VM’s would continue to just work. It also comes with a number of great advantages:

In our opinion, ZFS is simply the only file system that should ever be used – period.

Compression, ARC and ZIL work incredibly well, especially for DalmatinerDB which achieves amazing throughput partially thanks to being purpose ...

October 4th, 2015

# Hacking environmental protection features

Interesting:

Well, it turns out that all this environmental friendliness is starting to trip over itself, because most devices now start up in standby mode. So you can’t just power them on to power them on, you have to power them on and hit a button on the remote. Some devices, by pure dumb luck I assume, will accept the switch already being pressed in when they start. If that’s the case, you can hack the behaviour you want if you ...

September 26th, 2015

# The Recursive Function Pattern Matching Pattern

Sean Johnson has a great video up. Common in Erlang, useful in Clojure, 3 functions, one to start, one to do the work, one to end, all defined by arity, using Pattern Matching.

Also called the “Start, Work, End” pattern.

This is often done in Clojure with zipmap, but this does not make things as clear as the Erlang version:

This is much more clear:

Source

September 26th, 2015

# Best practice pattern matching in Clojure

If your function starts with a conditional, replace that with pattern matching:

Source

September 16th, 2015

# Clean up your act with lein-checkall

I am a bit late but I’ve only recently become consistent about using a linter to check my Clojure style. And I’ve only today discovered the Leiningen plugin lein-checkall, which combines lein check && lein kibit && lein eastwood && lein bikeshed.

Venanti has written about the importance of these:

Eastwood

Eastwood is a Clojure linter, invoked with lein eastwood. As a general request, please use a linter. Some of my favorite moments in the last year have come from people trying to ...

September 13th, 2015

# Calculating and Visualizing Voronoi Diagrams using the Quad-Edge Structure – Alan Shaw

Allan Shaw talks about a data structure called a “quad edge”. There were a lot of ideas here that were new to me. The data structure keeps rotation as one value, and the orientation of the edge as one value, but anything like coordinates are kept in a field called “data”. This separates the topology from the geometry. There is also a “next” function to find the next edge. This, apparently, gives great advantage when calculating a whole network of ...

September 12th, 2015

# Microservies mean freedom of future action

Interesting:

In short, maximize future freedom of action. This heuristic also answers the when and why questions for microservices.

To prove it, let’s start with Sprott & Wilkes on SOA. Wait, Service Oriented Architecture? SOA is microservices loving parent. The philosophy trickles down:

When the service is abstracted from the implementation it is possible to consider various alternative options for delivery and collaboration models. […] It is entirely realistic to assume that certain services will be acquired from external sources because it ...

September 11th, 2015

# Why are microservices happening now?

The term “microservices” only goes back to 2013. I would have trouble saying why I love the style so much, except that I hated dealing with Ruby On Rails and the PHP framework known as Symfony. I was willing to deal with a lot to get away from the pain of those systems. The growth of great package managers also seemed to weaken the need for monoliths.

But Martin Fowler says the most important change was the move to Continuous ...

September 11th, 2015

# The difference between Clojure and Common Lisp

Fascinating bit by Giles Bowkett. This uses recursion and is how you would do it in Common Lisp:

(defn build [list-1 list-2] (if (nil? list-1) () (concat (map (fn [bubble] (list (first list-1) bubble)) list-2) ...

September 11th, 2015

# Microservices are different

It does feel like the world is moving closer to distributed IPC. We are not there, but the push toward microservices seems a step in that direction. The warnings of the past feel less relevant now.

When I wrote Patterns of Enterprise Application Architecture, I coined what I called the First Law of Distributed Object Design: “don’t distribute your objects”. In recent months there’s been a lot of interest in microservices, which has led a few people to ask whether microservices ...

September 11th, 2015

# Zach Tellman — the need for backpressure in queues, and the limits

This is very good. A lot of this interesting, but perhaps the biggest surprise is when he adds 16 consumers of tasks, but also 16 producers of tasks, the crisis point comes suddenly, compared to when there was only 1 consumer and 1 producer. With multiple consumers the consumption is averaged out, so each machine works fine right up to the moment of crisis.

His main theme is “Unbounded queues are fundamentally broken because it puts the stability of our ...

September 11th, 2015

# Dynamic scope in Clojure

The interesting thing here is that the dynamic vars are wrapped in functions, which makes them a bit safer than dynamic scope would imply.

Each of these sub-clauses is very similar to the parent structures, and can be arbitrarily nested. As such, it is most easily constructed with a recursive function. However, Clojure’s let bindings are lexical, and don’t extend into recursive calls. We could make the memoized gensym call a parameter, but since we also need a separate generator ...

September 11th, 2015

# This instance doesn’t do much, but it does have a nice property: it’s only equal to itself

This is a very clever hack:

Graphviz assumes explicit identities for nodes, so what about when we want to represent a tree? We can’t simply use the identity of the node, because the same value at different positions within the tree must be treated as separate nodes. One possibility is to represent each node as a tuple of its value and its position in the tree, but this forces us to consider how to represent positions, which is neither obvious nor ...

September 11th, 2015

# The 8 fallacies of distributed computing

This is great:

The IT group usually has different administrators, assigned according to expertise–databases, web servers, networks, Linux, Windows, Main Frame and the like. This is the easy situation. The problem is occurs when your company collaborates with external entities (for example, connecting with a business partner), or if your application is deployed for Internet consumption and hosted by some hosting service and the application consumes external services (think Mashups). In these situations, the other administrators are not even under ...

September 10th, 2015

# Is Gradle the best build tool for the JVM?

Gradle continues to innovate with new features.

Is Buildr dead? Buildr is now part of Apache and yet its last update was over a year ago. I tried to use Buildr this year but I ran into errors and when I searched on Google I found all of the advice was many years old. No one responded to my question on StackOverflow. I’m left thinking that Buildr is dying.

Gradle has an awesome bunch of plugins and the eco-system is growing. ...

September 10th, 2015

# No locks for performance under load with concurrency

Interesting:

It’s quite clear that the lock-free approach scales a lot better under contention. This follows our intuition because lock-freedom allows system-wide progress even when a thread is blocked. If one goroutine is blocked on an insert or lookup operation, other operations may proceed. With a mutex, this isn’t possible.

Matchbox performs well, particularly in multithreaded environments, but there are still more optimizations to be made. This includes improvements both in memory consumption and runtime performance. Applying the Ctrie techniques to ...

September 10th, 2015

# Use Specter to transform Clojure’s lists

Interesting:

To change the name of all the stations named “Barista” to “Coffee Master”, I can use a for comprehension:

(for [site sites station (:stations site)] (if (= (:name station) "Barista") (assoc station :name "Coffee Master") station))

But using Specter:

(->> sites (transform [ALL :stations ALL #(= (:name %) "Barista")] #(assoc % :name "Coffee Master")))

Now, this example is not actually doing justice to Specter. But in a previous (superseded) versions of the ...

Check out my book:

March 27, 2018 8:49 am

"Chris, thank you for writing. In terms of the evidence, the case seems overwhelming. It is damning that the pe..."

March 26, 2018 6:57 pm

"you haven't really established your thesis - that women are being pushed away from tech. the last time i looke..."

March 11, 2018 10:49 am

"Hasen Judy, thank you for writing. About this: "the tech industry is hostile to women or society discourage..."

March 11, 2018 4:32 am

"Saying that 5% of women are awesome at technology is not really generous on your part. First we have to ask..."

March 5, 2018 11:35 pm

"I don't know who you are, or why your copyright is as out of date as your SSL certs, but good post. Subscribed..."

March 5, 2018 12:21 am

"You should immediately read the book "The Expert Beginner". Possibly your hate for design pattern originate..."

February 27, 2018 6:43 pm

"Man! I simply love you! I've always thought with myself: "why the heck should someone want to use OOP in PHP!?..."

February 26, 2018 8:35 pm

"Bubuche, thank you for writing. As to whether encapsulation is right or wrong, much depends on the definition...."

February 26, 2018 4:58 pm

"Richard Remer, thanks for writing. I agree, it is often best to simply have a powerful server. I think the clo..."

February 26, 2018 1:52 pm

"If the AWS way already works for you, then it's useful. But if the AWS way does not, it's like pulling teeth ..."

February 25, 2018 9:13 pm

"I find this article quite interesting. However, there was something itching me all along the reading. From t..."

February 5, 2018 2:53 pm

"We are going to construct a physique of information which we can truly declare for ourselves and which we ar..."

January 29, 2018 12:09 am

"Josh Rehman, thank you for writing. What you write is interesting. Kessler is not explicit about what she ..."

January 27, 2018 8:23 pm

"Kessler hints at her goal with this line in the last paragraph: "The blog post displays a lack of awareness fo..."

January 11, 2018 8:51 am

"the great kathy sierra has covered one aspect of this in one of her posts : http://headrush.typepad.com/creat..."