Shanghai Building to be Demolished

Technology

August 10th, 2018

In Technology

3 Comments

Why does anyone use Python?

Over the last 20 years I’ve written in software in PHP and Ruby and Javascript and Java and Swift and Scala and Perl and Coffeescript and TypeScript and Clojure and Python. My favorite is Clojure. My least favorite is Scala, but Python is competitive in the race for “most annoying eco-system.”

Despite the fact that PHP has some hilarious essays about how broken it is I used to enjoy writing PHP code, especially PHP 4, before they added unnecessary Object ...

Read More Source

July 28th, 2018

In Technology

No Comments

Ghostery has killed my use of Google Chrome

For the last 2 years I had Ghostery installed on Google Chrome (on a Mac) and I was using Chrome as my main web browser. I love the way it protected me from spam advertising. I’d open my favorite magazine and a purple box would open in the bottom right part of the browser window, where Ghostery would list all the advertising widgets it had blocked. The list was eye opening for me. On some magazine sites there were as ...

Read More Source

July 23rd, 2018

In Technology

No Comments

The cost of Test Driven Development

Interesting:

“The nice thing about working at Microsoft,” Nagappan says, “is that the development organization is large enough that we could select teams that allowed for an apples-to-apples comparison. We picked three development projects under the same senior manager and looked at teams that used TDD and those that didn’t. We collected data from teams working on Visual Studio, Windows, and MSN and also got data from a team at IBM, since the project was a joint study.”

The study and its ...

Read More Source

July 19th, 2018

In Technology

No Comments

How is an operability review different from monitoring?

I read this article but I didn’t understand what was gained, compared to good monitoring pushing stuff into good graphs, thus giving people a real-time view of the system.

Architecture & Deployment diagram: The first thing is to have the deployment & architecture diagrams. Where architecture depicts the logical flow of request/ response inside the system, deployment diagram is focussed on the actual infrastructure. But if the system is not too complex, it makes sense to have them in a single ...

Read More Source

July 18th, 2018

In Technology

No Comments

Overview of the TICK stack

It’s odd that I spend all day reading tech news, and there are still so many project that I know nothing about:

This project is a very young one, only about one year old (first commit in github was on aug 2015). It is, however, backed by the world’s most popular TimeSeries database: InfluxDB. When diving in, you can see this is a very energetic group, which has covered a lot of ground in a very short time and is presenting ...

Read More Source

July 18th, 2018

In Technology

No Comments

The “index” is where Riemann keeps all of its information

Interesting:

The index is a table of the current state of all services tracked by Riemann. Each event is uniquely indexed by its host and service. The index just keeps track of the most recent event for a given (host, service) pair.

Streams run quickly and retain as little state as possible. The index is Riemann’s state table; its picture of the world. The dashboard, network clients, and streams can all query the index to see what the system looks like now. ...

Read More Source

July 18th, 2018

In Technology

No Comments

What does ‘cider-jack-in’ do

I was a bit curious about this::

When you ask Emacs to start a new REPL using cider-jack-in it actually sends this request to Leiningen, the Clojure build automation tool. Once Leiningen has started the REPL, then a connection is made to that REPL from Emacs using the nREPL protocol.

This feature is from the CIDER package for Emacs, a Leiningen plugin called cider-nrepl and the Clojure tools.nrepl library are required.

Cider configuration (version 0.10 and earlier) In earlier versions of CIDER ( Source

July 18th, 2018

In Technology

No Comments

Details on AWS private/public subnets

A note to myself. I’m doing this with Terraform, but of course, one has to know how it will all actually work. Connecting a private subnet to the public world:

Resolution

1. List the Availability Zones that have the instances you want to attach to the load balancer.

2. Create an equal number of public subnets in the same Availability Zones where your private instances exist. To ensure that the load balancer can scale properly, verify that each subnet for the load balancer ...

Read More Source

July 15th, 2018

In Technology

No Comments

Use Packer to build AMIs that use ZFS for the file system

This is a great article: Building ZFS Root Ubuntu AMIs with Packer. You can do the same thing with Docker but I would not want my software engineers taking on those devops tasks. Indeed, one reason I’m so hostile to Docker is because it disrupts the specialization of tasks that I want to see on my teams. I expect to work with great devops people, and I’ll leave devops decisions to them.

From the article:

For all applications of importance or ...

Read More Source

July 15th, 2018

In Technology

No Comments

Things that allow you to recover from having a bad tech team

Over the years, I’ve come to realize that there are 2 failsafes that make it easier for an early stage technology startup to recover from making bad hires on the tech team.

1.) Hire a great project manager. A great project manager is able to get good work out of mediocre programmers. Also, great project managers have a good intuitive feel for how long tasks should take, so if some task misses its deadline, the project manager will know why. ...

Read More Source

July 14th, 2018

In Technology

No Comments

Elasticsearch Backup : Snapshot and Restore on AWS S3

Because Terraform will delete all of your databases when you rebuild your infrastructure, and that is a nasty shock. So you should manually back up ElasticSearch:

In Aws you have many options to allow [users|servers] access AWS resources. The idea we have is to setup a snapshot repository on Amazon Aws S3 and doing a restore from that specific location. In order to do that, the servers that have to access S3 must be authorised. Thankfully to Amazon we have many ...

Read More Source

July 12th, 2018

In Technology

No Comments

Nasty moments in Open Source development

Two bad incidents, in PHP and Python.

Escalate? Oh how I wish I had someone to escalate to. Painful to read. Here is someone filing a bug report and attacking Ramus Lerdorf, the guy who created PHP.

[2010-01-08 21:51 UTC] endosquid at endosquid dot com This is going to cause us MONTHS or fixing code for no real benefit since this behavior change is arbitrary and seemingly, was made for no reason. We are all engineers and developershere, and ...

Read More Source

July 10th, 2018

In Technology

No Comments

A meta technology hype roadmap

Very good:

Let me write a meta technology hype roadmap, so we can place these sorts of articles: * Old technology is deemed by people too troublesome or restrictive.

* They come up with a new technology that has great long-term disadvantages, but is either easy to get started with short-term, or plays to people’s ego about long-term prospects.

* Everyone adopts this new technology and raves about how great it is now that they have just adopted it.

* Some people warn that the ...

Read More Source

June 20th, 2018

In Technology

No Comments

Mobile operators appear to have increasingly turned to new techniques for tracking users — HTTP header enrichment

Interesting:

Our analysis reveals a range of HTTP headers injected into mobile user traffic by 13% of the 299 mobile operators in our dataset (including both MNOs and MVNOs). We classify the headers in our dataset into three categories, based on their likely purpose:

• Privacy-compromising headers (5 operators): HTTP headers leaking sensitive information that can uniquely identify the device (IMEI), the subscriber (MSISDN or phone number), or the subscriber’s location.

• Tracking headers (6 operators): operator-generated UIDs (subscriber-unique identifiers) that enable user ...

Read More Source

June 13th, 2018

In Technology

No Comments

Deep Learning Needs Clojure – Carin Meier

Source

June 13th, 2018

In Technology

No Comments

Jenkins pipelines must be built so they can run across multiple executors, even if you only run Jenkins with one executor

A bit hidden, and important for anyone using Jenkins, This fact should be better known:

Serialization and the “node” Block Jenkins pipelines must be built so they can run across multiple executors, even if you only run Jenkins with one executor. This is done using the node block. With it, a job can be split into smaller pieces that can be scheduled to run on any executor on any agent in the cluster. Once a node block completes, Jenkins serializes the ...

Read More Source

May 30th, 2018

In Technology

No Comments

SELinux provides tools to more finely control the activities allowed to users, processes, and daemons

Interesting:

SELinux provides tools to more finely control the activities allowed to users, processes, and daemons to limit the potential damage from vulnerabilities.

In the third and final part of our server security series, we will look at how we can enhance the security of Linux-based AWS EC2 instances with SELinux. We will learn how to set up SELinux on Amazon Linux, and we will walk through a simple example on Red Hat Enterprise Linux (RHEL).

In Linux, we can easily control access ...

Read More Source

May 17th, 2018

In Technology

No Comments

Markov Decision Process versus Reinforcement Learning

I’m still struggling to learn when to use the various different Machine Learning approaches:

To solve a problem like this, you can use MODEL-BASED approaches if you know how likely it is that the robot will move from one state to another (that is, the transition probabilities for each action) or MODEL-FREE approaches (you don’t know how likely it is that the robot will move from state to state, but you can figure out a reward structure).

Markov Decision Process (MDP) – ...

Read More Source

May 17th, 2018

In Technology

No Comments

Sean Hull offers a great tutorial regarding Terraform and Amazon’s Elastic Container Service

This is very much worth reading if you are using or considering Terraform:

It takes a bit of getting used to, but This terraform how to, should get you moving. You need an EC2 host to run your containers on, you need a task that defines your container image & resources, and lastly a service which tells ECS which cluster to run on and registers with ALB if you have one.

If you don’t know, the ALB is the Application Load Balancer. ...

Read More Source

May 17th, 2018

In Technology

No Comments

Impressive things about Terraform

I am impressed with the way that Terraform allows us to specify a setup of servers and load balancers and firewalls and whitelists and set it up, and use it for development, and then the day we decide to go public, we only have to run one command to re-create everything in production. That is a high level of automation.

Good people have insisted that this is only possible while using Docker containers, but I don’t see that at all. ...

Read More Source

May 16th, 2018

In Technology

No Comments

mountebank is the first open source tool to provide cross-platform, multi-protocol test doubles over the wire

This is interesting. I mostly just use the “curl” command to test APIs over the Internet, but this could be useful when I need a lot of mock services:

mountebank employs a legion of imposters to act as on-demand test doubles. Your test communicates to mountebank over http using the api to set up stubs, record and replay proxies, and verify mock expectations. In the typical use case, each test will start an imposter during test setup and stop an imposter ...

Read More Source

May 13th, 2018

In Technology

33 Comments

Docker is the dangerous gamble which we will regret

Summary: don’t use Docker, or any other container technology. Use Terraform and Packer instead. It’s one less level of virtualization, and therefore one less level of complexity.

There is perhaps one good argument for using Docker. It is hidden by the many bad arguments for using Docker. I’m going to try to explain why so much Docker rhetoric is stupid, and then look at what reason might be good.

Every time I criticize Docker I get angry responses. When I ...

Read More Source

May 13th, 2018

In Technology

2 Comments

Docker protects a programming paradigm that we should get rid of

This comment from 2014 still seems relevant in 2018:

I can see the advantage for dev boxes where a developer might want to setup a load of containers on their machine to emulate a staging or production environment. But I don’t really understand why you’d want to base your entire production infrastructure on it.

The most upvoted response demonstrates a sad confusion:

It’s the UNIX philosophy applied to applications, everything must be as small as possible, do only one task and do it ...

Read More Source

May 13th, 2018

In Technology

3 Comments

The Coral Reef Pattern of incremental improvement

The Acme Legal Firm needed a small office, and there was little available for rent in their city, so they decided they would build their own place. They hired a builder and asked her what they should do.

“Well, I think we should build this office with wood. I’m a big believer in wood. It is beautiful and fairly strong, and it is easy to hire workers who know how to work with wood. The failure modes of wood are ...

Read More Source

May 12th, 2018

In Technology

No Comments

How much pain does it take before Kubernetes is not worth it?

Interesting:

For our particular project, I think we’re still getting started when you take into consideration the likely overall lifetime. Still, I suspect that moving an existing architecture wholesale to Kubernetes is a bad idea in general; if you’ve not grown up within its particular limitations, they will prove to be an endless annoyance4, even if there are a host of other tangible benefits over going solo. This may not be limited to Kubernetes, it may just be true for anything, ...

Read More Source

May 12th, 2018

In Technology

No Comments

How to change complex network systems once you’ve made important decisions?

Interesting:

Lets use service discovery as an example. We’ve used service discovery along with health checks and self diagnostics to be able to do some fairly interesting things during outages and, more importantly, slowdowns. These are hard problems, and we’ve solved them rather crudely in places, but these solutions still give us a level of proven, necessary sophistication for system stability.

Unfortunately, our approach is slightly incompatible with Kubernetes’ centralized one. Usually, when this happens, your needs are just too sophisticated, ...

Read More Source

May 12th, 2018

In Technology

No Comments

Why programmers are biased to the stuff they already know

Interesting:

In the real world, these biases are amplified because “things we build ourselves” is actually “things we already know”, and this includes “things we already learned”. I suspect that a significant source of programmers flooding into management is the decreasing wavelength of full scale re-education on how to accomplish the exact same thing slightly differently1 with diminshing returns on tangible benefits.

I’ve been “learining” Kubernetes for a significant migration we are attempting at $WORK. I’ve found it tough going. It’s a ...

Read More Source

May 11th, 2018

In Technology

No Comments

Systems of process orchestration

I get that Linux Cgroups might be acceptable, in the sense that “perfect is the enemy of the good” and Cgroups are maybe good enough for now. I’d prefer it if the industry could rethink what a multi-user computer is, get rid of Unix, take the best ideas forward into a new system, and build something where something as clunky as Cgroups and Docker are not needed — it should be possible to have a “fat process” which does what ...

Read More Source

May 11th, 2018

In Technology

No Comments

What are Linux cgroups

This is not the way I would have gone, but whatever. Here we are now:

cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes.

Engineers at Google (primarily Paul Menage and Rohit Seth) started the work on this feature in 2006 under the name “process containers”.[1] In late 2007, the nomenclature changed to “control groups” to avoid confusion caused by multiple ...

Read More Source

May 8th, 2018

In Technology

No Comments

So many things can go wrong with Docker

Sean Hull always has interesting things to say, and so it is worth reading what tripped him up with a recent project where he had to Dockerize everything:

When a service is run, ECS wants to have *all* of the containers running together. Just like when you use docker-compose. If one container fails, ecs-agent may decide to kill the entire service, and restart. So you may see weird things happening in “docker logs” for one container, simply because another failed. What ...

Read More Source

May 7th, 2018

In Technology

No Comments

Is there anyone who can keep up with everything AWS offers?

As I’ve suggested before, unless you a full time specialist tracking AWS, you won’t be able to keep track of all the services coming out of AWS. The one I just learned about is ALB:

Amazon Web Services (AWS) just announced a new Application Load Balancer (ALB) service.

I spent some time playing with the new service to understand what it offers and to see how it fits into our cloud architecture.

In summary, ALB is a massive improvement over ELB in almost ...

Read More Source

April 30th, 2018

In Technology

No Comments

Confluence is very difficult to set up

I had endless errors with Confluence. Each time I started over, I set up a new EC2 instance. I eventually learned that it was possible to restart the setup process by deleting the file confluence.cfg.xml so I did this:

/etc/init.d/confluence stop

rm /var/atlassian/application-data/confluence/confluence.cfg.xml

/etc/init.d/confluence start

but now I’m trying to finish the setup, and all I get is a blank page:

Does anyone know how to fix this?

Source

April 12th, 2018

In Technology

No Comments

Excessive abstraction in PHP frameworks

I’m working with Laravel now, a PHP framework. I see this on StackOverflow:

With Laravel, everything is more complex than it needs to be. Yesterday I was dealing with the issue of logging and errors. How to get Laravel to output errors somewhere that I could see them? Well, that involves overriding some logging classes. The tone of the documentation is always “See, it’s very simple, and flexible, just override the default classes, inherit from the parent, override the method, choose ...

Read More Source

April 9th, 2018

In Technology

No Comments

Do the staff at Jezebel manually manage the trolls in the comments?

Apparently so, this is interesting.

Someone named Jinni writes:

Our Muqaddimah of Code 45* is still banned, alas. A bright point in my own days feels notably dimmed. For those of you who didn’t read (or quite ‘get’) Code 45*; it was a hallucinatory tour into the subconscious of Donald Trump as evidenced by the odd and illogical punctuation of his disordered tweets. Code 45*, like a creative, and slightly mad psychotherapist, attempted to wrest vestiges of meaning from the morass of ...

Read More Source

April 8th, 2018

In Technology

No Comments

PHP code for Apple Push Notification Service

I just lost a week of my life trying to send a message from my server to the APNS servers. A previous team had given us some bad PHP code which I could not get working. I finally tore it out and replaced it with the example I found on StackOverflow:

/* We are using the sandbox version of the APNS for development. For production environments, ...
Read More Source

April 8th, 2018

In Technology

No Comments

A pem tool to generate the pem file for Apple Push Notification Service

Having lost the last week of my life trying get a pem file that will let me talk to APNS, I will link to this, because it seems to be a solution for the future:

Automatically generate and renew your push notification profiles

Tired of manually creating and maintaining your push notification profiles for your iOS apps? Tired of generating a pem file for your server?

pem does all that for you, just by simply running pem.

Source

April 6th, 2018

In Technology

No Comments

Why don’t any articles about Apple Push Notifications agree about what commands to use?

Apparently there are a lot of ways to do this? I am suspicious, because I lost days trying to get this to work. This article says:

Now we need to encode these two files in Base64:

openssl base64 -in cert.pem -out cert.txt

openssl base64 -in key.pem -out key.txt

But this gist says:

Step 1: Create Certificate .pem from Certificate .p12

Command: openssl pkcs12 -clcerts -nokeys -out apns-dev-cert.pem -in apns-dev-cert.p12

Step 2: Create Key .pem from Key .p12

Command : openssl pkcs12 -nocerts -out apns-dev-key.pem -in apns-dev-key.p12

None ...

Read More Source

April 5th, 2018

In Technology

No Comments

Apple Push Notifications converting p12 to pem files

None of the commands listed on this page worked for me, and yet that is the topped ranked page in Google. What did work for me:

openssl pkcs12 -in apns_certificates.p12 -out apns_certificates.pem -nodes openssl pkcs12 -in apns_key.p12 -out apns_key.pem -nodes chmod 0400 apns_key.pem chmod 0400 apns_certificates.pem openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert apns_certificates.pem -key apns_key.pem cat apns_certificates.pem apns_key.pem > Peeroapp.pem Source

April 3rd, 2018

In Technology

No Comments

The difficulties of starting a career in tech

This is a good summary of some issues with starting a career in tech. This one jumps out at me, though I realize the issue of portfolios is controversial in some quarters.

More than a couple of times have I met developers looking for web development gigs without a proper portfolio to show off.

Having a portfolio helps to demonstrate what you are capable of doing.

You’ve probably been told that having a personal domain puts you a mile ahead in your ...

Read More Source

March 27th, 2018

In Technology

No Comments

I just lost a week of my life dealing with one bug in a Cordova/Ionic mobile app

Over the last week I had two marathon days where I worked more than 12 hours, and I had 4 normal days where I worked 6 to 8 hours, and all of it was wasted on one bug.

I’ve a friend who wanted an app built that would work for both iOS and Android. He contracted with a team in India to build it. They used Cordova and Ionic.

When they were done with the project they sent it to ...

Read More Source

March 25th, 2018

In Technology

No Comments

Apple takes two weeks to update a forgotten password?

I am amazed by this. On Friday I tried to log into:

https://developer.apple.com

None of my regular passwords worked. Only later did I realize I’d probably set up a separate account for this, using some unusual password. Anyway, I figured this would not be a big deal. I clicked “I forgot password”. I put in my email and then my phone number.

Apple then sent a text message to my phone, with a code that I was to type into the website. ...

Read More Source

March 22nd, 2018

In Technology

No Comments

Even a basic iOS app generates a huge number of errors in Xcode

I’ve decided I should learn how to build native iOS apps. So I am following a basic tutorial for Xcode and iOS.

As soon as I’ve created the project, I try to run it. It does compile and run, but the log is full of hundreds of errors. I’m putting a small sampling below.

Does Apple realize how confusing this is for someone who is just beginning?

I learned how to program back in 1999, writing PHP scripts. One great ...

Read More Source

March 19th, 2018

In Technology

No Comments

CSS is terrible because it is global state

CSS is broken because the style sheets are global. Once upon a time, this didn’t seem terrible because it was thought that CSS would operate as CONSTANTS. But they are not constants. On any modern website or app, the styles are always changing. So the styles are really a global variable, and they have all the problems that 60 years of computer science has taught us to expect from global variables.

There are other problems with CSS. Perhaps the biggest ...

Read More Source

March 19th, 2018

In Technology

No Comments

Online education in the 1960s

Interesting:

As Dear tells it, students were far more interested in using PLATO to chat with one another than to complete their lessons. They even created their own emoticons by taking advantage of a feature within PLATO’s operating system that allowed them to stack letters on top of each other instead of typing them side by side. Typing the word “WOBTAX” in a stack produced a smiley face.

“This was a great pastime of many undergraduates, many of whom would flunk out ...

Read More Source

March 10th, 2018

In Technology

No Comments

The lack of volume in a high-dimensional cube

I wish this felt more intuitive to me:

This is a starting point for intuition:

To provide some intuition consider the situation in two dimensions, as shown in Figure 10. For a point on the circle to be close to the equator, its y-coordinate must be small.

So, this is sort of an expression of the Curse Of Dimensionality?

But perhaps the most amazing thing is that we can not figure out how many spheres might surround a central sphere, in such a ...

Read More Source

March 8th, 2018

In Technology

No Comments

In Apache you can setup a reverse proxy with either RewriteRule or ProxyPass

With Apache I have always done something like this:

ProxyPass / http://127.0.0.1:8080/ ProxyPassReverse / http://127.0.0.1:8080/

But I just saw this interesting bit on StackOverflow:

RewriteEngine On RewriteRule ^(/.*)$ http://app-server:8181/jellyfish$1 [P] ProxyPassReverse / http://app-server:8181/jellyfish/

Which does the same thing. The RewriteRule says “grab everything after the ...

Read More Source

February 12th, 2018

In Technology

No Comments

Containers like Docker will not fix the sick politics of your company

Interesting:

Deciding what’s yours and what’s theirs is the basis of every sibling-rivalry détente. In Eric Brewer’s CAP theorem you can pick two of consistency, availability, and partition tolerance as long as one of them is partition tolerance, because, as distributed systems expert Caitie McCaffrey puts it, “physics and math.” In a distributed system that contains humans in multiple time zones, you’re inevitably going to have partitions, and waiting 10 hours for headquarters to wake up and make a decision is ...

Read More Source

February 3rd, 2018

In Technology

No Comments

When things go wrong on a software project

I like this:

Whenever things are not progressing as expected, it is a natural instinct to fall back to micromanagement, but it seldom truly cures the ill. It’s probable that your team disagrees with your vision or its feasibility – and that you’re either not listening to their feedback, or they don’t think you’d care. It’s good to assume that most of your employees are as smart or smarter than you; barking your orders at them more loudly or more frequently ...

Read More Source

February 2nd, 2018

In Technology

No Comments

A very long debate about when to use a library and when to write your own code

The whole thing is interesting, but Jens Stimpfle’s comment comes close to representing my view:

“I see the value in learning by building yourself, but from a software engineering point of view, using a tried and tested framework is likely to give you higher quality product in less time.”

Well, the framework I was to use last (Qt) had bugs that simply can’t be fixed by users (memory leaks, double free leading to segfault when exiting after reloading QML engine) and immature ...

Read More Source

January 18th, 2018

In Technology

No Comments

The continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers

Interesting:

The conclusion seems inescapable that with at least certain kinds of large programs, the continued adaptation, modification and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers who are closely and continuously connected with them.

Source

January 17th, 2018

In Technology

No Comments

Chris Zheng is angry at the Clojure community

This initially sounds like it is going to be interesting:

I say it with much admiration and respect to all the members of community. I’ve learnt so much from being in this microcosm of dynamism, ideas and learning over the past 8 years. Clojure has allowed me to get to know so many amazing people, to travel to a whole bunch of places and to do things that I had never thought possible.

Having said that, I’ve gone from a wide-eyed, ...

Read More Source

January 16th, 2018

In Technology

No Comments

What is up with Narrative.io?

When I go to the website of Narrative.io all I see is black. Is this because I have an ad blocker installed? If so, this is the most dramatic ad-block caused bug that I’ve seen. Something like 20% of everyone on the Web is now using ad-blocking. Narrative.io should keep that in mind when they design their site.

Source

January 6th, 2018

In Technology

No Comments

Yegor Bugayenko: 5 things to improve Java

Yegor Bugayenko offers suggestions for making Java better:

Global Variables. There are Singletons in Java, which, as we all know, are nothing but global variables. Wouldn’t it be great to enable global variables in Java and get rid of Singletons. PHP, JavaScript, Ruby and many other languages have them, why doesn’t Java?

Global Functions and Namespaces To group static methods together we create utility classes, where we have to define private constructors to prevent their instantiation. Also, we have to remember which ...

Read More Source

January 5th, 2018

In Technology

No Comments

How should your company handle data safety and version control?

I’ve been writing software professionally since 1999. The first few years, I didn’t work at a company that used version control. Colin Steele mentioned Subversion to me in 2005. I used it till 2011. Since 2011, every company I’ve worked at has used Git.

From 2005 to 2011 we used Subversion to keep track of all projects. This was at both startups that I helped run, and also while doing most client work. During these years, I only had one ...

Read More Source

January 5th, 2018

In Technology

No Comments

The problem with HTML

(originally from a longer essay)

Just to offer some perspective about where I’m coming from, I’ll start by saying that I think of these as tools that made it easy for beginners to be productive:

Hypercard

Visual Basic (obviously I mean the classic versions, before .NET. The stuff in the 1990s was genius)

Adobe Flash

Adobe Dreamweaver

(Of these links, I especially recommend the one about Hypercard.)

I spent 1995 learning HTML and putting together simple websites. I was thinking this is something I’d like to ...

Read More Source

December 26th, 2017

In Technology

No Comments

Jakub Holý on the awesome productivity of Clojurescript

I have found the REPL useful when working with Clojure. I plan to write about this soon. Jakub Holý has a great write-up regarding both Clojure and Clojurescript:

Aside – about us: Telia Norge is Norway’s second largest mobile operator. Our team has spent the past three years with Node.js, React, and (eventually) Redux creating a great webshop (for hardware and subscriptions) to replace an off-the-shelf one, with a great success. Now we set out to make also the lives ...

Read More Source

December 25th, 2017

In Technology

No Comments

Jakub Holý: how to build a shopping cart

Interesting:

However there are other concerns than simplicity and time-to-production that we could have been taken into account, I believe. But I failed to communicate those in time. The current solution would have been perfect if the network transfer times were negligible and if the backend service was 100% reliable. None of that is true. Every call takes at least 300ms and the backend can be overloaded or (as yesterday) unavailable due to network issues. I there is a high ...

Read More Source

December 25th, 2017

In Technology

No Comments

Jakub Holý: don’t add unnecessary checks to your code

This is a very good point:

Imagine you need to call .indexOf on product_code. You might be tempted to add

if (!_.isString(product_code)) return false;

Now you are safe and your code can’t fail. But when I read this code, I immediately think:

Oh, I believed product_code to be always a string but obviously it is not always the case. It must have happened in the past or the author wouldn’t have added this check. I wonder what are the possible non-string values. And what ...

Read More Source

December 18th, 2017

In Technology

No Comments

The penalty of saying something controversial on HackerNews

I made a comment on HackerNews and it was heavily downvoted. I made a good point about the limitations of Git, but people did not want to hear it. This raises an interesting issue about voting systems, in general, is there any way to find stuff that is true, reliably, when the voters might have incorrect beliefs? I think Slashdot went furthest, they took voting systems seriously, and they tried to weight the votes of those who were themselves upvoted. ...

Read More Source

December 12th, 2017

In Technology

No Comments

The Krubner Law Of Ugly Code

The Krubner Law Of Ugly Code states that any computer programming language that has a “return” keyword will tend to be ugly. “Tend” means the tendency is there, though of course, as I’m sure hundreds will rush to remind me, you can write beautiful code in any language.

Here is a simple example from a well-respected project. Here the return is used in a guard clause. This is good code, for the Java language, in that it puts the guard ...

Read More Source

December 12th, 2017

In Technology

No Comments

To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice?

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

http://www.artima.com/intv/interop.html

Bill Venners: To what extent is “DLL Hell” a failure of interface contracts to work adequately in practice? If everyone fully understands and adheres to the contract of the functions of a particular DLL, shouldn’t updating that DLL in theory not break any code?

Anders Hejlsberg: ...

Read More Source

December 12th, 2017

In Technology

No Comments

Camille Fournier (Rent the Runway) – Consensus Systems for the Skeptical Architect

(Tonight I’m dumping some of my notes onto my blog. Not sure why I kept these for years without publishing. I was probably waiting for a moment when I could write a longer essay about them.)

ZooKeeper owns your availability. Worth watching. I thought it was amazing when she said that even 5 years ago, even at Google, most programmers were not thinking in terms of distributed computing. Instead, its been the boom in microservices that has driven the adoption of ...

Read More Source

December 12th, 2017

In Technology

No Comments

Sam Newman’s book Building Microservices

I’m posting here some quotes I like from this book:

Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a grid to create a pool of pricing workers, allowing ...

Read More Source

December 12th, 2017

In Technology

No Comments

Why I prefer dynamic-typing over static-typing: the speed of adapting to change

I am a fan of dynamic typing. So is corporate America. The widespread use of PHP, Ruby, Python and Javascript suggest that dynamic typing is useful. Static typing tends to be used in areas where legal regulations create needs that outweigh programmer productivity. If static typing lead to greater programmer productivity (via a reduction in bugs) then corporate America would only use statically-typed languages. But it doesn’t.

In How ignorant am I, and how do I formally specify that in ...

Read More Source

December 11th, 2017

In Technology

No Comments

Watching the AI bots on a game fight each other is like testing a data analysis model

I was researching something else and I followed a link to the game Wesnoth. I downloaded it and spent some time testing it. The game is entirely built by volunteers. The game is slow and somewhat tedious, but they did a great job with the art and they did a great job enabling a variety of scenarios.

They claim they put some time into the AI engine that lets bots play a character. I was curious about that. I set ...

Read More Source

November 25th, 2017

In Technology

No Comments

That’s supposed to be an umlaut in the title

This sounds like a very serious criticism of WordPress:

Yes, I know that’s supposed to be an umlaut in the title. I just can’t persuade WordPress to do it.

As far as I know, WordPress has handled UTF8 for a long time. Does anyone know what is going on?

Source

November 25th, 2017

In Technology

No Comments

VisualBasic could have saved Microsoft Excel

Companies are rebelling against Excel:

Finance chiefs say the ubiquitous spreadsheet software that revolutionized accounting in the 1980s hasn’t kept up with the demands of contemporary corporate finance units. Errors can bloom because data in Excel is separated from other systems and isn’t automatically updated.

Older versions of Excel don’t allow multiple users to work together in one document, hampering collaboration. There is also a limit to how much data can be pulled into a single document, which can slow down analysis.

“Excel ...

Read More Source

November 2nd, 2017

In Technology

No Comments

Denormalization to diverse apps is the correct path forward for media companies

I am very pleased to see that the New York Times now has this cutting edge architecture, which almost certainly the right way forward at this time:

The denormalized log and Kafka’s Streams API

The Monolog is great for consumers that want a normalized view of the data. For some consumers that is not the case. For instance, in order to index data in Elasticsearch you need a denormalized view of the data, since Elasticsearch does not support many-to-many relationships between ...

Read More Source

October 31st, 2017

In Technology

No Comments

Matthew Vagnoni suggests the blockchain can resolve the issue of patient consent

Back in 2015, when I worked at Rollio, we hired Matthew Vagnoni for 2 weeks to come and teach us about Natural Language Processing. Vagnoni is a fantastic person to work with: he is a true expert in his field, yet he manifests very little ego about his extraordinary skill. He was friendly and calm and also authoritative, as he walked us through the various strategies we might take, when connecting to Salesforce, using NLP.

I only recently realized that ...

Read More Source

October 31st, 2017

In Technology

No Comments

XLSX2CSV in Java or Clojure

This page does not seem to rank very high on Google, which makes it difficult to find a good example of how to use Java to convert an Excel file to CSV. Last year, when I worked on this, I recall it took me many days of searching before I found this example.

Also good:

http://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/poi/ss/examples/ToCSV.java

Source

October 27th, 2017

In Technology

1 Comment

A developer asks for help, then gets angry at those who help them

This is pretty wild. Some idiot named “Amo” asks why Supervisord is using 100% of the CPU. A fellow named Paul Calabro then gives what sounds like the correct answer. Instead of saying “Thank you” Amo then threatens to downvote Calabro. Either Amo is vicious or they completely misunderstood the answer. Either way, Amo is an idiot.

If I was Amo, I would delete the question, and thus make it private, because the whole thing is embarrassing. In case ...

Read More Source

October 25th, 2017

In Technology

4 Comments

Why would anyone choose Docker over fat binaries?

[ [ In this essay, I use the phrase "fat binary" to refer to a binary that has included all of its dependencies. I am not using it to refer to the whole 32 bit versus 64 bit transition. If I was only writing about the world of Java and the JVM, I would have used the word "uberjar" but I avoided that word because I also wanted to praise the Go language and its eco-system. ] ]

They are ...

Read More Source

October 24th, 2017

In Technology

No Comments

You can set environment variables from the Supervisord config

I often tell clients to consolidate their config files, from all their various apps, inside of a folder inside of /etc (I assume Linux servers).

But an interesting alternative is to set the config in the Supervisord config, and thus consolidate all logging and config info in one place.

[program:django] environment = SITE=domain1, DJANGO_SETTINGS_MODULE=foo.settings.local, DB_USER=foo, DB_PASS=bar command = python manage.py command

One could simply have different entries for development, certain branches, ...

Read More Source

October 24th, 2017

In Technology

No Comments

Infinite scroll is a terrible idea 99% of the time

Pinterest introduced the modern infinite scroll, so women would keep scrolling their home page. For a while, Pinterest was the fastest growing large site in the world. Infinite scroll is useful if you’re simply encouraging people to browse over random items. It is a terrible design idea if you are trying to communicate information. As Edward Tufte says, long lists are fantastic if you are trying to communicate choices or many types of information.

So check out the Web ...

Read More Source

October 15th, 2017

In Technology

No Comments

The resulting passive-aggression kills productivity faster than any methodology or technology decision

These are the truest 6 paragraphs that I’ve ever read about software development ideologies:

Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results.

In his 2003 thesis People and methodologies in ...

Read More Source

October 15th, 2017

In Technology

No Comments

What are the benefits of the fictions that guide software development?

Very interesting and well written:

I’m not going to beat up on any of these paradigms, because what’s the point? If software methodologies didn’t exist we’d have to invent them, because how else would we work together effectively? You need these fictions in order to function at scale. It’s no coincidence that the Agile paradigm has such a quasi-religious hold over a workforce that is immensely fluid and mobile. (If you want to know what I really think about software ...

Read More Source

October 11th, 2017

In Technology

No Comments

NLP challenges: word ordering reverses meaning

Another good example:

When words are used in conjunction with each other, they express an idea that is bigger or more meaningful than each word in isolation. The two clauses I’m not happy I’m working and I’m happy I’m not working contain the sames words, in close proximity, but have quite different meanings.

Source

October 5th, 2017

In Technology

No Comments

AWS does not protect you from devops

Once upon a time, people made fun of Linux GUIs because they would give you a graphical way to edit a configuration file, but you still had to know everything in the configuration file, and what the settings did. People made the reasonable point that slapping a pretty graphical interface on a configuration file did not make it any easier to work with Linux.

Nowadays, the clients I interact with say things like “We want to use AWS because then ...

Read More Source

September 20th, 2017

In Technology

No Comments

Where should a spec be defined relative to a function

This is an interesting conversation on the Clojure mailist, and this question is relevant for more than just Clojure. It touches on the main question of language design.

Didier feels it should be possible to define the spec with the function, and this functionality should be added to the Clojure core, so that this would become idiomatic Clojure:

Gary Trakhman argues that this would make the core “defn” macro too slow, and therefore everything about spec should be left to ...

Read More Source

September 18th, 2017

In Technology

No Comments

How to use Zach Tellman’s Clojure library Manifold

These examples are great, and I wish this was more highly ranked in Google, so I’m linking to it.

Source

September 11th, 2017

In Technology

No Comments

Niche facts that Google can’t find

Google works so well that I’m shocked when it doesn’t work. I recall reading an early RFC, from the early 1970s, when someone proposed allowing variable length bytes. I tried searching Google for this. No luck. Google only shows me the famous RFCs, not the failed experiments.

Source

September 11th, 2017

In Technology

No Comments

Why can’t modern tools match Smalltalk?

I often have the impression that Smalltalk in the late 80s and early 90s hit a peak that has yet to be matched. Great effort was instead put toward languages that are less intelligent, and which limit the computer programmer. Does this have something to do with the deskilling of computer programming, as Stanislav says?

“It amazes me just how blindly complacent programmers have been in the face of the ongoing and very successful deskilling of their profession. ...

Read More Source

September 8th, 2017

In Technology

No Comments

The limitations of decision trees

This article has some nice setups with the visuals:

If you are going to buy a house, you will make a list of pros/cons. That much is common sense. It is also an implicit decision tree. Most of these concepts are common sense. But I am struck, yet again, and how quickly this can seem to be unimaginably complex, once you involve the abstract jargon of math:

And once again, I note the sheer awfulness of math notation. How much faster ...

Read More Source

September 8th, 2017

In Technology

No Comments

Gini Impurity is quick to measure and easy to adapt to

I’m trying to convince a client that we need to match records using a Random Forest approach. They have hit the limits of what can be done with simple string matching. I like basic articles like this, for explaining things clearly, and making it obvious how easy Random Forests can be (especially compared to Neural Nets).

Which is the better split? This is a subjective question. In practice, people use different metrics for evaluating splits. The most commonly used metric ...

Read More Source

September 8th, 2017

In Technology

No Comments

Logs are the future of software

Not sure why it has taken so many decades to become obvious, but clearly a history of all events is better than a relational database for keeping track of data. There is no reason why any company should ever use a relational SQL database. The source of truth should be the log, and one’s consumers should build their own denormalized datasets — and these will often work best in a document store, rather than a relational database.

I am very ...

Read More Source

September 7th, 2017

In Technology

No Comments

Peter Williams on Humane Interfaces

Still caught by the nostalgia I mentioned in the 2nd previous post, I re-read Peter Williams post, linked via the same debate. I recall reading this in 2005.

Martin Fowler has posted a nice article on humane interface design (as opposed to minimal interface design). I am definitely on the side of right and good (read: humane interfaces) in this debate. Nothing takes the fun out of programming faster than having to write a bit of code that you know ...

Read More Source

September 7th, 2017

In Technology

No Comments

The failure of the X Window system paved the road for the rise of the Web

This is an amazing article about the absolute failure of the X Window system. Copyright (C) 1994, so this was written just as the Web was gaining momentum, but before it was obvious that the Web would become the main way that different devices could establish a single interface. Among the many good bits, there is this:

Figuring out where a particular resource value is for a running application is much fun, as resource can have come from any of ...

Read More Source

September 7th, 2017

In Technology

No Comments

The loneliness of advanced skills

I decided to re-read The departure of the hyper-enthusiasts. I’ve linked to that essay many times before. It was formative for me, at a formative moment for the Web, so it was formative in many ways, and it captured something important about that moment, and how things were changing.

I find myself getting nostalgic. That is not interesting in and of itself. To be nostalgic for a formative time, about a formative essay, is ordinary enough.

But I had an ...

Read More Source

September 7th, 2017

In Technology

No Comments

Blue or Not Blue decision trees

This is a point worth remembering:

“Before we get started I need to clarify something. Theoretical decision trees can have two or more branches protruding from a single node. However, this can be computationally expensive so most implementations of decision trees only allow binary splits.”

Source

September 7th, 2017

In Technology

No Comments

If NodeJS is so great, why are the problems so common?

Callback hell. Take a deep breath and ask yourself why this website needs to exist:

Source

August 12th, 2017

In Technology

No Comments

Supervisord has spent 5 years discussing whether or not it should support a timing mechanism?

I’m not sure what to make of this incredible thread in the issue tracker on Github for the Supervisord repo. Someone asked, in 2012, for a way to time when various apps start, so that they could start app A, B and C in order. On April 16 of this year, 2017, someone wrote:

This issue has been open since 2012, does anyone have ownership over it? It’s clearly important to many users; is there any consensus amongst supervisor contributors?

I ...

Read More Source

August 4th, 2017

In Technology

No Comments

A pure GUI for IP/TCP, a technology with a single mandate, with no burden of also offering semantics or structure or hierarchy

Someone on Hacker News said most developers don’t “get” the Web, and I posted this in response:

My history is surprisingly similar to yours, I started in 1999, I used Notepad as my first text editor, and by 2003 I got caught up in the movement towards making markup strict, which I felt was the mark of professionalism. However, by 2006 I had mostly rejected the notion of “strictness”. There were several things that turned me against strictness. One of ...

Read More Source

August 2nd, 2017

In Technology

No Comments

Cultural differences between Clojure and NodeJS

Right now I have to work with NodeJS, because I have a client who uses NodeJS. I am getting used to the asynchronous style of NodeJS. I notice a subtle cultural difference between Clojure and NodeJS. Most of the tutorials for NodeJS assume the person reading is a beginner, someone who may not understand big complex words like “asynchronous”. In the world of Clojure, there is the assumption that the person reading the article has some experience. With Clojure, there ...

Read More Source

July 24th, 2017

In Technology

No Comments

NodeJS remains single threaded

This is a good example of the limits of NodeJS:

async.parallel([ function(callback){ setTimeout(function(){ callback(null, 'one'); }, 200); }, function(callback){ setTimeout(function(){ callback(null, 'two'); ...
Read More Source

July 19th, 2017

In Technology

No Comments

No one should ever use NodeJS, part MMCVXII

I previously wrote about a very surprising bug I discovered in NodeJS/HapiJS.

But today I face a new issue: How to parallelize work in NodeJs? I discovered this fantastic library called paralleljs. It assigns work to child processes. Fantastic idea, very similar to Python. And how do I get required libs into the child process? That is very difficult. From the issues on Github:

to make things a bit more concrete, this snippet would be representative of what I want ...

Read More Source

July 18th, 2017

In Technology

No Comments

Do people love DigitalOne servers?

I’m looking to host my next web site at DigitalOne. If you have experience with them, please let me know.

Source

July 18th, 2017

In Technology

No Comments

When Nginx becomes the bottleneck

This makes me sad. I love Nginx as a reverse proxy, so long as it is invisible and I never have to think about. Realizing that it, too, can be a source of problems really is discouraging.

47,135 connections in TIME_WAIT! Moreover, ss indicates that they are all closed connections. This suggests the server is burning through a large portion of the available port range, which implies that it is allocating a new port for each connection it’s handling. Tweaking ...

Read More Source

July 18th, 2017

In Technology

No Comments

Trying to explicitly enumerate everything that is true is hopeless

This is good:

Let’s take a look at some existing representations. The most famous representation is WordNet. In WordNet, the symbols are groups of words that have the same meaning, called synsets. One synset could be the set consisting of “car” and “automobile.” Each word can be in multiple synsets. For example, “bank” could be in the synset that means river bank and also in the synset that means a place where money is deposited. There are a few kinds ...

Read More Source

July 8th, 2017

In Technology

No Comments

A simple intro to tokenizing with OpenNLP in Clojure

A nice intro:

Finding sentences

Words (tokens) aren’t the only structures that we’re interested in, however. Another interesting and useful grammatical structure is the sentence. In this recipe, we’ll use a process similar to the one we used in the previous recipe, Tokenizing text, in order to create a function that will pull sentences from a string in the same way that tokenize pulled tokens from a string in the last recipe.

Getting ready

We’ll need to include clojure-opennlp in our project.clj file:

(defproject com.ericrochester/text-data ...

Read More Source

July 8th, 2017

In Technology

No Comments

Neural Networks beat Support Vector Machine beat Random Forests

I’ve written about my experience at a startup that used NLP to translate a salesperson’s text message into an entry in Salesforce. At the time, our NLP developer used the Stanford NLP library to try to build a model, but mostly they used a lot of regex and string matching.

I’ve been thinking about how we might have done that project faster and better. I’ve recently been thinking the right approach would have been Random Forests. We were lucky ...

Read More Source

July 8th, 2017

In Technology

No Comments

The struggle to find meaning in words, via word counts in matrixes

Interesting comment:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is the ...

Read More Source

July 8th, 2017

In Technology

No Comments

The problems with DynamoDB

Everything I’ve read suggests to me that companies are drifting towards Cassandra as the best of the NoSQL databases. But some of my clients give me directives such as “Please restrict yourself to AWS technologies”. So I was considering using DynamoDB. And then I read this.

Interesting:

When a customer does an action, it is reasonable to think that this kicks off a number of actions in your system. One action might spawn a number of events to be sent into ...

Read More Source

June 29th, 2017

In Technology

No Comments

The curse of too many dimensions

Damn, this is interesting:

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic optimization.

There are multiple phenomena referred to by this name in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining, and ...

Read More Source

June 29th, 2017

In Technology

No Comments

Morphism is the starting point of category theory

I guess I knew this, but still it is interesting to read:

In algebra, a homomorphism is a structure-preserving map between two algebraic structures of the same type (such as two groups, two rings, or two vector spaces). The word homomorphism comes from the ancient Greek language: ὁμός (homos) meaning “same” and μορφή (morphe) meaning “form” or “shape”.

Homomorphisms of vector spaces are also called linear maps, and their study is the object of linear algebra.

The concept of homomorphism has been ...

Read More Source

June 29th, 2017

In Technology

No Comments

All the weird characters in Clojure explained

This is a very useful guide.

Source

May 31st, 2017

In Technology

No Comments

Cost cutting at British Airways caused the catastrophic failure of its database service

This is obviously a lie:

BA insisted that the computer system outages that grounded hundreds of flights were not linked to ferocious cost-cutting and the outsourcing of work to contractors in its IT department.

There is absolutely no way to get a melt down of this scale unless you’ve just fired your key staff, so you’ve nobody left who knows how to bring the system back online. I’ve worked at very small startups, with just 3 people on staff, and we ...

Read More Source

May 27th, 2017

In Technology

No Comments

Is Clojure dying?

Interesting.

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure ...

Read More Source

May 9th, 2017

In Technology

No Comments

Don’t use PM2 with NodeJS

Interesting:

I don’t use any of the “node” process managers because they don’t solve the problem of starting your process at server boot. I use init scripts, or upstart, or runit or systemd like you’re supposed to for a service. Anyone using forever or pm2 likely doesn’t understand sysops.

Source

May 5th, 2017

In Technology

No Comments

Very long GC pauses when NodeJS is about to die from memory exhaustion

I sometimes see stuff like this in my logs, when my NodeJS app has reached critical:

856870 ms: Mark-sweep 5647.1 (6029.8) -> 5625.8 (6029.8) MB, 10733.9 / 0.0 ms [allocation failure] [GC in old space requested]. 867774 ms: Mark-sweep 5625.8 (6029.8) -> 5625.8 (6030.8) MB, 10903.4 / 0.0 ms [allocation failure] [GC in old space requested]. 878580 ms: Mark-sweep 5625.8 (6030.8) -> 5625.8 (5996.8) MB, 10806.3 / 0.0 ms [last resort gc]. 889353 ms: Mark-sweep ...
Read More Source

April 26th, 2017

In Technology

No Comments

More NLP for Clojure

This looks fantastic:

To do its magic, postagga extracts the phrase structure of your input, and tries to find how do this structure compare to its many semantic rules and if it finds a match, where in this structure shall he extract meaningful information.

Let’s study a simple example. Look at the next sentence:

“Rafik loves apples” That is our “Natural language input”

First step in understanding this sentence is to extract some structure from it so it is easier to interpret. One common way ...

Read More Source

April 26th, 2017

In Technology

No Comments

TCP can fail in many, many ways

Amazing that the world depends on a technology with so many quirks:

So, if we read that data first, and LINGER, are we good to go? Not really. The close() call really does not convey what we are trying to tell the kernel: please close the connection after sending all the data I submitted through write().

Luckily, the system call shutdown() is available, which tells the kernel exactly this. However, it alone is not enough. When shutdown() returns, we still have no indication ...

Read More Source

April 23rd, 2017

In Technology

No Comments

Thinking about Concurrency, Raymond Hettinger, Python core developer

A basic overview of concurrency, most of which I knew, but he had one good line at the end: “If you add enough locks to your concurrent code, it is no longer concurrent. It is completely sequential, you’ve simply made it a lot more complicated than single threaded code.”

Source

April 13th, 2017

In Technology

No Comments

Python has greenlets

I’m sad that Python is doing so well. It gets all the attention that should go to Clojure. They are both great for AI and NLP, but Python doesn’t have as many great ideas for concurrency. Or rather, the language doesn’t, but the eco-system makes up for that with a wealth of interesting ideas — and it is sad to see so much effort go to see making Python work, rather than just starting over with something that has a ...

Read More Source

April 13th, 2017

In Technology

No Comments

Katharine Jarmul describes the bias in Google’s Word2Vec software

Interesting:

I must warn you that parts of this post are disgusting, disturbing and awful. If you are having a rough day, feel free to save for another time. If you are already sick of seeing hateful language, this is likely not a post to read at present. That said, I feel my duty as a former journalist to look at it, expose it, and hope to spark better conversations around how we handle both implicit and explicit bias and prejudice ...

Read More Source

April 11th, 2017

In Technology

No Comments

Why are people ignoring the problems with NodeJS?

Two weeks ago I wrote “A surprising NodeJS failure mode: deterministic code becomes probabilistic under load“. Since then I’ve been reading up on NodeJS and learning more about its substantial failure modes. I’m left feeling very surprised at the success that NodeJS is having.

Just to give you a sense of what I mean, this is how The New Stack sums up the success of NodeJS:

Ready for a Long Term Node Relationship?

In just seven short years, Node.js has gone ...

Read More Source

March 30th, 2017

In Technology

No Comments

A surprising NodeJS failure mode: deterministic code becomes probabilistic under load

I have a client. Let’s say they run some scripts that scrape the web for data, then they run some NLP scripts to pull out the facts they need, then they need to insert that into their MySQL database. When they gather facts, they need to know if the facts are about an organization that is already in the database. So they asked me to create an API that could take their new facts, and try to match those facts ...

Read More Source

March 29th, 2017

In Technology

No Comments

Skip Lists: A Probabilistic Alternative to Balanced Trees

Interesting:

Skip lists are a data structure that can be used in place of balanced trees. Skip lists use probabilistic balancing rather than strictly enforced balancing and as a result the algorithms for insertion and deletion in skip lists are much simpler and significantly faster than equivalent algorithms for balanced trees

A node that has k forward pointers is called a level k node. If every (2i)th node has a pointer 2i nodes ahead, then levels of nodes are distributed in a simple pattern: 50% are level 1, 25% ...

Read More Source

March 29th, 2017

In Technology

No Comments

Novelty seeking multi agent systems

Interesting:

This paper considers novelty-seeking multi-agent systems as a step towards more efficient generation of creative artifacts. We describe a simple multi-agent architecture where agents have limited resources and exercise self-criticism, veto power and voting to collectively regulate which artifacts are selected to the domain i.e., the cultural storage of the system. To overcome their individual resource limitations, agents have a limited access to the artifacts already in the domain which they can use to guide their search for novel artifacts. Creating geometric images called spirographs as a case study, we show ...

Read More Source

March 29th, 2017

In Technology

No Comments

Bushy join trees for snowstorm queries

I am taken aback by the amount of jargon here:

Many workloads for analytical processing in commercial RDBMSs are dominated by snowstorm queries, which are characterized by references to multiple large fact tables and their associated smaller dimension tables. This paper describes a technique for bushy join tree optimization for snowstorm queries in Oracle database system. This technique generates bushy join trees containing subtrees that produce substantially reduced sets of rows and, therefore, their joins with other subtrees are generally much ...

Read More Source

March 29th, 2017

In Technology

No Comments

How to combine novelty search with fitness-based evolution

I need to go back and read this whole article:

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It is still unclear, however, how some key parameters and algorithmic components influence the evolutionary dynamics and performance of novelty search. In this paper, we conduct a comprehensive empirical study focused on novelty search’s algorithmic components. We ...

Read More Source

March 29th, 2017

In Technology

No Comments

Novelty search

Interesting:

Novelty search is a recent algorithm geared toward exploring search spaces without regard to objectives. When the presence of constraints divides a search space into feasible space and infeasible space, interesting implications arise regarding how novelty search explores such spaces. This paper elaborates on the problem of constrained novelty search and proposes two novelty search algorithms which search within both the feasible and the infeasible space. Inspired by the FI-2pop genetic algorithm, both algorithms maintain and evolve two separate populations, ...

Read More Source

March 27th, 2017

In Technology

No Comments

What every object on a network needs

There is no need to rely on the Web for these things:

The problem with operations is that we have to define, a-priori, the semantics of each individual operation. We cannot tell, unless we know beforehand, whether the operation is safe to call multiple times, whether the result of an operation will be the same every time we call it, whether the result can be cached, and if so for how long. Many years of building distributed applications have told ...

Read More Source

March 27th, 2017

In Technology

No Comments

Use pseudo URIs for an id

A long article, but this is interesting:

When facing the problems above, my team at SoundCloud started exploring alternatives that would allow for us to have simple, scalar values that were still rich enough to act as good identifiers across our hundreds of microservices. Reading through decades of industry work on the matter, we found something simple that could help us: Uniform Resource Names, or URNs. URNs were a type of Uniform Resource Identifiers (URIs) that, as opposed to URLs, were ...

Read More Source

March 27th, 2017

In Technology

No Comments

Is NewSQL the next big thing?

I like the fact that Kinesis has an SQL interface:

I can see this being useful for Business Intelligence. But that seems like a niche to me. Both SQL and NoSQL solved big, universal problems.

But maybe the SQL is just window dressing? Maybe the real breakthrough is a distributed system with strong consistency guarantees? It would be a very big deal if someone found a way around the CAP Theorem. But otherwise, NewSQL is just more bad marketing for ...

Read More Source

March 27th, 2017

In Technology

No Comments

Designing a neural net is really hard

Interesting:

2bitencryption says:

Designing a neural network is a thousand times harder than I imagined.

After AlphaGo, I tasked myself with creating a neural network that would use Q-Learning to play Reversi (aka Othello).

At that point, I had already utilized Q-Learning (the tabular version, not using a neural network) for some very simple and mostly proof-of-concept projects, so I understood how it worked. I read up only perceptrons, relu, the benefits/disadvantages of having more/fewer layers, etc.

Then I actually started on the project, thinking ...

Read More Source

March 26th, 2017

In Technology

No Comments

Training a neural net

A great intro:

Training

Training is how we teach a neural network what we want it to learn. It follows a simple five step process:

1.) Create a training data set, which we will call x and load its labels as targets y

2.) Feed the x data forward through the network with the result being predictions y’

3.) Figure out the “loss” of the network, which is the difference between the predictions y’ and the correct targets y

4.) Compute the “gradient” of the loss ...

Read More Source

March 26th, 2017

In Technology

No Comments

Transform real data into a tensor, that is, tensor flow

A good introduction:

Source

March 17th, 2017

In Technology

No Comments

What if we had computer bits that allowed more than two values?

The Red-Black tree is interesting because it can store the color using only bit, and one bit can have one of two values, often represented as either a 0 or a 1. That is the nature of our silicon hardware. But if we had hardware where the basic charge could be one of several values, then it would possible to do multi colored trees. If we had hardware that could hold one of 20 values, it would be possible to ...

Read More Source

March 17th, 2017

In Technology

No Comments

An InputStream requires more memory as it has to buffer the whole file

This is exactly the opposite of what I’ve always understood, and what I’ve read elsewhere:

When opening a workbook, either a .xls HSSFWorkbook, or a .xlsx XSSFWorkbook, the Workbook can be loaded from either a File or an InputStream. Using a File object allows for lower memory consumption, while an InputStream requires more memory as it has to buffer the whole file.

The stream requires more memory? That is crazy. It’s a stream. And what is a File object if not ...

Read More Source

March 8th, 2017

In Technology

No Comments

Joe Armstrong: computer programming is the alteration of values in a key value database

An incredible assertion that would be dismissed if it was coming from someone less talented than Joe Armstrong. It is interesting to think that RAM is a key value store, and so anything that updates RAM is altering the values of a key value store.

He starts by saying that he often ends up with miscellaneous code:

When I’m programming I often get to the point were I say there shoulda function foo/2 in lists.erl but their isn’t. There should be ...

Read More Source

March 1st, 2017

In Technology

No Comments

How to get Nginx to wait for your app

I’ve been struggling with a slow NodeJS app which is behind a reverse proxy handled by Nginx.

I found this bit useful:

Syntax: proxy_max_temp_file_size size; Default: proxy_max_temp_file_size 1024m; Context: http, server, location When buffering of responses from the proxied server is enabled, and the whole response does not fit into the buffers set by the proxy_buffer_size and proxy_buffers directives, a part of the response can be saved to a temporary file. This directive sets the maximum size of the temporary file. The size of data written to ...
Read More Source

February 28th, 2017

In Technology

No Comments

How do we teach self-driving cars to avoid hitting people on bicycles?

This is a good conversation:

ChuckMcM :

“Now Johnny, when you ride your bike you must wear your I-am-a-bike vest and follow these patterns or the cars are likely to kill you.” :-) Summary is that “people riding bikes” (PRB) is a much denser image set than “people in cars” (PIC) and “people walking or jogging” (PWJ), and the PRB objects have a much higher dynamic angular vector capability (they can change direction extremely quickly) combined with a wider dynamic velocity vector ...

Read More Source

February 14th, 2017

In Technology

No Comments

Different NLP approaches

A nice summary:

With LSA, each document is transformed into a single vector that has the length of the vocabulary. The length of the vocabulary is the number of unique words across all documents. If a word is present in a document, it is represented as a 1 in the vector and 0 if it is not. So after this transformation, the text is transformed in an D by V matrix where D is the number of documents and V is ...

Read More Source

February 11th, 2017

In Technology

No Comments

Tony Arcieri’s very good defense of Rails

I just had a project where I had to write an app that could fire several million requests against an API, with data pulled from a database. I wrote the whole thing in NodeJS and HapiJS, and it was an agony. The uncontrolled async of NodeJS tripped me up. If I need to fire 10 million HTTP requests, how do I do that without blowing the stack? As it was, the functions would pile up, and then I’d get a ...

Read More Source

February 7th, 2017

In Technology

No Comments

systemd is no longer interested in the grubby details of reality

Interesting:

Unfortunately, since then systemd developers have shown an unfortunate and increasing streak of idealism. More and more, systemd seems not to be interested in dealing with the world as it actually is, with all of its grubby inconvenient mess; instead it simply assumes some pure and ideal version of things. If the world does not measure up to how it is supposed to be, well, that is not systemd’s problem. Systemd will do the nominally right thing no matter how ...

Read More Source

January 28th, 2017

In Technology

No Comments

NLP as a simple four-step formula: embed, encode, attend, predict

Interesting:

Let’s say you have a great technique for predicting a class ID from a dense vector of real values. You’re convinced you can solve any problem that has that particular input/output “shape”. Separately, you also have a great technique for predicting a single vector from a vector and a matrix. Now you have the solutions to three problems, not two. If you start with a matrix and a vector, and you need a class ID, you can obviously compose the ...

Read More Source

January 27th, 2017

In Technology

No Comments

Paul Phillips denounces Scala

Pacific Northwest Scala 2013 We’re Doing It All Wrong by Paul Phillips For a computer programming tech conference, this talk has a surprising amount of emotional rawness. Phillips is denouncing Scala both on a personal level and on a technical level.

Source

January 27th, 2017

In Technology

No Comments

One team that gave up on Scala

It’s interesting how Scala was initially seen as “A Better Java” but now is seen as having been to experimental in its approach.

I remember when I first saw the potential issues of scaling Scala at Gravity back in 2009/10ish. It was close to the end of the day when we had a major issue reported in production that was affecting our large customers. Several of us started investigating and were able to track the source of the issue. The ...

Read More Source

January 27th, 2017

In Technology

No Comments

Scale programmers do not trust Martin Odersky

This whole conversation is incredible. Here is a forum full of Scala programmers, and they seem to be talking against Odersky’s proposal. And with some anger. I’ve never seen a Rubyist talk about Matz like that. I’ve never heard a Clojurist talk about Rich Hickey that way. Java programmers are, at most, muted in their criticism of Gosling, with the overall recognition that “mistakes were made” but made in good faith. I suppose you could argue that Pythonists are split ...

Read More Source

January 27th, 2017

In Technology

No Comments

The failure of hierarchy based on category

This is good:

The Hierarchy Problem

Every time I start at a new company, I struggle with the problem when I’m creating a place to put my Company Documents, e.g. the Employee Handbook.

Do I create a folder called Documents and then create a folder called Company in that?

Or do I create a folder called Company and then create a folder called Documents in that?

Both work. But which is right? Which is best?

The idea of Categorical Hierarchies was that there were Base Classes ...

Read More Source

January 27th, 2017

In Technology

No Comments

Logicians hate randomness

My strongest subject was always logic, even though I was terrible at most other forms of math. And though I have doubts about randomness, I certainly love inconsistency, which can be approached in a logical way. Every computer programmer who writes concurrent code has to deal with inconsistent data, and it would be great if logicians thought more deeply about how to formalize this.

With Gödel, at first there was a lot of shock. As a student in the late ...

Read More Source

January 24th, 2017

In Technology

No Comments

Movio.co decides to switch from Scala to Go, and they reference my essay

Since they cite me (and my essay from 2014) as part of their decision making process, I want to throw in 2 cents here. They ended up deciding on Go, whereas I have ended up preferring Clojure, yet I agree with a lot of what they say, so I’ll try to clarify why I ended up with a different decision than what they made.

I understand what they mean when they write:

I think the first time I appreciated the positive ...

Read More Source

January 18th, 2017

In Technology

No Comments

Modern programming depends on DSLs, and Lisp is the best as DSLs

Anders Hovmöller seems to be stuck with a mentality shaped by the C++ language, and so he doesn’t get the important of DSLs.

In C++ land it’s well known that you must keep to a subset of the language in a code base to keep sane. It’s also known that every time you increase the size of the subset you are making the situation worse even if it solves a small problem at hand.

In Lisp land on the other hand, all ...

Read More Source

January 8th, 2017

In Technology

No Comments

This year’s goal: move away from WordPress

Like a lot of people, I first used WordPress because it was quick and easy to get going, and because my designer knew it very well. But I dislike the dependence on MySQL, I dislike its slowness, and I dislike the security problems that I’ve had (this site has been hacked twice).

For my modest goals, I only need a static site, so I plan to switch to Cryogen. I was interested by this article:

For a while I ...

Read More Source

January 8th, 2017

In Technology

No Comments

Different people have different strengths

I love this :

I’ve had similar self-doubt in the past, I considered myself fast but sloppy. I changed my opinion after a workshop on different team roles at my previous job. A lot of it was boring workshop fluff, but I loved the core message: that many personality traits aren’t purely positive or negative. Perfectionists are nitpickers. Fast developers are sloppy. Experienced ones overthink stuff. Bleeding-edge evangelists ruin long-term stability etc. There are two sides to every coin. We tend to ...

Read More Source

January 8th, 2017

In Technology

No Comments

Can event logs, as a software architecture, really work?

This is an interesting bit of criticism of this architecture, an architecture which certainly has gotten a lot of attention over the last 4 years:

I have worked on, or cleaned up, 4 different CQRS/ES projects. They have all failed. Each time the people leading the project and championing the architecture were smart, capable, technically adept folks, but they couldn’t make it work.

There’s more than one flavor of this particular arch, but Event Sourcing in general is simply not very ...

Read More Source

December 15th, 2016

In Technology

No Comments

Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

I posted this question to StackOverflow and I would appreciate any help figuring out what the memory issue is:

Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

Source

December 14th, 2016

In Technology

No Comments

Ugly engineering

This sounds hideous:

I used to work on the DynamoDB team. Throwaway account because my normal account can be tied back to my real name.

“Each hash key resolves to a number of possible servers the data can be on. Data is replicated across several of these servers. For redundancy. The hash key determines which shard to use. On individual machines, each set of data is stored by a compound key of hash key and sort key (if there is a sort ...

Read More Source

December 13th, 2016

In Technology

No Comments

Beautiful engineering

Wow. Sometimes I witness some engineering that leaves me feeling joy and inspiration. I felt that way about the Golden Gate Bridge. But sometimes I feel that way about code. For instance, right now, I just learned that the defun macro for Clojure allows this function for calculating a fibonacci number.

A fibonacci function: (defun fib ([0] 0) ([1] 1) ([n] (+ (fib (- n 1)) (fib (- n 2))))) Output: (fib 10) ;; 55

That ...

Read More Source

December 7th, 2016

In Technology

No Comments

The crazy process of trying to release a Salesforce app

Right now I’m helping a client release an app for Salesforce in the Salesforce App Exchange. The process for releasing an app is completely insane and very poorly documented (or rather, there is an abundance of documentation for various parts of the process, but there is no summary of the process). I was lucky to stumble upon this post, which offers the nearest thing to a summary of the process:

1) Create a Dev Org

2) Become a Salesforce.com Partner

3) Create Test ...

Read More Source

November 26th, 2016

In Technology

No Comments

Clojure for XML

This is a fantastic overview of different approaches:

Zippers are probably the easiest way to manage xml – once you grok them.

Zippers are a strange beast. Wikipedia describes them as:

A technique of representing an aggregate data structure so that it is convenient for writing programs that traverse the structure arbitrarily and update it’s contents…

I like to think of a zipper as a kind of pointer to part of a tree – at any time if you have a tree of nodes ...

Read More Source

November 26th, 2016

In Technology

No Comments

How far can code be optimized

This is funny and interesting:

When you’re competing with another player, they will probably find a way to beat your score by just a few points. Let’s say my score is 340 and a friend beats me with a score of 335. (lower is better. The score is just the number of executed instructions) What follows is a bunch of head-scratching about how you could possibly get any more cycles out of the algorithm. After an hour of staring and trying ...

Read More Source

November 14th, 2016

In Technology

No Comments

NLP with Clojure and OpenNLP

This is a fantastic introduction:

Natural Language Processing (NPL) opens the door to the possibility of turning otherwise inert text into meaningful or, more interestingly, actionable information. It is the latter that I am interested in and what this installment will focus on. I will explore the basics of NLP using the OpenNLP library and Clojure to convert a sentence into a useful structure to store or act on. More specifically, my goal is to take simple sentences that indicate the ...

Read More Source

November 13th, 2016

In Technology

No Comments

The best interview technique: pay people $200 to attempt some task over the weekend

I like this idea:

In regards to what works the best, I found that these 2 ideas work the best when combined.

PAID Sample project assignment (err on the side of paying fairly — say $100+/hour for estimated completion time — if the problem should require 2 hours to complete, offer $200)

Bring the candidate in and discuss the solution. Let the candidate talk about their design decisions, challenge them as you would any team member and let them provide their reasoning.

Paying candidates to work on a ...

Read More Source

November 5th, 2016

In Technology

No Comments

The perfect Javascript file uploader?

This looks amazing:

The default behavior of Fine Uploader is to immediately attempt to upload files as they are selected. One option allows you to simply queue all files, and then start uploading at a later time by calling uploadStoredFiles() on your Fine Uploader instance. You can also easily allow your users to edit the names of each submitted file before uploading.

This Fine Uploader instance below also demonstrates the edit file name feature, which allows you to edit the name of ...

Read More Source

October 31st, 2016

In Technology

No Comments

Stripe almost got burned by Bitcoin

It’s amazing how many disasters seem to happen whenever companies try to use crypto-currency:

For a variety of reasons, it is sometimes necessary to refund bitcoin transactions: For example, a customer cancelling their order; accidentally sending in the wrong number of bitcoins; or even sending in the correct number of bitcoins, but not within the requisite time window, resulting in their value being lower than necessary. Consequently, Stripe allows for bitcoin transactions to be refunded — with the caveat that, for ...

Read More Source

October 27th, 2016

In Technology

No Comments

Software doesn’t need options

At least for the consumer. An API for a developer is something else completely.

Great software is:

Opinionated Options are a cop out. It’s the designer’s job to figure out the best way to use the product, and delegating that choice to the user is pure laziness. Hive only resorts to options as a last resort.

Source

October 3rd, 2016

In Technology

No Comments

How process becomes bloated and bureaucratic over time

This is good:

Checkout/Ordering team: “Your Payment Processing release broke Checkout in production.”

Payment Processing team: “Didn’t know this would affect Checkout. Had no time to look into it because we were too busy working on the new payment functionality in the iOS app.”

Mobile App Product Visionary: “I want my own dedicated team. This happened because Payment Processing is spread too thin to work on the app and the backend at the same time.”

Development Manager: “Org chart change – let’s have all ...

Read More Source

September 18th, 2016

In Technology

No Comments

Rapidly Exploring Random Trees

Interesting. It says the RRT proceeds by breaking the Voronoi spaces into smaller Voronoi spaces. The algorithm has a bias towards unexplored territory, unlike a purely random algorithm, which tends to loop back to explored space. I didn’t dig into the algorithm, but I can imagine how this would be useful in some situations.

Source

September 18th, 2016

In Technology

No Comments

As a consolidation client for Kinesis, ZeroMQ is much better than Kafka

This is fantastic:

With the problem of real-time log consolidation solved with ZeroMQ, supporting durable storage of logs for later post-processing becomes a smaller and a much more manageable challenge. For example, off the shelve solutions like Logstash are capable of capturing data from ZeroMQ and publishing it to a variety of destinations. In our particular situation at Auth0, we are already using AWS Kinesis, ElasticSearch, and Kibana as a log processing pipeline in other parts of our operations. As such we ...

Read More Source

September 18th, 2016

In Technology

No Comments

I always learn what I need to know about a year too late

At Celolot I wrote a bunch of Clojure apps that talked to each other over Redis thanks to Carmine/Nippy. We passed around maps, and each app checked a field in the map to see if the map was a message that it was suppose to respond to.

But I bet this is a thousand times faster:

ZeroMQ does not have a first class notion of a topic, yet it does have a first class concept of a subscription filter. Subscription filters ...

Read More Source

September 18th, 2016

In Technology

No Comments

Camille Fournier on ephemeral nodes in etcd

I missed this the first time. In etcd a service needs to constantly update the fact that it is alive. If it dies, its info disappears. So other services can contact etcd and ask “Who is alive right now?”

ZooKeeper allows stateful connections. Thick clients. But what happens when you try to use ZooKeeper from a language that does not allow threads? Do you block on the only thread that you have?

For pub-sub she recommends RabbitMQ or ZeroMQ.

Source

September 18th, 2016

In Technology

No Comments

Why would I ever want to use Docker?

I’m trying to be open minded about Docker, but it is a struggle to grok why this is good. Do I want to bind all dependencies to my app? Great, then I’ll build uberjars that combine everything into 1 binary. But how will I work with Ruby On Rails? That’s easy: it is time to move away from Ruby On Rails. If Ruby can only be kept alive by using Docker, then we should give up on Ruby.

How will ...

Read More Source

September 18th, 2016

In Technology

No Comments

Unary Numeral System

Interesting:

Addition and subtraction are particularly simple in the unary system, as they involve little more than string concatenation. The Hamming weight or population count operation that counts the number of nonzero bits in a sequence of binary values may also be interpreted as a conversion from unary to binary numbers. However, multiplication is more cumbersome and has often been used as a test case for the design of Turing machines.

Compared to standard positional numeral systems, the unary system is inconvenient ...

Read More Source

September 18th, 2016

In Technology

No Comments

It took me 10 years to write that 5 line app

There is the story where Picasso paints a quick sketch and wants a huge amount of money for it, and the art collector says “It only took you 15 minutes to make that sketch!” and Picasso says “I spent 60 years learning how to do that in 15 minutes.”

Maybe something like this applies when it comes to microservices. When I advocate for microservices, I bring 17 years of experience to the conversation. The first 7 of those years meant ...

Read More Source

September 18th, 2016

In Technology

No Comments

For the last 10,000 years, the banking industry has relied on eventual consistency

Why do computer programmers feel smug offering the following example, when it is so clearly wrong?

Transactions, to a database, are important, because banks must keep track of money. Suppose a person were to move $100 from their Savings account to their Checking account. Suppose $100 is added to checking, and just then the electricity dies, and the computers die, before $100 can be subtracted from their Savings account. The person now has an extra $100, which they should not ...

Read More Source

September 17th, 2016

In Technology

No Comments

I keep really good company

This article by Yegor Bugayenko cites the following people:

Edsger W. Dijkstra

Alan Kay

Paul Graham

Richard Mansfield

Eric Raymond

Jeff Atwood

Linus Torvalds

Oscar Nierstrasz

Rich Hickey

Eric Allman

Joe Armstrong

Rob Pike

John Barker

Lawrence Krubner (me)

Asaf Shelly

I must be doing something right to get cited alongside Edsger W. Dijkstra, Alan Kay, Paul Graham, Linus Torvalds, and Rich Hickey.

Source

September 17th, 2016

In Technology

No Comments

When is consistency needed in a database?

This is a great comment :

Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. “Low level” operations as part of settlement may us transactions, but the bank system is “designed” (more like it has grown by accretion) to function almost entirely by settlement and reconciliation rather than holding onto any notion of consistency. The real world rarely involves ...

Read More Source

September 11th, 2016

In Technology

No Comments

Why did Salesforce use markup tags for VisualForce?

By now, I think we can agree, using HTML for the web was a mistake. But Sir Tim Berners-Lee could not have known that back in 1989. He was thinking that he could create a Semantic Web, and for that, the use of a markup language was defensible. Only in retrospect do we see that HTML was mostly used as a GUI for TCP/IP. Eventually Sir Tim Berners-Lee conceded the point, and so he went off and created RDF.

But ...

Read More Source

September 5th, 2016

In Technology

No Comments

Comparing data sets that are imbalanced

Interesting:

Research on imbalanced classes often considers imbalanced to mean a minority class of 10% to 20%. In reality, datasets can get far more imbalanced than this. —Here are some examples:

1.) About 2% of credit card accounts are defrauded per year1. (Most fraud detection domains are heavily imbalanced.)

2.) Medical screening for a condition is usually performed on a large population of people without the condition, to detect a small minority with it (e.g., HIV prevalence in the USA is ~0.4%).

3.) Disk ...

Read More Source

September 5th, 2016

In Technology

No Comments

Base 10 numbers need log(10) digits to be described

I’m sure I knew this example once, but I’d completely forgotten it, and it is so perfectly obvious when we talk to non-technical people and they ask for an example of what logarithmic growth looks like:

In mathematics, logarithmic growth describes a phenomenon whose size or cost can be described as a logarithm function of some input. e.g. y = C log (x). Note that any logarithm base can be used, since one can be converted to another by multiplying ...

Read More Source

September 5th, 2016

In Technology

No Comments

Accessing any single element in an array takes constant time as only one operation has to be performed to locate it.

This seems like it could be used as a trick question that would trip me up during a job interview:

An algorithm is said to be constant time (also written as O(1) time) if the value of T(n) is bounded by a value that does not depend on the size of the input. For example, accessing any single element in an array takes constant time as only one operation has to be performed to locate it. However, finding the minimal ...

Read More Source

September 5th, 2016

In Technology

No Comments

Beware Big O Notation in higher level languages

This essay has a nice bit of details about a general point that needs to be made:

Here we get the result that will be counterintuitive to many. No matter how large n gets, the Array List still performs better overall. In order for performance to get worse, the ratio of inserts to iterations has to change, not just the length of the collection. Note that isn’t an actual failure of Big O analysis, it is merely a common human failure ...

Read More Source

September 5th, 2016

In Technology

No Comments

The gamification of Slack makes it addictive but also makes it a drain

Very interesting:

Everything bad about Slack is fundamentally Slack’s fault. Slack’s sane default for a new user is to play a sound and send a desktop notification every time anything happens anywhere. There is no way to simply turn off the screaming red circle on the dock icon. There is no way to simply fold away the sidebar so you can focus on whatever it is you are trying to do. These UI nitpicks help us understand Slack’s conception of the ...

Read More Source

September 5th, 2016

In Technology

No Comments

It’s not even XML!

Dave Winer said he is deeply depressed. Sometimes he seems frightened by the march of change. I suppose that is an aspect of depression. His initial fear of JSON seems like an expression of fear:

I’ve been hearing, off in the distance, about something called JSON, that proposes to solve a problem that was neatly solved by XML-RPC in 1998, the encoding of arrays and structs in a format that could easily be processed by all programming languages. The advantage ...

Read More Source

September 5th, 2016

In Technology

No Comments

Finding novelty via friends is a metric not yet captured by recommendation engines

This is a good point:

The traditional regime of recommendation systems has been obsessed with (1). What’s the uplift of recommendation algorithm A vs recommendation algorithm B? Which is driving more click-thrus and conversions?

There’s something fundamentally broken in the this way of thinking though. I don’t care what the computer says. I care about what my friends say. The meaningful music I’ve discovered over the last 10 years has been music liked by someone I respect. Friends. Other musicians I ...

Read More Source

September 5th, 2016

In Technology

No Comments

Detecting voting rings with HyperLogLog

While this seems like a clever trick, I typically want a lot of metrics regarding voting, so throwing away the metadata doesn’t seem like an option to me. As an aggregate tool whose only purpose is finding voting rings, maybe this useful maybe?

But consider what would happen if we created a HyperLogLog counter for every user on Reddit, and any time that a user receives an upvote, we update the corresponding HyperLogLog counter with the id of the user ...

Read More Source

September 5th, 2016

In Technology

No Comments

Great browser software in 1996

These technologies sound so amazing, I wonder what happened to them?

Atlas is the precursor to the next step in Navigator’s evolution, Version 3.0. It’s “alpha” code; in other words, that celebrated tower in Pisa is more stable. When it works, Atlas promises to deliver:

VRML viewing. VRML (Virtual Reality Modeling Language) is one of the competing standards for expressing three-dimensional information in compact form. VRML documents can be static or interactive. For example, you could create a Web page that ...

Read More Source

September 5th, 2016

In Technology

No Comments

The Big O cheat sheet

I just discovered the Big O cheat sheet and I think this is very useful:

Source

September 4th, 2016

In Technology

No Comments

Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring

I just realized that I’ve been confusing Faraday and Maxwell. It’s Maxwell who did the 4 equations.

Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring, and found that upon passing a current through one coil a momentary current was induced in the other coil.[2] This phenomenon is now known as mutual induction.[44] The iron ring-coil apparatus is still on display at the Royal Institution. In subsequent experiments, he found that if ...

Read More Source

August 19th, 2016

In Technology

No Comments

HAProxy for MySQL

Three years ago, I was intrigued to read that Twitter had implemented dynamic network topologies. Something similar is becoming more common, thanks to the use of Nginx/HAProxy. One can use this for many things, such as connecting to a pool of MySQL servers.

However, there are some downsides worth remembering:

You have to be careful to tune your xinetd script to increase the cps and per_source limits accordingly to the load that your pool of servers generate with this proxy forwarding. Assuming that ...

Read More Source

August 19th, 2016

In Technology

No Comments

Clever uses of a custom TCP stack

Interesting:

Fun anecdote, at Blekko we had people who tried to scrape the search engine by fetching all 300 pages of results. They would do that with some script or code and it would be clear they weren’t human because they would ask for each page right after the other. We sent them to a process that Greg wrote on a machine that did most of the TCP handshake and then went away. As a result the scrapers script would hang ...

Read More Source

August 16th, 2016

In Technology

No Comments

When algorithms betray you

Interesting:

You decide to share an article about Brexit from “The Atlantic”, which will somehow shed light to all your friends as to why it happened. The article is 1,000 words long — you only read half of it, but that’s good enough. It captures all the arguments you’ve been wanting to make for the past two months to your friends. Will this be the Facebook post that finally spurns your friends into action? You realize your Facebook friends all agree with your ...

Read More Source

August 11th, 2016

In Technology

No Comments

Only good leadership can create good software

This is very true, and it applies to more than software:

Thirteen years ago, Eric Raymond’s book The Cathedral and the Bazaar (O’Reilly Media, 2001) redefined our vocabulary and all but promised an end to the waterfall model and big software companies, thanks to the new grass-roots open source software development movement. I found the book thought provoking, but it did not convince me. On the other hand, being deeply involved in open source, I couldn’t help but think that it ...

Read More Source

August 11th, 2016

In Technology

No Comments

Clever hacks whereby sites can see aspects of your browser history

Very interesting hacks:

Up until mid-2010, any rogue website could get a good sense of your browsing habits by specifying a distinctive :visited CSS pseudo-class for any links on the page, rendering thousands of interesting URLs off-screen, and then calling the getComputedStyle API to figure out which pages appear in your browser’s history.

After some deliberation, browser vendors have closed this loophole by disallowing almost all attributes in :visited selectors, spare for the fairly indispensable ability to alter foreground and background ...

Read More Source

August 8th, 2016

In Technology

No Comments

The control mania of Scrum

This is very good:

Scrum inhibits deep understanding and innovation This is actually my biggest gripe about Scrum. As mentioned above, in Scrum, the gods of story points per sprint reign supreme. For anything that doesn’t bring in points, you need to get the permission of the product owner or scrum master or someone who has a say over them. Refactoring, reading code, researching a topic in detail are all seen as “not working on actual story points, which is what you ...

Read More Source

August 8th, 2016

In Technology

No Comments

The daily Scrum meeting is a pointless ritual

I love this:

The daily standup deserves a blog post of its own. This religious ritual has become a staple of every team in the world. Ten minutes of staring into the void, talking about what you did while no one else listens, because they were in the middle of something five minutes ago and will go back to it in another five minutes, and waiting for everyone else to finish. I know this sounds cynical, but it is the ...

Read More Source

August 8th, 2016

In Technology

No Comments

If you measure programmers by points then they will optimize for points

The worst thing with Agile/Scrum is that you can hit all your 2 week targets and yet a year later have software that no one wants to use, so to the extent that Agile is suppose to reduce risk, it fails completely.

But why does it fail? This is good:

No matter how you define story points, the real issue with them doesn’t go away. The main purpose of points is making planning more reliable, and providing a temporal perspective ...

Read More Source

August 8th, 2016

In Technology

No Comments

Problems with the Scrum process

This is a great conversation:

cageface says:

But the author’s criticisms of the incentives of Scrum are on point I think. Because the stories are always articulated in terms of user facing features they encourage developers to hack things together in the most expedient way possible and completely fail to capture the need to address cross cutting concerns, serious consideration of architecture, and refactoring.

This is how you can get two years into a project and have managers and clients that think that ...

Read More Source

August 3rd, 2016

In Technology

No Comments

The JustWorks job interview

One of the most popular posts I’ve written is “Embarrassing code I wrote under stress at a job interview“. People get a laugh out of the dumb things I do doing job interviews. Here is another such post.

I did a job interview at JustWorks. They asked me to write the code to solve this problem:

The cost of a stock on each day is given in an array, find the max profit that you can make by buying once ...

Read More Source

August 3rd, 2016

In Technology

No Comments

Is Docker ready for production?

I prefer uberjars to Docker, and I prefer fat binaries, such as those allowed by Go, over Docker. And still there is the question “Is Docker ready?”

Senex says:

I’ve been tracking the beta for a while. I’m confused about this announcement. These issues still seem unresolved? (1) docker can peg the CPU until it’s restarted

(2) pinata was removed, so it can’t be configured from CLI scripts

(3) it’s not possible to establish an ip-level route from the host to a container, which ...

Read More Source

July 27th, 2016

In Technology

No Comments

How to package dependencies in Golang

Very interesting, as I’m a fan of the idea of “vendoring”:

An attempt to fix packaging in Go:

Manul is a vendoring utility for Go programs.

What’s the reason for yet another utility?

Because all other vendor utilities suffer from the following:

Some wrap the go binary and spoof the GOPATH env variable. You will have a non-go-gettable project which needs additional software in order to compile and run;

Some copy the source code of dependencies into the vendor directory:

It will be nearly impossible to find ...

Read More Source

July 26th, 2016

In Technology

No Comments

Google Maps does not name Nachitschewan?

I can not find a zoom level where Google Maps names Nachitschewan. Is this a political thing?

Source

July 25th, 2016

In Technology

No Comments

The advantage of a weight-balanced B-tree

A very interesting data structure of which I know nothing:

Source

July 22nd, 2016

In Technology

No Comments

A string of characters is best parsed using a finite state machine

I already knew this, but I don’t think I knew the extent of what was sacrificed to give Perl backreferences s:

Notice that Perl requires over sixty seconds to match a 29-character string. The other approach, labeled Thompson NFA for reasons that will be explained later, requires twenty microseconds to match the string. That’s not a typo. The Perl graph plots time in seconds, while the Thompson NFA graph plots time in microseconds: the Thompson NFA implementation is a million ...

Read More Source

July 21st, 2016

In Technology

No Comments

The problems with Clojure

The conflict between Object Oriented Programming and Functional programming comes up a lot with Clojure, because to do any serious work you have to eventually use some Java, which takes you back into the world of Object Oriented Programming. This sums up my feelings:

The thing I like most about Elixir is the low friction between it and it’s host language, Erlang. Erlang is a functional language right from the start, and the BEAM is designed to run a functional ...

Read More Source

July 21st, 2016

In Technology

No Comments

The RegX that killed Stack Overflow

A great post-mortem of a crash at Stack Overflow:

The regular expression was: ^[\s\u200c]+|[\s\u200c]+$ Which is intended to trim unicode space from start and end of a line. A simplified version of the Regex that exposes the same issue would be \s+$ which to a human looks easy (“all the spaces at the end of the string”), but which means quite some work for a simple backtracking Regex engine. The malformed post contained roughly 20,000 consecutive characters of whitespace on a ...

Read More Source

July 21st, 2016

In Technology

No Comments

The difference between Kafka and Kestrel

For my purposes, for the next 6 months, it seems that Kestrel will be all I need. I’d have to be very successful before I would need anything as complex as Kafka.

The biggest conceptual difference is that Kestrel is a simple stand alone queue where as Kafka is a full fledged queuing system.

Kestrel runs on a single machine and has no concept of clustering or failover or any other features you might expect in a queuing system. Instead the clients ...

Read More Source

July 21st, 2016

In Technology

No Comments

RabbitMQ is difficult, Kafka is awesome, Kestrel is beautifully simple

A great review of these 3 queues:

Not mentioned below is that RabbitMQ works hard to guarantee delivery of a message, so it is slow, but that is because it is in some ways doing more than Kafka.

RabbitMQ:

I created 4 queues, wrote a ruby client and started inserting messages. I got a publishing rate of about 20k/s using multiple threads but I got a few stalls caused by the vm_memory_high_watermark, from my understanding during those stalls it writing to disk. ...

Read More Source

July 21st, 2016

In Technology

No Comments

Kestrel is as simple as Redis but Kestrel is an actual queue

I feel stupid that I didn’t look at Kestrel sooner. I feel especially stupid that I’ve been using Redis when I didn’t actually need it.

Redis has a beautifully flexible API which makes it tempting to use Redis for everything, but Redis is really a cache that focuses on speed above all else. Everything in Redis has to fit in memory, and Redis will drop anything that can’t fit in memory. Also, the stuff in Redis can be mutated, ...

Read More Source

July 21st, 2016

In Technology

No Comments

Only use Amazon SQS if you need a high-latency high-concurrency service

Also interesting, don’t use Amazon SQS unless your needs fit this very specific model where you can deal with the latency and make up for it by being highly concurrent.

I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is ...

Read More Source

July 21st, 2016

In Technology

No Comments

For Apache Storm, use Kestrel

There is a lot in this thread that is interesting. This is coming from Nathan Marz, the guy who invented Storm. He says he uses Kestrel, so that is a big endorsement.

You want to make sure that your spout source can support the out of order acking that Storm requires for guaranteed message processing. We use Kestrel because it has this property and is the simplest. RabbitMQ is another good one to consider.

Source

July 20th, 2016

In Technology

No Comments

The problem with RabbitMQ

Seems like RabbitMQ is good but not perfect:

Much like you’d chuck memcached on each of your web servers and access them in a ring, Darner can occupy a small niche on each box in your fleet’s resources. Tens of MB of RAM and negligible CPU opens up hundreds of gigabytes of queue spool per node. As queue size grows, memory usage remains constant.

Contrast this with Redis, which is speedy but limited in queue size to what will fit ...

Read More Source

July 20th, 2016

In Technology

No Comments

ZeroMQ is the opposite of a queue

Reading this again, several years after the first time I read it, and now it occurs to me: I would use ZeroMQ only in exactly those situations where I would not use a queue. If I only need a queue, I can use Kestrel. But if I need some messaging pattern that is not supported by any queue, then I would use ZeroMQ. That is, if I had a truly unique situation that needed a unique pattern, then ZeroMQ would ...

Read More Source

July 20th, 2016

In Technology

No Comments

Everything you need to know about queues

This looks like an amazing resource and I’m sure I’ll go back and read more soon.

Source

July 20th, 2016

In Technology

No Comments

The New York Times uses RabbitMQ

RabbitMQ is written in Erlang, so I was inclined to think well of it, though I heard criticism of it. And then the New York Times used it. A very surprising vote of confidence in RabbitMQ:

This architecture – Fabrik – has dozens of RabbitMQ instances spread across 6 AWS zones in Oregon and Dublin. The instances are organized into “wholesale” and “retail” layers. Connection to clients is via websockets/sockjs.

Upon launch today, the system autoscaled to ~500,000 users. Connection times ...

Read More Source

July 20th, 2016

In Technology

No Comments

Gauges uses Kestrel which is based on work from Blaine Cook

Why does Kafka get all the attention if Kestrel is so reliable? I assume this is because Kafka can do so much more, though of course the devops work of Kafka can be frightening. I didn’t know about the Blaine Cook connection (of Twitter fame):

Before I get too far a long with this fairy tail, let’s talk about Kestrel — what is it and why did I pick it?

Kestrel is a simple, distributed message queue, based on Blaine Cook’s ...

Read More Source

July 20th, 2016

In Technology

No Comments

Kestrel versus Resque

Interesting note about these queues, which certainly convinces me to use Kestrel:

There are a lot of things that could be done with either Kestrel or Resque. Because Resque is backed by Redis, you have to remember that all of the messages waiting to be processed have to be able to fit in the RAM of the Redis server, with Kestrel you could queue millions or billions of messages and then start to pull them off.

The biggest difference between the ...

Read More Source

July 20th, 2016

In Technology

No Comments

How bad can MongoDB be with Dirty Reads?

Very worrisome, though no sane person would use MongoDB to track something involving money. That’s simply not what MongoDB is for.

How bad are dirty reads?

Read uncommitted allows all kinds of terrible anomalies we probably don’t want as MongoDB users.

For instance, suppose we have a user registration service keyed by a unique username. Now imagine a partition occurs, and two users–Alice and Bob–try to claim the same username–one on each side of the partition. Alice’s request is routed to the ...

Read More Source

July 20th, 2016

In Technology

No Comments

How to return a disallowed field in GraphQL

Best Practices for GraphQL are still in a very immature stage, but these two ideas both have some merit:

First, return null for the requested field. This seems to work well in cases where there is no real harm in asking for a particular set of data and no real harm in denying it.

A good example would be asking for the email of a user where the backend only provides the user’s email to that user themselves. If I request my ...

Read More Source

July 20th, 2016

In Technology

No Comments

Why has Google released gRPC

This is interesting:

Google has been using a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across our data centers for over a decade. Our internal systems have long embraced the microservice architecture gaining popularity today. Having a uniform, cross-platform RPC infrastructure has allowed for the rollout of fleet-wide improvements in efficiency, security, reliability and behavioral analysis critical to supporting the incredible growth seen in that period.

Stubby has many great features – ...

Read More Source

July 13th, 2016

In Technology

No Comments

Why I hate Ruby On Rails

I’m trying to help a friend with a very small Ruby On Rails project, where she needs a few elements of the interface tweaked. But this is Ruby On Rails, so of course I’m losing 3 hours trying to get setup. I’ve been drawn into the endless swamp of upgrading various libraries, which seems to be an automatic part of dealing with Ruby On Rails. If this was Clojure, or NodeJS, setup would have taken 5 minutes and I would ...

Read More Source

July 13th, 2016

In Technology

No Comments

Mixins are harmful — the Javascript edition

Over the years there have been arguments against mixins, but the dominance of Object Oriented Programming meant that mixins still survived and had theoretical justification. Now, the spread of the Functional Paradigm has put renewed pressure on mixins. Dan Abramov at Facebook just came out with an essay against mixins:

“How do I share the code between several components?” is one of the first questions that people ask when they learn React. Our answer has always been to use ...

Read More Source

July 10th, 2016

In Technology

No Comments

Edges is just a wrapper around the real results that adds extra data used for slicing the result cursor

I think the “edges” and “nodes” terminology is one of the most confusing things about React / Relay / GraphQL, at least at first:

Relay connection defines a relationship an an object that has two root fields: edges and pageInfo. Edges is just a wrapper around the real results that adds extra data used for slicing the results (cursors), PageInfo has metadata related to the current page. This is how it looks like:

connectionName { edges { ...
Read More Source

July 10th, 2016

In Technology

No Comments

Make the Web compatible?

It must be 1999 again because someone is writing about browser incompatibilities:

Users won’t switch browsers, they’ll switch sites

You might think that users will switch browsers to use your site. But many won’t or can’t.

Users have no patience for things that don’t work, and they’ll just go to a competitor’s site instead. Failing at a critical point could turn a potential user away forever. According to Akamai,

32% of users who encounter a problem on your site are less likely to make ...

Read More Source

June 30th, 2016

In Business, Technology

No Comments

The Agile process of software development is often perverted by sick politics

For those of you who don’t want to read this whole essay, here is the summary:

The word “agile” has a plain meaning in standard English, and that meaning was considered something positive by software developers, so much so that the most successful new development process of the last 30 years calls itself “Agile”. However, at many of the companies that I have worked, actual agility is suppressed because of various political factors. Fear wins out over trust. Instead of actual ...

Read More Source

June 29th, 2016

In Technology

No Comments

How much time and energy should a computer programmer spend learning a proprietary server system?

By itself, this is an anecodote about Google’s particular system: (but see my point at bottom)

Is there a lesson in this? Well, if I were in the business of programming Google App Engine, a few days’ effort up front to get it going might seem not to be a big deal. However, when I visit a team who have just added a new person, I invariably find that person struggling to set up their workstation. There’s usually someone in the room ...

Read More Source

June 29th, 2016

In Technology

No Comments

Is Phoenix going to take over the tech industry?

If not, why? This has been one of the big puzzles of the last 30 years: if some technologies are clearly superior, then why don’t they take over? Apparently because they lack an element that appeals to management in big corporations? That’s always been the argument against Lisp: it is great for the individual master craftsman, but it doesn’t work in a big corporation full of badly paid, mediocre programmers.

Erlang has been the most safe, resilient technology out there ...

Read More Source

June 23rd, 2016

In Technology

No Comments

Pagination with React / Relay / GraphQL

This is an awesome article:

Relay proposes a standard to define a has-many relationship for a GraphQL field. This standard defines a common structure that allows Relay to paginate and filter the results in an efficient way by using cursors, which I’ll explain in a bit.

This is the definition of a Relay connection (from the Relay connection specs):

Relay’s support for pagination relies on the GraphQL server exposing connections in a standardized way. In the query, the connection model provides a standard ...

Read More Source

June 15th, 2016

In Technology

No Comments

Demanding ssh key login to a server is safer than allowing passwords

This is good and true:

Require ssh key authentication

We tend to avoid passwords for logging into servers. There was a lot of discussion around this after Bryan’s original guide came out, but I tend to fall into this camp as well. Here are a few notes on this:

ssh keys are better than passwords only because they contain and require more information.

Passwords can be brute forced. Guessing a public key is so essentially impossible that they can be considered perfectly secure

What ...

Read More Source

June 14th, 2016

In Technology

No Comments

Object/Relational Mapping is rarely worth it

I think this can be stated simply: for your basic operations (CREATE, UPDATE, READ, DELETE) then an ORM offers some convenience. But ORMs break down as soon as you have a complex query. And yet, if you start using an ORM because it helps you with the simple queries, it will seem natural to you to extend it and try to use it for the complex queries.

This is a strained analogy to the Vietnam war:

Object systems are typically characterized ...

Read More Source

June 11th, 2016

In Technology

4 Comments

AWS is inappropriate for small startups because its complexity demands a specialist

Sean Hull references a conversation that he and I had in Slack. I would like to expand on the argument that I made then. You might want to read his essay first, where he makes these points about AWS:

1. Over 70 services offered

2. Still complex to build high availability

3. Need a dedicated devops

4. Orchestration involves many moving parts

5. Troubleshooting failed deployments is difficult

At the time of our conversation I had crashed an AWS instance and we were having trouble fixing ...

Read More Source

June 11th, 2016

In Technology

No Comments

GraphQL is the best thing about React / Relay / GraphQL

React / Relay / GraphQL:

The big breakthrough here is obviously GraphQL. Not React or Relay, since there are many competing implementations that do similar things, but GraphQL. Even though there are graph query languages out there (Gremlin, etc) they were not suited to querying JSON over the wire. GraphQL is ideal. For my next project, I hope to do a Clojure implementation of most of GraphQL, because I think it can be married to Om.Next in a very powerful way.

Actually, ...

Read More Source

June 10th, 2016

In Technology

No Comments

Krubner, you saved my day

Aw man, after all the mistakes I make, it is nice to know I help people sometimes:

Source

June 10th, 2016

In Technology

No Comments

Sometimes I make a bad decision

It’s very, very, very rare, almost unheard of, but sometimes I make a mistake.

I was helping at Open Road this spring. Sean Hull was also helping. Open Road runs a bunch of WordPress sites, such as The Line Up. All of the images for these sites were stored on a central NFS server. All of the machines were starving for hard disk space. The previous Director Of Technology had made many, many bad decisions, including the decision to save ...

Read More Source

June 7th, 2016

In Technology

No Comments

The Belief that Tests are Smarter than Code Telegraphs Latent Fear or a Bad Process

This part is really good:

Programmers have a tacit belief that they can think more clearly (or guess better) when writing tests when writing code, or that somehow there is more information in a test than in code. That is just formal nonsense. The psychological perspective is instructive here, and it’s important because that — rather than any computational property — most drives developer behaviour. If your coders have more lines of unit tests than of code, it probably means one of several things. They may be ...

Read More Source

June 7th, 2016

In Technology

No Comments

The problem with unit tests

Interesting:

Unit testing is of course not just an issue in object-oriented programming, but the combination of object-orientation, agile software development, and a rise in tools and computing power has made it de rigueur. As a consultant I often get questions about unit testing, including this real one from a recent client of mine, Richard Jacobs at Sogeti (Sogeti Nederland B.V.): My second question is about unit tests. If I remember correctly you said that unit tests are waste. First, I was surprised by that. Today however, my ...

Read More Source

June 6th, 2016

In Technology

No Comments

Ruby On Rails is obsolete

This is a strange conversation to be having in the year 2016. Ruby On Rails changed the technology industry in 2004, but it can not claim to be winning in 2016. It still lacks a story for concurrency. It is being undermined by 2 forces:

1.) the need for concurrency, which is offered by other eco-systems, such as those that run the JVM. A language like Clojure offers the high level of meta-programming that Rubyists love, but with the vast arsenal ...

Read More Source

June 2nd, 2016

In Technology

No Comments

Sometimes peer review delays progress for many decades

This is what real leadership looks like:

Morris intentionally consumes H. pylori. Like Marshall, he becomes ill, but unlike Marshall, he is not completely cured by antibiotics. The infection will remain with him for three years.

I wish he had been given a medal for his self-sacrifice. He was attempting to save millions from suffering.

Interesting, though very disappointing:

1881 Klebs notes the presence of bacteria-like organisms in the lumen of the gastric glands.

1889 Walery Jaworski describes spiral organisms in sediment washings ...

Read More Source

May 22nd, 2016

In Technology

No Comments

Why does Rails become awful?

Interesting:

Let me tell you a story about what can happen in a convoluted Rails codebase. Once, I joined an existing project. It was a huuuuge app which was running an on-line shopping community website. Complicated sales model, complicated promotions, complicated product setups, coupons, user groups, messages – it had it all. I joined them to help ship a few new features. One of my early tasks was to…add a link to something on some page. It took me few days ...

Read More Source

May 15th, 2016

In Technology

No Comments

The dumbest name for a good architecture is “serverless”

I strongly agree that the idea is good, but the name “serverless” is really stupid:

So… this is backend infrastructure (i.e. it runs on a server) to let your team deploy “severless” apps…

So they’re serverless. Except the server you’re running it on. And the lamba/style code they wrote and uploaded to it.

But serverless, because no ops staff required. Except the ones who installed and maintain this.

This is like a snake eating its own tail and wondering what hurts.

Edit: despite my sarcasm, ...

Read More Source

May 10th, 2016

In Technology

No Comments

The sad, slow way a system of cron scripts becomes ugly

Do you have a chore that needs to run in the background, maybe once a day, or once an hour? Cron scripts will save you! They are the most amazingly amazing thing God has invented since Adam and Eve! For sheer wonderfulality they have no equal among the products of mortal fallen flesh!

At least at first.

The simple cron script is wonderfully direct and efficient. But the first can lead to a second. The second can lead to a third. ...

Read More Source

April 10th, 2016

In Technology

No Comments

What do AWS services actually do?

I love this:

Data Pipeline Should have been called Amazon ETL

Glacier Should have been called Really slow Amazon S3

Kinesis Should have been called Amazon High Throughput It’s like Kafka

I have used this but I did not know the name:

Snowball Should have been called AWS Big Old Portable Storage Use this to Get a bunch of hard drives you can attach to your network to make getting large amounts (Terabytes of Data) into and out of AWS It’s like Shipping a Network Attached Storage device to AWS

These are useful names that actually describe what ...

Read More Source

March 27th, 2016

In Technology

No Comments

The fanaticism of “test driven development” is slowly fading

This was one of those trends that went too far and I’m glad to see people talking about reasonable limits:

If you’re part of the Ruby On Rails community for a long time, you’ve probably read tons of articles about testing Rails application (less these days, though). Although there always have been diverging opinions on the matter, it seems the common wisdom was to say that you had to test everything: models, controllers, views and full-stack tests. Oh, and you had ...

Read More Source

March 26th, 2016

In Technology

No Comments

RESTful APIs are dead, long live GraphQL

We can argue whether RESTful APIs ever existed, since very few ever implemented HATEOAS. Ruby On Rails claimed it offered RESTful APIs, but it never offered HATEOAS. After 10 years of failure, the world is looking for something new.

Interesting:

Imagine we have a simple application that fetches a list of stories, and some details about each one. Here’s how that might look in resource-oriented REST:

// Fetch the list of story IDs but not their details: rest.get('/stories').then(stories => // This ...
Read More Source

March 26th, 2016

In Technology

No Comments

Picking React isn’t a technology decision, it’s a business decision

Interesting:

The clear winner right now, is React. Components all the way down makes your application much easier to reason about. The learning curve is very flat. The important APIs would fit on one page. JSX is awesome. You get all the power of JavaScript and its tooling when writing your markup. It is the natural match for Flux and Redux (more on that later). The React community is amazing, and produced many best of breed tools such as Redux (also more on that later). Writing high ...

Read More Source

March 26th, 2016

In Technology

No Comments

Pay discrimination in tech

This sounds bad:

Women who write the software that runs on mainframe computers earn on average 72 cents per dollar earned by their male counterparts, according to research conducted by Glassdoor Inc., the online job information firm. That pay gap exists even after controlling for age, education, experience, job title, employer and location.

This suggests that the gap widens with age:

However, not all tech jobs pay men and women so differently. Among mobile developers, there is just a 2.9% gap between ...

Read More Source

March 26th, 2016

In Technology

No Comments

Left-Pad as a service

Everyone is laughing over this. An 11 line function as its own NPM module? I am unsure how I feel. The trend has been to smaller and smaller pieces of functionality. I don’t find this outrageous. Still, the parody is funny:

c4n4rd is game:

This is really exciting!!! I was a bit disappointed that the right-pad will be out only in 2017. I am looking forward to that release because there is a high demand for it now.

What kind of ...

Read More Source

March 26th, 2016

In Technology

No Comments

null

Testing WordPress to see if a title of “Null” works.

Source

March 18th, 2016

In Technology

No Comments

The end of the test-first mania

It worries me that the tech industry is driven by such intense manias, which have cost me several jobs. For a long time you were not allowed to question Object Oriented Programming. I still go to job interviews and I’m asked the basics of Object Oriented Programming, and it is clear I have to say nice things about Object Oriented Programming or I won’t get the job. Thankfully none of these people ever check Wikipedia, where I am cited as ...

Read More Source

March 17th, 2016

In Technology

No Comments

Onyx tested by Jespen is genius tested by genius

Good lord, this makes me happy. I think Michael Drogalis is a very smart guy and everyone knows that Kyle Kingsbury is a very, very smart guy. Kingsbury’s work on Jespen is the finest work that anyone has ever done on the problems of distributed data. Onyx is exciting as an a Clojure answer to Apache Storm. Sad to say, Storm is written in Scala. The idea of testing Onxy with Jespen is one of the finest ideas ...

Read More Source

March 14th, 2016

In Technology

No Comments

Unix processes have gotten bigger and slower over the years

Interesting:

The obvious question thus is: Why state machines? Why not processes or threads?

And the obvious answer is: Performance.

When UNIX was still young, scheduling was supposed to be done by the OS on per-process basis. When implementing a network server, for example, you were supposed to fork a new instance of the process for each TCP connection and rely on the OS scheduler to switch between the processes.

I guess it made sense from performance point of view back then. All ...

Read More Source

March 10th, 2016

In Technology

No Comments

Jakub Holý on copying :pre and :post conditions

Obviously I have read this article before (I offered response and was quoted before) but I’ve gone back to read it again and this jumped out at me:

Do you repeat the same checks again and again? Then you could either copy them using with-meta (they end-up in metadata anyway) or reuse the explicitly:

(defn with-valid-car [f] (fn [car] {:pre [:make :model :year]} (f car))) (def count-price (with-valid-car (fn [car] (do-something car)))) ;; or make & use a macro to make it nicer

That ...

Read More Source

February 15th, 2016

In Technology

No Comments

Sick politics is the driving force of useless ceremony

This is part 7 of a 12 part series:

1.) Quincy’s Restaurant, a parable about concurrency

2.) Why I hate all articles about design patterns

3.) Clojure has mutable state

4.) Immutability changes everything

5.) Mutable iterators are the work of the Devil

6.) Get rid of all Dependency Injection

7.) Sick politics is the driving force of useless ceremony

8.) Functional programming is not the same as static data-type checking

Interlude

9.) Inheritance has nothing to do with Objects

10.) Is there a syntax for immutability?

11.) Immutability enables concurrency

12.) Quincy’s ...

Read More Source

February 13th, 2016

In Technology

No Comments

Immutability changes everything

(Acknowledgements:

I offer a huge “Thank you” to Natalie Sidner for the tremendous editing she did on the rough draft of this post. To the extent that this article is readable, it is thanks to her. Any mistakes are entirely my fault, and I probably added them after she was done editing. If you need to hire a good editor, contact Natalie Sidner at “nataliesidner at gmail dot com”.

Also, I thank Blanche Krubner for reviewing this work. As Mrs Krubner studied ...

Read More Source

February 13th, 2016

In Technology

No Comments

Functional programming is not the same as static data-type checking

(Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!)

This is part 8 of a ...

Read More Source

February 10th, 2016

In Technology

No Comments

Interlude

I am mostly done writing the second half of this series, however, it needs to be edited, some code needs to be written, all of the code needs to be checked, and I need to work with Leah McCloskey to develop further cartoons for illustrating the points made in this series. Also, I have a full time job. So it will likely be 2 or 3 months before I can publish the second half of this series.

The series is ...

Read More Source

February 10th, 2016

In Technology

No Comments

Get rid of all Dependency Injection

This is part 6 of a 12 part series:

1.) Quincy’s Restaurant, a parable about concurrency

2.) Why I hate all articles about design patterns

3.) Clojure has mutable state

4.) Immutability changes everything

5.) Mutable iterators are the work of the Devil

6.) Get rid of all Dependency Injection

7.) Sick politics is the driving force of useless ceremony

8.) Functional programming is not the same as static data-type checking

Interlude

9.) Inheritance has nothing to do with objects

10.) Is there a syntax for immutability?

11.) Immutability enables concurrency

12.) Quincy’s ...

Read More Source

February 10th, 2016

In Technology

No Comments

Mutable iterators are the work of the Devil

The assignment from Hell

Assume you die, and then you wake up in a beautiful place, floating on a cloud, and you hear a trumpet in the distance, and you hear beautiful singing. Then a voice from up on high says to you, “I would like you to count all of the positive integers. I will give you all eternity to do it.” You assume you are now in heaven, and God is giving you your first heaven-based assignment. Can you ...

Read More Source

February 10th, 2016

In Technology

No Comments

Clojure has mutable state

Warning: This post is an intro to mutable state in Clojure. If you already know Clojure, you can skip this. If you have no interest in learning Clojure, you can skip this.

This is part 3 of a 12 part series:

1.) Quincy’s Restaurant, a parable about concurrency

2.) Why I hate all articles about design patterns

3.) Clojure has mutable state

4.) Immutability changes everything

5.) Mutable iterators are the work of the Devil

6.) Get rid of all Dependency Injection

7.) Sick politics is the ...

Read More Source

February 10th, 2016

In Technology

2 Comments

Why I hate all articles about design patterns

Sometimes play is a lie

Imagine two children, both nine years old, decide to play tea. They rummage around in the closets of their respective parents. They dress up as they’ve seen their parents dress up. The one goes over to the other’s house.

“Mr. Worthington, it is so nice to see you,” says the girl who is acting as host. “Would you like some tea?”

“Why, yes, Mrs. Kentworth, I would be delighted to have some tea.”

The host brings an empty cup ...

Read More Source

February 10th, 2016

In Technology

No Comments

A parable about concurrency (demonstrated with comical cartoons)

(Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!)

This is part 1 of a ...

Read More Source

February 9th, 2016

In Technology

No Comments

Why does Erlang allow so many processes to crash and how does promote reliability?

Why does Erlang encourage crashes?

Back-burning and controlled burns are a real world example of fighting fire with fire. In Saguenay–Lac-Saint-Jean, the region I come from, blueberry fields are routinely burnt down in a controlled manner to help encourage and renew their growth. To prevent forest fires, it is fairly frequent to see unhealthy parts of a forest cleaned up with fire, so that it can be done under proper supervision and control. The main objective there is to remove ...

Read More Source

February 6th, 2016

In Technology

No Comments

Hindley-Milner and the lambda calculus

A nice attempt to explain the crazy syntax of the math that formalizes the idea that the type of an expression can be deduced from the expression itself:

Okay, so we want to talk about expressions. Arbitrary expressions. In an arbitrary language. And we want to talk about inferring types of these expressions. And we want to figure out rules for how we can infer types. And then we’re going to want to make an algorithm that uses these rules to ...

Read More Source

February 3rd, 2016

In Technology

No Comments

Beware of Ruby libraries that generate way too many objects

This is several years old and Ruby garbage collection has gotten better, but still, the point about certain libraries being excessive remains valid.

Be aware that you are allocating objects, for instance something as simple as 100.times{ ‘foo’ } allocates 100 string objects (strings are mutable and therefore each version requires its own memory allocation).

Make sure to evaluate the libraries you use, for instance switching a Sinatra XML rendering action from Builder to Nokogiri XML Builder saved us about ...

Read More Source

February 3rd, 2016

In Technology

No Comments

Pat Shaughnessy dissects how much work Ruby has to do to give you a string

Easy is difficult, and this is a great look at how much work Ruby has to do so that you, the software developer, can change your mind about what kind of string you want:

The standard and most common way for Ruby to save string data is in the “heap.” The heap is a core concept of the C language: it’s a large pool of memory that C programmers can allocate from and use via a call to the malloc ...

Read More Source

February 3rd, 2016

In Technology

No Comments

Software development is a complex system of multiple poorly understood feedback loops and interactions

I wouldn’t use exactly the same words that this article uses, but I agree with the gist of this part, especially where small, fast-moving startups are involved :

Finely grained management of software developers is compelling to a business. Any organization craves control. We want to know what we are getting in return for those expensive developer salaries. We want to be able to accurately estimate the time taken to deliver a system in order to do an effective cost-benefit analysis ...

Read More Source

February 2nd, 2016

In Technology

No Comments

Leonardo Borges’s Imminent for futures and promises

Very interesting:

For the impatient, I’ve included a couple of examples below. I’ve chosen to translate the examples presented by Ben Christensen – of RxJava – in this gist. Albeit them being in Java, they highlight perfectly the problem with blocking futures. Here’s their Clojure equivalent:

;; ;; Example 1, 2 & 3 are handled by the approach below ;; Original examples: https://gist.github.com/benjchristensen/4671081#file-futuresb-java-L13 ;;

(defn example-1 [] (let [f1 (remote-service-a) f2 ...
Read More Source

February 2nd, 2016

In Technology

No Comments

Why the tech community rejected XML

Interesting:

Java was a very limited language, and it was extremely verbose. There were type declarations and coersions everywhere. Almost all looping was managed with for loops (there were no higher order functions, and recursion wasn’t tail cail optimized… also there was no loop-recur macro…).

So XML wasn’t just there to be data, it was also code.

Everyone wrote little mini-languages into XML, because some things were so damn painful to express in Java that it was easier to just implement ...

Read More Source

February 2nd, 2016

In Technology

No Comments

Clojure community problems

Interesting:

What an amazing language. Relatively frequent, consistently stable releases. A pleasure to use. A friendly, smart community. I feel very lucky to be a Clojure user! Thank you for all of your hard work.

The Clojure contrib process frustrates me more than any technical or community aspect of the language.

Clojure gets a lot right, but as has been repeatedly discussed the pace of evolution and the maintainership’s dim view of 3rd party non-bugfix work flatly leads to worthy but minor ...

Read More Source

January 31st, 2016

In Technology

No Comments

We want loose-coupling and high cohesion

This seems like a great rule of thumb for microservices:

“We want to avoid dumb, anemic services that are little more than CRUD wrappers”

But it doesn’t cover the old territory which, if you were using Ruby or PHP, you would cover with a cron script. I suppose all the cron scripts must become functions that live inside the “service” which deals with a given part of the datastore. But that is not how my friends talk about “microservices”.

Page 58, ...

Read More Source

January 31st, 2016

In Technology

No Comments

A message hospital (or dead letter queue), where messages got sent if they failed

I like this book very much. I also like the idea of a “message hospital”.

Page 57, Building Microservices, Sam Newman:

Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a ...

Read More Source

January 31st, 2016

In Technology

No Comments

Jon Williams, Fractional CTO: Include business teams in the Agile process

Jon Williams offers a smart idea about getting the business people and the tech team to share as much information as possible, and even have the business people join the scum sessions.

The best implementation of the Agile process that I ever participated in was at ShermansTravel.com, back in 2011/2012. We had fairly good communication between the tech team and the folks running the actual parts of the business: editorial, advertising, marketing and more. A lot of the success of ...

Read More Source

January 31st, 2016

In Technology

No Comments

Joe Armstrong figured out the right way to do everything, and nobody cared

I am puzzled why good ideas so rarely win out in the tech world. I do know the old saying “Most industries have a top player with the best marketing and the second best technology, with a second place player that has the best technology and mediocre marketing.” Back in the 1990s it was common to apply that slogan to Microsoft and Apple (when Apple had mediocre marketing! Such long forgotten days!).

Still, the lack of innovation in this industry ...

Read More Source

January 30th, 2016

In Technology

No Comments

When technology goes on strike

Interesting:

Amazing but true When I’d finished this article, I wanted to spell check the content. emacs-ispell mode decided to go on strike. It could not find aspell, the program that I use for spelling checking.

My emacs spell checker has worked faithfully on this machine for several years. And just when I complain that I spend half my life fixing things that shouldn’t be broken the emacs spell checker decides to break.

I don’t believe in malicious Gods, nor that the laws ...

Read More Source

January 24th, 2016

In Technology

No Comments

Groping in the dark as a method to discover module boundaries in microservices

I’m not sure that we will ever have a better way (but then, we can never pretend to be like other types of engineers, can we?):

Having clients talk to the application service instead of an ORM or the data backends directly lets you forget that there is a MASSIVE PROBLEM with ORM frameworks (for anything more interesting than, say, a blog website framework), because you will still have to write a translation between your application representation and the database representation. ...

Read More Source

January 24th, 2016

In Technology

No Comments

Why I love Erlang

And why I dislike Scala. I would bet that Scala was designed in the opposite fashion:

Everything was very problem oriented and we did not have as goals that Erlang should be a functional language or that we should implement the actor model. We knew nothing of the actor model until later when we heard that Erlang implements it. :-) …

Again our goal was to solve the problem, not design a language with a predefined set of primitives.

Source

January 22nd, 2016

In Technology

No Comments

HTML is the failed GUI for TCP/IP

I posted this to Hacker News. I was surprised that someone did not immediately understand what I meant when I referred to HTML as the GUI of TCP/IP. Someone had surveyed 80 frontend designers and found they lacked basic knowledge of HTML and CSS. I responded:

The contrarian argument is that this signifies an important truth, that HTML never worked the way it was suppose to. In the same way that we might argue that a misunderstood product is the fault ...

Read More Source

January 20th, 2016

In Technology

No Comments

An Amazon DSL in Clojure

Good lord. This is an impressive list of supported Amazon APIs:

Api Gateway Autoscaling CloudFormation CloudFront CloudSearch CloudSearchV2 CloudWatch CodeCommit CodeDeploy CodePipeline Config DataPipeline DeviceFarm DirectConnect Directory DynamoDBV2 EC2 EC2 Container Registry ECS ElastiCache ElasticBeanstalk ElasticFileSystem ElasticLoadBalancing ElasticMapReduce ElasticTranscoder Glacier IdentityManagement Kinesis KinesisFirehose KMS Logs Lambda MachineLearning OpsWorks RDS Redshift Route53 S3 SimpleDB SimpleEmail SimpleWorkflow SNS SQS StorageGateway

Source

January 20th, 2016

In Technology

No Comments

A 3D DSL in Clojure

Very interesting, and a nice example of how much Clojure allows the creation of new languages:

My first step in the HOLO design process was the creation of a sufficiently flexible GP playground for my later experiments and to evolve path-finding agents to create a (typographic) form as an initial design idea. Related to this, though a year prior, I created the Mophogen DSL, partially done as a component of my commission for the Barbican / Google DevArt exhibition. Morphogen ...

Read More Source

January 16th, 2016

In Technology

No Comments

Microservices first

Stefan Tilkov adds clarity to an argument that I’ve been making since 2013:

If you are actually able to build a well-structured monolith, you probably don’t need microservices in the first place. Which is OK! I definitely agree with Martin: You shouldn’t introduce the complexity of additional distribution into your system if you don’t have a very good reason for doing so.

(So what would be a good reason? There are many, but to me the most important one is to ...

Read More Source

January 16th, 2016

In Business, Technology

No Comments

I give good tech advice to startups, and different good advice to Enterprises

I had a job interview recently that went well. But then, the CTO read something that I had written a year and a half ago.

Then the CTO said to me “We can’t hire you. We are a tiny startup. We are facing some serious deadlines. We need to push a product out the door.”

I said, “I want to help you push a product out the door. I can work hard and help you guys move fast.”

He said, ...

Read More Source

January 16th, 2016

In Technology

No Comments

The cost of using patterns is that I have to give up the illusion that I am infinitely creative

This is true of all creative fields:

The cost of using patterns is that I have to give up the illusion that I am infinitely creative. I don’t invent a new programming language with optimizing compiler and novel operating system and complete programming environment for every line of code. To make best use of my three billion seconds, I should ignore most of my options and focus on a few degrees of freedom that really matter right now.

Source

January 16th, 2016

In Technology

No Comments

Paralyzed by too much thinking

I like this:

When I started programming again, I vowed not to type a single character unless I knew what pattern I was applying as I did so. The result was incredibly frustrating. I want a class called “Stack”, but why “Stack” and not something else. Then I would go and write the patterns for naming classes and then I could type “Stack”. Then I would want to make its first method public, but why public?

At first, it was like ...

Read More Source

January 14th, 2016

In Technology

No Comments

The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise

Interesting

What is the black king in chess? This is a strange question, and the most satisfactory way to deal with it seems to be to sidestep it slightly. What more can one do than point to a chessboard and explain the rules of the game, perhaps paying particular attention to the black king as one does so? What matters about the black king is not its existence, or its intrinsic nature, but the role that it plays in the game.

The ...

Read More Source

January 13th, 2016

In Technology

No Comments

Invoxia will allow Alexa to figure out who is speaking

I will be pleased if this becomes more than vaporware:

Invoxia, one of the recipients of an Alexa Fund investment from Amazon last September, announced this week at CES that it’s the first third-party hardware maker to incorporate all the power of Alexa into a product other than the Echo. The company’s Triby, as it’s called, is a colorful, magnet-backed Bluetooth speaker resembling an old-school radio. It’s designed to let family members, including young children, make internet-based phone calls with ...

Read More Source

January 7th, 2016

In Technology

No Comments

CQRS Read Models

Interesting:

We’ve been using this mechanism with great success during development. It let us freely modify the schema by only changing the Java classes and never worrying about table definitions. Thanks to combination with CQRS, we could even maintain long-running demo or pilot customer instances. Data has always been safe in the event store. We could develop the read model schema incrementally and have the changes automatically deployed to a running instance, without data loss or manually writing SQL migration scripts.

Obviously ...

Read More Source

January 6th, 2016

In Technology

No Comments

No one realizes that there is an app store for the Amazon Echo?

More from Jose Jaquinta:

I ran into some people during various New Year’s parties who had Echos, but had no clue there were skills for them. I laboriously explained how to launch the companion app, find the skills tab, and enable the skill.

It would a lot simpler if a user could enable/disable skills without having to go through the companion app. For example:

Alexa, enable Demotivate Me

I realize this is a request for The Echo, and not the Alexa Skills Kit. However, ...

Read More Source

January 6th, 2016

In Technology

No Comments

The Amazon Alexa certification process has gone out of control

This is sadly accurate:

The certification process has gone out of control.

This is one example of the… I’m struggling to keep to professional language… questionable judgement… show by the certification team. I’ve had two skills come back with a similar objection. I want to ask them if they have even used Alexa regularly? It gets REALLY tedious to have it endlessly repeat things. It’s like, “Yes, I know what I can do next. Please let me get on with ...

Read More Source

January 1st, 2016

In Technology

No Comments

Mark Lee Smith fights functional programming in 2011

I find it interesting to go back and look at the sites where was learning about Functional Programming back in 2011.

It’s interesting to read those who were on the losing side of history, fighting against Functional Programming:

If you’re writing 2000-line classes and 500-line methods then you’re in no way qualified to be speaking about object-oriented programming, let alone comparing it with the latest FAD ideology.

Christian Toivola gave what has become the standard answer:

@Mark Lee Smith: With the ...

Read More Source

January 1st, 2016

In Technology

No Comments

When did the Internet start?

Interesting:

TCP version 1 was designed in 1973. This was documented through RFC 675. TCP version 2 was documented in March 1977. In August 1977, Jon Postel realized they were going the wrong direction with the protocol. “We are screwing up in our design of internet protocols by violating the principle of layering. Specifically we are trying to use TCP to do two things: serve as a host level end to end protocol, and to serve as an internet packaging and ...

Read More Source

December 31st, 2015

In Technology

No Comments

RFC 128: We are contemplating allowing sender and receiver to specify different byte sizes

Wow. Can you imagine how that would have turned out? On the bright side, Unicode would not be necessary, since a company in China could specify a byte size large enough to represent any Chinese characters, and likewise for Iran, Korea, Hindi, etc. .

Source

December 31st, 2015

In Technology

No Comments

NLP versus Conversational AI

Obviously, I have written a lot about this subject during the year. See my post “What happens when the Board Of Directors begins to panic“.

But this is getting big:

To date, commercial NLP research has been focused on understanding the many ways people say commands like, “Set my alarm for 7 am tomorrow” or “Find a good pizza place nearby.” To unlock the power of conversational computing, new technology is required that extracts semantics across multi-turn natural language ...

Read More Source

December 31st, 2015

In Technology

No Comments

Allowing an object to exist in an invalid state is an Anti-Pattern

This post uses the language of Object Oriented Programming, but its point would also be true to all forms of programming. In Clojure, if you have HashMaps that you then convert to JSON and save to MongoDB, then why would you ever allow the HashMap to exist in an invalid state? Real immutability might imply that the HashMap is created with everything that it will eventually need. This is an interesting idea.

Why do we validate data? Typically, to make ...

Read More Source

December 31st, 2015

In Technology

No Comments

Websockets are an all or nothing proposition

Interesting:

I’m also a heavy websocket user, and agree with most points. I have previously used websockets on top of a traditional web app, and have been disappointed with the results.

My opinion now is that websockets are an all-or-nothing proposition. And I have gone all-in. My latest project has:

- websockets-only api (except a some image uploads and oauth/login)

- https-only

- single page app, central store with observable data, using a single connection servicing the whole application

- using SignalR (supports fallbacks for IE9 ...

Read More Source

December 31st, 2015

In Technology

No Comments

Any growing Rails app runs into the problem of fat models

This is a good comment:

A lot of things in Rails are are anti-patterns in large code bases, but pragmatic in small ones. ActiveRecord itself is a prime example: when you start an app, putting your business logic directly in ActiveRecord objects works pretty well in most cases, but later on as the models proliferate and grow, you realize that some of them contain business logic which is far too complex to warrant being conflated with persistence concerns. The result ...

Read More Source

December 18th, 2015

In Technology

No Comments

How to find a major security breach at Facebook

Facebook offers a bounty for bugs, but Facebook did not play fair on this one. Instead, Facebook immediately reached out to the hacker’s employer and accused the hacker of unethical behavior. The Facebook CSO later offered the cheap excuse that the hacker might have been working in an official capacity for the employer. The more likely story is that Facebook was embarrassed and wanted this story to be remain hidden. Interesting:

With the newly obtained AWS key, I browsed several ...

Read More Source

December 17th, 2015

In Technology

No Comments

How to test the Salesvoice skill for the Amazon Echo

1.) if you have access to a Salesforce account, use that. If there is an admin at your company who manages the Salesforce account, talk to them. Otherwise, you can sign up for a free 30 day trial on Salesforce. Be sure to go for the Enterprise Edition.

2.) You must enable the SalesVoice Connected App

3.) Relax IP restrictions, or whitelist the IP from which your Amazon Echo will be in use

4.) You will need to enter 4 pieces of ...

Read More Source

December 15th, 2015

In Technology

No Comments

The importance of the Module class to NodeJs

Interesting:

The Module type found in module.js has two main roles inside of Node.js. First, it provides a foundation for all Node.js modules to build off of. Each file is given a new instance of this base module on load, which persists even after the file has run. This is why we are able attach properties to module.exports and return them later as needed.

The module’s second big job is to handle Node’s module loading mechanism. The stand-alone require function that we ...

Read More Source

December 15th, 2015

In Technology

No Comments

Require and Export in NodeJs

Interesting:

Finally, the last thing to consider is what happens when you directly export a function:

var powerLevel = function(level) { return level > 9000 ? “it’s over 9000!!!” : level; }; module.exports = powerLevel;

When you require the above file, the returned value is the actual function. This means that you can do:

require(‘./powerlevel’)(9050);

Which is really just a condensed version of:

var powerLevel = require(‘./powerlevel’) powerLevel(9050);

Source

December 14th, 2015

In Technology

No Comments

How to build conversations via the Amazon Echo

If you read my post about the startup I was this year, then you know the toughest challenge we faced was the building the finite state machine that could handle conversations. So I read this about the Amazon Echo with great interest:

My task would have ended here if creating an event would only require a date and time. But to be useful, I would like to include a duration, a topic, perhaps even a location. To be even more ...

Read More Source

December 13th, 2015

In Technology

No Comments

Using a glottal stop to force the Amazon Echo to correctly pronounce “tw”

This an impressive bit of phoneme hacking, by Joseph Jaquinta:

It’s also occasionally off on some words. A particular problem for me was Alexa’s inability to correctly pronounce “tweets”. It can say “tweet” just fine, but “tweets” comes out as “wheets”. Since one of upcoming skills, Tweet Poll, centers around producing election statistics based on tweets, it uses the word “tweets” a lot.

So I was pretty excited when I saw that the new update included features for giving more fine control ...

Read More Source

December 13th, 2015

In Technology

No Comments

Dialogue designers replace graphic designers when creating voice interfaces

Over the last 30 years we’ve gotten used to the Graphical User Interfaces in software that appears on a computer screen. And most of us (who work in the tech industry) have had the experience of working with a person who, depending on their skills, will be given a title such as “graphic designer” or “user experience designer” or sometimes “product designer”.

Working with the Amazon Echo, myself and my partner have come to realize that we still need a ...

Read More Source

December 13th, 2015

In Technology

No Comments

How can I connect the Amazon Echo to wifi in an Enterprise with secure wifi?

I posted this question on the TalkEcho forum. Any suggestions are welcome.

One big problem we face (developing Enterprise apps for the Amazon Echo) is when we do a demo at a large company. Then we are typically facing the WPA2 Enterprise version of WiFi, which the Echo does not seem to support. When I look here:

https://www.amazon.com/gp/help/customer/display.html?nodeId=201549640

I see it says:

“Amazon Echo connects to dual-band Wi-Fi (2.4Ghz / 5Ghz) networks that use the 802.11 a/b/g/n standard. It does ...

Read More Source

December 13th, 2015

In Technology

No Comments

The old debates about Clojure

This was written in 2009 and I linked to it in 2011, and I just stumbled upon it again. It is interesting to remember the old fights about Clojure, the struggle for respectability that Clojure faced in those old days. For the most part, that particular fight is no longer facing Clojure.

Source

December 12th, 2015

In Technology

No Comments

The Amazon Echo Blogosphere

I will be updating this blog post as time goes by. I hope to build a complete catalog of blogs that are primarily focused on the Amazon Echo.

Here is what I found after 30 minutes of searching:

LoveMyEcho is one of the few blogs that seems devoted to the Amazon Echo

Joseph Jaquinta has a blog on LinkedIn devoted to the Amazon Echo

Engadget has a category for Amazon that sometimes covers the Echo

Gizmodo has a category specifically for the ...

Read More Source

December 12th, 2015

In Technology

No Comments

Custom commands for the Amazon Echo

Interesting:

Telling Amazon’s Alexa to play some music or add an item to your shopping list is pretty cool, but there comes a point where you start to feel the limits of the Echo’s pre-programmed voice commands. It doesn’t have to be that way anymore: Amazon and the automation wizards at IFTTT just announced a new feature that allows you to create custom voice phrases to make Alexa do whatever you want. Unfortunately, there’s a catch: every command has to ...

Read More Source

December 12th, 2015

In Technology

No Comments

Ellery Coffman demonstrates home automation via Alexa

Ellery Coffman gets Alexa to start a movie on the TV, then stop it, and then play an album on the record player, and then stop it. This is impressive in all ways, except I was surprised at how slow it was. I’ve got an Alexa app running on a Rackspace server, which pulls data from Salesforce, and the response is much faster, despite the fact that there are 2 API calls involved, and despite the fact that Salesforce ...

Read More Source

December 4th, 2015

In Technology

No Comments

The advantages of a Natural Language Processing interface

“admin” of Daily Hacker News is critical of my ideas regarding a Natural Language Processing interface for software. Some of their points conflate Natural Language Processing with the text version of Natural Language Processing, and right now I’m mostly working on Natural Language Processing via voice interfaces, so I’m going to reply with voice examples, so as to remind everyone that Natural Language Processing is a broad topic.

“admin” starts with this point:

Let’s go back to Lawrence’s example. Posit, ...

Read More Source

December 2nd, 2015

In Technology

No Comments

People want to be able to talk to their computer

When GUI interfaces were new in the 80s we were told they were better than command line interfaces because they offered “discoverability”. But it turned out they didn’t offer nearly enough discoverability, so in the 90s “Wizards” became big things, especially in the Microsoft world. A Wizard provided the context that a GUI by itself did not. A Wizard would tell a user “Do this, now do this, now do this, and then do this.” But Wizards were visually complex. ...

Read More Source



Check out my book:





RECENT COMMENTS

August 11, 2018 8:10 pm

From lawrence on Why does anyone use Python?

"I should add, I feel that Python 1.x and 2.x added something to the world of technology. Early Python was both..."

August 11, 2018 10:44 am

From lawrence on Why does anyone use Python?

"Ryan, about this: " the default should be to not try to install packages system-wide". Agreed. I think that is..."

August 10, 2018 8:23 pm

From Ryan on Why does anyone use Python?

"I suspect your problem is that you missed the step in the instructions where it told you to run all those comm..."

August 5, 2018 4:41 pm

From Howard Ormsby on A developer asks for help, then gets angry at those who help them

"This one is very intense reading thank you..."

July 27, 2018 12:04 am

From Just An Observer on Why do women drop out of co-ed sports?

"Many coed leagues require a certain %age of women players, like volleyball 2 out of 5, in a lot of leagues. ..."

July 15, 2018 3:17 pm

From Anonymous on Friendly, intelligent flash cards?

"Good to know that I wasn't too late :)..."

July 15, 2018 8:32 am

From lawrence on Friendly, intelligent flash cards?

"Thank you for these comments regarding Anki. It sounds useful. I can imagine having the full context of the se..."

July 15, 2018 6:32 am

From Anonymous on Friendly, intelligent flash cards?

"Anki limits how much you review. I have 10000+ cards across my decks but the program arranges them in such a w..."

July 6, 2018 3:42 pm

From lawrence on Docker is the dangerous gamble which we will regret

"RK, your comment is irresponsible and pure FUD. These technologies are not being deprecated. Some of them have..."

July 5, 2018 10:21 am

From RK on Docker is the dangerous gamble which we will regret

"Deprecation warning: The Packer, Artifact Registry and Terraform Enterprise (Legacy) features of Atlas will no..."

June 25, 2018 11:58 am

From Barry Jones on Docker is the dangerous gamble which we will regret

"Solid read. I will say that it seems like there is a lot of momentum growing around Ansible these days. It see..."

June 21, 2018 11:25 pm

From Anon on Docker is the dangerous gamble which we will regret

"Thank you for this. Sometimes I feel like I'm living in a bizarre, pro-Docker for everything world. I am prima..."

May 30, 2018 4:35 pm

From Thomas on Docker is the dangerous gamble which we will regret

"Thank you for this beautifully written and relevant piece. I think it has saved me a whole world of pain. ..."

May 29, 2018 10:46 am

From lawrence on Docker is the dangerous gamble which we will regret

"Andrew Reilly, thank you for writing. You make several good points, especially this: "Also an extra layer o..."

May 28, 2018 11:17 pm

From Andrew Reilly on Docker is the dangerous gamble which we will regret

"Nice article, thanks! I think that it's interesting to also look around at the problem from a greater dista..."