## Technology

January 8th, 2017

# This year’s goal: move away from WordPress

Like a lot of people, I first used WordPress because it was quick and easy to get going, and because my designer knew it very well. But I dislike the dependence on MySQL, I dislike its slowness, and I dislike the security problems that I’ve had (this site has been hacked twice).

For my modest goals, I only need a static site, so I plan to switch to Cryogen. I was interested by this article:

For a while I ...

January 8th, 2017

# Different people have different strengths

I love this :

I’ve had similar self-doubt in the past, I considered myself fast but sloppy. I changed my opinion after a workshop on different team roles at my previous job. A lot of it was boring workshop fluff, but I loved the core message: that many personality traits aren’t purely positive or negative. Perfectionists are nitpickers. Fast developers are sloppy. Experienced ones overthink stuff. Bleeding-edge evangelists ruin long-term stability etc. There are two sides to every coin. We tend to ...

January 8th, 2017

# Can event logs, as a software architecture, really work?

This is an interesting bit of criticism of this architecture, an architecture which certainly has gotten a lot of attention over the last 4 years:

I have worked on, or cleaned up, 4 different CQRS/ES projects. They have all failed. Each time the people leading the project and championing the architecture were smart, capable, technically adept folks, but they couldn’t make it work.

There’s more than one flavor of this particular arch, but Event Sourcing in general is simply not very ...

December 15th, 2016

# Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

I posted this question to StackOverflow and I would appreciate any help figuring out what the memory issue is:

Clojure / Java app dying while using XSSF and Sax parsers to import Excel files

Source

December 14th, 2016

# Ugly engineering

This sounds hideous:

I used to work on the DynamoDB team. Throwaway account because my normal account can be tied back to my real name.

“Each hash key resolves to a number of possible servers the data can be on. Data is replicated across several of these servers. For redundancy. The hash key determines which shard to use. On individual machines, each set of data is stored by a compound key of hash key and sort key (if there is a sort ...

December 13th, 2016

# Beautiful engineering

Wow. Sometimes I witness some engineering that leaves me feeling joy and inspiration. I felt that way about the Golden Gate Bridge. But sometimes I feel that way about code. For instance, right now, I just learned that the defun macro for Clojure allows this function for calculating a fibonacci number.

A fibonacci function: (defun fib ([0] 0) ([1] 1) ([n] (+ (fib (- n 1)) (fib (- n 2))))) Output: (fib 10) ;; 55

That ...

December 7th, 2016

# The crazy process of trying to release a Salesforce app

Right now I’m helping a client release an app for Salesforce in the Salesforce App Exchange. The process for releasing an app is completely insane and very poorly documented (or rather, there is an abundance of documentation for various parts of the process, but there is no summary of the process). I was lucky to stumble upon this post, which offers the nearest thing to a summary of the process:

1) Create a Dev Org

2) Become a Salesforce.com Partner

3) Create Test ...

November 26th, 2016

# Clojure for XML

This is a fantastic overview of different approaches:

Zippers are probably the easiest way to manage xml – once you grok them.

Zippers are a strange beast. Wikipedia describes them as:

A technique of representing an aggregate data structure so that it is convenient for writing programs that traverse the structure arbitrarily and update it’s contents…

I like to think of a zipper as a kind of pointer to part of a tree – at any time if you have a tree of nodes ...

November 26th, 2016

# How far can code be optimized

This is funny and interesting:

When you’re competing with another player, they will probably find a way to beat your score by just a few points. Let’s say my score is 340 and a friend beats me with a score of 335. (lower is better. The score is just the number of executed instructions) What follows is a bunch of head-scratching about how you could possibly get any more cycles out of the algorithm. After an hour of staring and trying ...

November 14th, 2016

# NLP with Clojure and OpenNLP

This is a fantastic introduction:

Natural Language Processing (NPL) opens the door to the possibility of turning otherwise inert text into meaningful or, more interestingly, actionable information. It is the latter that I am interested in and what this installment will focus on. I will explore the basics of NLP using the OpenNLP library and Clojure to convert a sentence into a useful structure to store or act on. More specifically, my goal is to take simple sentences that indicate the ...

November 13th, 2016

# The best interview technique: pay people $200 to attempt some task over the weekend I like this idea: In regards to what works the best, I found that these 2 ideas work the best when combined. PAID Sample project assignment (err on the side of paying fairly — say$100+/hour for estimated completion time — if the problem should require 2 hours to complete, offer $200) Bring the candidate in and discuss the solution. Let the candidate talk about their design decisions, challenge them as you would any team member and let them provide their reasoning. Paying candidates to work on a ... Read More Source November 5th, 2016 No Comments # The perfect Javascript file uploader? This looks amazing: The default behavior of Fine Uploader is to immediately attempt to upload files as they are selected. One option allows you to simply queue all files, and then start uploading at a later time by calling uploadStoredFiles() on your Fine Uploader instance. You can also easily allow your users to edit the names of each submitted file before uploading. This Fine Uploader instance below also demonstrates the edit file name feature, which allows you to edit the name of ... Read More Source October 31st, 2016 No Comments # Stripe almost got burned by Bitcoin It’s amazing how many disasters seem to happen whenever companies try to use crypto-currency: For a variety of reasons, it is sometimes necessary to refund bitcoin transactions: For example, a customer cancelling their order; accidentally sending in the wrong number of bitcoins; or even sending in the correct number of bitcoins, but not within the requisite time window, resulting in their value being lower than necessary. Consequently, Stripe allows for bitcoin transactions to be refunded — with the caveat that, for ... Read More Source October 27th, 2016 No Comments # Software doesn’t need options At least for the consumer. An API for a developer is something else completely. Great software is: Opinionated Options are a cop out. It’s the designer’s job to figure out the best way to use the product, and delegating that choice to the user is pure laziness. Hive only resorts to options as a last resort. Source October 3rd, 2016 No Comments # How process becomes bloated and bureaucratic over time This is good: Checkout/Ordering team: “Your Payment Processing release broke Checkout in production.” Payment Processing team: “Didn’t know this would affect Checkout. Had no time to look into it because we were too busy working on the new payment functionality in the iOS app.” Mobile App Product Visionary: “I want my own dedicated team. This happened because Payment Processing is spread too thin to work on the app and the backend at the same time.” Development Manager: “Org chart change – let’s have all ... Read More Source September 18th, 2016 No Comments # Rapidly Exploring Random Trees Interesting. It says the RRT proceeds by breaking the Voronoi spaces into smaller Voronoi spaces. The algorithm has a bias towards unexplored territory, unlike a purely random algorithm, which tends to loop back to explored space. I didn’t dig into the algorithm, but I can imagine how this would be useful in some situations. Source September 18th, 2016 No Comments # As a consolidation client for Kinesis, ZeroMQ is much better than Kafka This is fantastic: With the problem of real-time log consolidation solved with ZeroMQ, supporting durable storage of logs for later post-processing becomes a smaller and a much more manageable challenge. For example, off the shelve solutions like Logstash are capable of capturing data from ZeroMQ and publishing it to a variety of destinations. In our particular situation at Auth0, we are already using AWS Kinesis, ElasticSearch, and Kibana as a log processing pipeline in other parts of our operations. As such we ... Read More Source September 18th, 2016 No Comments # I always learn what I need to know about a year too late At Celolot I wrote a bunch of Clojure apps that talked to each other over Redis thanks to Carmine/Nippy. We passed around maps, and each app checked a field in the map to see if the map was a message that it was suppose to respond to. But I bet this is a thousand times faster: ZeroMQ does not have a first class notion of a topic, yet it does have a first class concept of a subscription filter. Subscription filters ... Read More Source September 18th, 2016 No Comments # Camille Fournier on ephemeral nodes in etcd I missed this the first time. In etcd a service needs to constantly update the fact that it is alive. If it dies, its info disappears. So other services can contact etcd and ask “Who is alive right now?” ZooKeeper allows stateful connections. Thick clients. But what happens when you try to use ZooKeeper from a language that does not allow threads? Do you block on the only thread that you have? For pub-sub she recommends RabbitMQ or ZeroMQ. Source September 18th, 2016 No Comments # Why would I ever want to use Docker? I’m trying to be open minded about Docker, but it is a struggle to grok why this is good. Do I want to bind all dependencies to my app? Great, then I’ll build uberjars that combine everything into 1 binary. But how will I work with Ruby On Rails? That’s easy: it is time to move away from Ruby On Rails. If Ruby can only be kept alive by using Docker, then we should give up on Ruby. How will ... Read More Source September 18th, 2016 No Comments # Unary Numeral System Interesting: Addition and subtraction are particularly simple in the unary system, as they involve little more than string concatenation. The Hamming weight or population count operation that counts the number of nonzero bits in a sequence of binary values may also be interpreted as a conversion from unary to binary numbers. However, multiplication is more cumbersome and has often been used as a test case for the design of Turing machines. Compared to standard positional numeral systems, the unary system is inconvenient ... Read More Source September 18th, 2016 No Comments # It took me 10 years to write that 5 line app There is the story where Picasso paints a quick sketch and wants a huge amount of money for it, and the art collector says “It only took you 15 minutes to make that sketch!” and Picasso says “I spent 60 years learning how to do that in 15 minutes.” Maybe something like this applies when it comes to microservices. When I advocate for microservices, I bring 17 years of experience to the conversation. The first 7 of those years meant ... Read More Source September 18th, 2016 No Comments # For the last 10,000 years, the banking industry has relied on eventual consistency Why do computer programmers feel smug offering the following example, when it is so clearly wrong? Transactions, to a database, are important, because banks must keep track of money. Suppose a person were to move$100 from their Savings account to their Checking account. Suppose $100 is added to checking, and just then the electricity dies, and the computers die, before$100 can be subtracted from their Savings account. The person now has an extra $100, which they should not ... Read More Source September 17th, 2016 No Comments # I keep really good company This article by Yegor Bugayenko cites the following people: Edsger W. Dijkstra Alan Kay Paul Graham Richard Mansfield Eric Raymond Jeff Atwood Linus Torvalds Oscar Nierstrasz Rich Hickey Eric Allman Joe Armstrong Rob Pike John Barker Lawrence Krubner (me) Asaf Shelly I must be doing something right to get cited alongside Edsger W. Dijkstra, Alan Kay, Paul Graham, Linus Torvalds, and Rich Hickey. Source September 17th, 2016 No Comments # When is consistency needed in a database? This is a great comment : Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. “Low level” operations as part of settlement may us transactions, but the bank system is “designed” (more like it has grown by accretion) to function almost entirely by settlement and reconciliation rather than holding onto any notion of consistency. The real world rarely involves ... Read More Source September 11th, 2016 No Comments # Why did Salesforce use markup tags for VisualForce? By now, I think we can agree, using HTML for the web was a mistake. But Sir Tim Berners-Lee could not have known that back in 1989. He was thinking that he could create a Semantic Web, and for that, the use of a markup language was defensible. Only in retrospect do we see that HTML was mostly used as a GUI for TCP/IP. Eventually Sir Tim Berners-Lee conceded the point, and so he went off and created RDF. But ... Read More Source September 5th, 2016 No Comments # Comparing data sets that are imbalanced Interesting: Research on imbalanced classes often considers imbalanced to mean a minority class of 10% to 20%. In reality, datasets can get far more imbalanced than this. Here are some examples: 1.) About 2% of credit card accounts are defrauded per year1. (Most fraud detection domains are heavily imbalanced.) 2.) Medical screening for a condition is usually performed on a large population of people without the condition, to detect a small minority with it (e.g., HIV prevalence in the USA is ~0.4%). 3.) Disk ... Read More Source September 5th, 2016 No Comments # Base 10 numbers need log(10) digits to be described I’m sure I knew this example once, but I’d completely forgotten it, and it is so perfectly obvious when we talk to non-technical people and they ask for an example of what logarithmic growth looks like: In mathematics, logarithmic growth describes a phenomenon whose size or cost can be described as a logarithm function of some input. e.g. y = C log (x). Note that any logarithm base can be used, since one can be converted to another by multiplying ... Read More Source September 5th, 2016 No Comments # Accessing any single element in an array takes constant time as only one operation has to be performed to locate it. This seems like it could be used as a trick question that would trip me up during a job interview: An algorithm is said to be constant time (also written as O(1) time) if the value of T(n) is bounded by a value that does not depend on the size of the input. For example, accessing any single element in an array takes constant time as only one operation has to be performed to locate it. However, finding the minimal ... Read More Source September 5th, 2016 No Comments # Beware Big O Notation in higher level languages This essay has a nice bit of details about a general point that needs to be made: Here we get the result that will be counterintuitive to many. No matter how large n gets, the Array List still performs better overall. In order for performance to get worse, the ratio of inserts to iterations has to change, not just the length of the collection. Note that isn’t an actual failure of Big O analysis, it is merely a common human failure ... Read More Source September 5th, 2016 No Comments # The gamification of Slack makes it addictive but also makes it a drain Very interesting: Everything bad about Slack is fundamentally Slack’s fault. Slack’s sane default for a new user is to play a sound and send a desktop notification every time anything happens anywhere. There is no way to simply turn off the screaming red circle on the dock icon. There is no way to simply fold away the sidebar so you can focus on whatever it is you are trying to do. These UI nitpicks help us understand Slack’s conception of the ... Read More Source September 5th, 2016 No Comments # It’s not even XML! Dave Winer said he is deeply depressed. Sometimes he seems frightened by the march of change. I suppose that is an aspect of depression. His initial fear of JSON seems like an expression of fear: I’ve been hearing, off in the distance, about something called JSON, that proposes to solve a problem that was neatly solved by XML-RPC in 1998, the encoding of arrays and structs in a format that could easily be processed by all programming languages. The advantage ... Read More Source September 5th, 2016 No Comments # Finding novelty via friends is a metric not yet captured by recommendation engines This is a good point: The traditional regime of recommendation systems has been obsessed with (1). What’s the uplift of recommendation algorithm A vs recommendation algorithm B? Which is driving more click-thrus and conversions? There’s something fundamentally broken in the this way of thinking though. I don’t care what the computer says. I care about what my friends say. The meaningful music I’ve discovered over the last 10 years has been music liked by someone I respect. Friends. Other musicians I ... Read More Source September 5th, 2016 No Comments # Detecting voting rings with HyperLogLog While this seems like a clever trick, I typically want a lot of metrics regarding voting, so throwing away the metadata doesn’t seem like an option to me. As an aggregate tool whose only purpose is finding voting rings, maybe this useful maybe? But consider what would happen if we created a HyperLogLog counter for every user on Reddit, and any time that a user receives an upvote, we update the corresponding HyperLogLog counter with the id of the user ... Read More Source September 5th, 2016 No Comments # Great browser software in 1996 These technologies sound so amazing, I wonder what happened to them? Atlas is the precursor to the next step in Navigator’s evolution, Version 3.0. It’s “alpha” code; in other words, that celebrated tower in Pisa is more stable. When it works, Atlas promises to deliver: VRML viewing. VRML (Virtual Reality Modeling Language) is one of the competing standards for expressing three-dimensional information in compact form. VRML documents can be static or interactive. For example, you could create a Web page that ... Read More Source September 5th, 2016 No Comments # The Big O cheat sheet I just discovered the Big O cheat sheet and I think this is very useful: Source September 4th, 2016 No Comments # Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring I just realized that I’ve been confusing Faraday and Maxwell. It’s Maxwell who did the 4 equations. Faraday’s breakthrough came when he wrapped two insulated coils of wire around an iron ring, and found that upon passing a current through one coil a momentary current was induced in the other coil.[2] This phenomenon is now known as mutual induction.[44] The iron ring-coil apparatus is still on display at the Royal Institution. In subsequent experiments, he found that if ... Read More Source August 19th, 2016 No Comments # HAProxy for MySQL Three years ago, I was intrigued to read that Twitter had implemented dynamic network topologies. Something similar is becoming more common, thanks to the use of Nginx/HAProxy. One can use this for many things, such as connecting to a pool of MySQL servers. However, there are some downsides worth remembering: You have to be careful to tune your xinetd script to increase the cps and per_source limits accordingly to the load that your pool of servers generate with this proxy forwarding. Assuming that ... Read More Source August 19th, 2016 No Comments # Clever uses of a custom TCP stack Interesting: Fun anecdote, at Blekko we had people who tried to scrape the search engine by fetching all 300 pages of results. They would do that with some script or code and it would be clear they weren’t human because they would ask for each page right after the other. We sent them to a process that Greg wrote on a machine that did most of the TCP handshake and then went away. As a result the scrapers script would hang ... Read More Source August 16th, 2016 No Comments # When algorithms betray you Interesting: You decide to share an article about Brexit from “The Atlantic”, which will somehow shed light to all your friends as to why it happened. The article is 1,000 words long — you only read half of it, but that’s good enough. It captures all the arguments you’ve been wanting to make for the past two months to your friends. Will this be the Facebook post that finally spurns your friends into action? You realize your Facebook friends all agree with your ... Read More Source August 11th, 2016 No Comments # Only good leadership can create good software This is very true, and it applies to more than software: Thirteen years ago, Eric Raymond’s book The Cathedral and the Bazaar (O’Reilly Media, 2001) redefined our vocabulary and all but promised an end to the waterfall model and big software companies, thanks to the new grass-roots open source software development movement. I found the book thought provoking, but it did not convince me. On the other hand, being deeply involved in open source, I couldn’t help but think that it ... Read More Source August 11th, 2016 No Comments # Clever hacks whereby sites can see aspects of your browser history Very interesting hacks: Up until mid-2010, any rogue website could get a good sense of your browsing habits by specifying a distinctive :visited CSS pseudo-class for any links on the page, rendering thousands of interesting URLs off-screen, and then calling the getComputedStyle API to figure out which pages appear in your browser’s history. After some deliberation, browser vendors have closed this loophole by disallowing almost all attributes in :visited selectors, spare for the fairly indispensable ability to alter foreground and background ... Read More Source August 8th, 2016 No Comments # The control mania of Scrum This is very good: Scrum inhibits deep understanding and innovation This is actually my biggest gripe about Scrum. As mentioned above, in Scrum, the gods of story points per sprint reign supreme. For anything that doesn’t bring in points, you need to get the permission of the product owner or scrum master or someone who has a say over them. Refactoring, reading code, researching a topic in detail are all seen as “not working on actual story points, which is what you ... Read More Source August 8th, 2016 No Comments # The daily Scrum meeting is a pointless ritual I love this: The daily standup deserves a blog post of its own. This religious ritual has become a staple of every team in the world. Ten minutes of staring into the void, talking about what you did while no one else listens, because they were in the middle of something five minutes ago and will go back to it in another five minutes, and waiting for everyone else to finish. I know this sounds cynical, but it is the ... Read More Source August 8th, 2016 No Comments # If you measure programmers by points then they will optimize for points The worst thing with Agile/Scrum is that you can hit all your 2 week targets and yet a year later have software that no one wants to use, so to the extent that Agile is suppose to reduce risk, it fails completely. But why does it fail? This is good: No matter how you define story points, the real issue with them doesn’t go away. The main purpose of points is making planning more reliable, and providing a temporal perspective ... Read More Source August 8th, 2016 No Comments # Problems with the Scrum process This is a great conversation: cageface says: But the author’s criticisms of the incentives of Scrum are on point I think. Because the stories are always articulated in terms of user facing features they encourage developers to hack things together in the most expedient way possible and completely fail to capture the need to address cross cutting concerns, serious consideration of architecture, and refactoring. This is how you can get two years into a project and have managers and clients that think that ... Read More Source August 3rd, 2016 No Comments # The JustWorks job interview One of the most popular posts I’ve written is “Embarrassing code I wrote under stress at a job interview“. People get a laugh out of the dumb things I do doing job interviews. Here is another such post. I did a job interview at JustWorks. They asked me to write the code to solve this problem: The cost of a stock on each day is given in an array, find the max profit that you can make by buying once ... Read More Source August 3rd, 2016 No Comments # Is Docker ready for production? I prefer uberjars to Docker, and I prefer fat binaries, such as those allowed by Go, over Docker. And still there is the question “Is Docker ready?” Senex says: I’ve been tracking the beta for a while. I’m confused about this announcement. These issues still seem unresolved? (1) docker can peg the CPU until it’s restarted (2) pinata was removed, so it can’t be configured from CLI scripts (3) it’s not possible to establish an ip-level route from the host to a container, which ... Read More Source July 27th, 2016 No Comments # How to package dependencies in Golang Very interesting, as I’m a fan of the idea of “vendoring”: An attempt to fix packaging in Go: Manul is a vendoring utility for Go programs. What’s the reason for yet another utility? Because all other vendor utilities suffer from the following: Some wrap the go binary and spoof the GOPATH env variable. You will have a non-go-gettable project which needs additional software in order to compile and run; Some copy the source code of dependencies into the vendor directory: It will be nearly impossible to find ... Read More Source July 26th, 2016 No Comments # Google Maps does not name Nachitschewan? I can not find a zoom level where Google Maps names Nachitschewan. Is this a political thing? Source July 25th, 2016 No Comments # The advantage of a weight-balanced B-tree A very interesting data structure of which I know nothing: Source July 22nd, 2016 No Comments # A string of characters is best parsed using a finite state machine I already knew this, but I don’t think I knew the extent of what was sacrificed to give Perl backreferences s: Notice that Perl requires over sixty seconds to match a 29-character string. The other approach, labeled Thompson NFA for reasons that will be explained later, requires twenty microseconds to match the string. That’s not a typo. The Perl graph plots time in seconds, while the Thompson NFA graph plots time in microseconds: the Thompson NFA implementation is a million ... Read More Source July 21st, 2016 No Comments # The problems with Clojure The conflict between Object Oriented Programming and Functional programming comes up a lot with Clojure, because to do any serious work you have to eventually use some Java, which takes you back into the world of Object Oriented Programming. This sums up my feelings: The thing I like most about Elixir is the low friction between it and it’s host language, Erlang. Erlang is a functional language right from the start, and the BEAM is designed to run a functional ... Read More Source July 21st, 2016 No Comments # The RegX that killed Stack Overflow A great post-mortem of a crash at Stack Overflow: The regular expression was: ^[\s\u200c]+|[\s\u200c]+$ Which is intended to trim unicode space from start and end of a line. A simplified version of the Regex that exposes the same issue would be \s+$which to a human looks easy (“all the spaces at the end of the string”), but which means quite some work for a simple backtracking Regex engine. The malformed post contained roughly 20,000 consecutive characters of whitespace on a ... Read More Source July 21st, 2016 No Comments # The difference between Kafka and Kestrel For my purposes, for the next 6 months, it seems that Kestrel will be all I need. I’d have to be very successful before I would need anything as complex as Kafka. The biggest conceptual difference is that Kestrel is a simple stand alone queue where as Kafka is a full fledged queuing system. Kestrel runs on a single machine and has no concept of clustering or failover or any other features you might expect in a queuing system. Instead the clients ... Read More Source July 21st, 2016 No Comments # RabbitMQ is difficult, Kafka is awesome, Kestrel is beautifully simple A great review of these 3 queues: Not mentioned below is that RabbitMQ works hard to guarantee delivery of a message, so it is slow, but that is because it is in some ways doing more than Kafka. RabbitMQ: I created 4 queues, wrote a ruby client and started inserting messages. I got a publishing rate of about 20k/s using multiple threads but I got a few stalls caused by the vm_memory_high_watermark, from my understanding during those stalls it writing to disk. ... Read More Source July 21st, 2016 No Comments # Kestrel is as simple as Redis but Kestrel is an actual queue I feel stupid that I didn’t look at Kestrel sooner. I feel especially stupid that I’ve been using Redis when I didn’t actually need it. Redis has a beautifully flexible API which makes it tempting to use Redis for everything, but Redis is really a cache that focuses on speed above all else. Everything in Redis has to fit in memory, and Redis will drop anything that can’t fit in memory. Also, the stuff in Redis can be mutated, ... Read More Source July 21st, 2016 No Comments # Only use Amazon SQS if you need a high-latency high-concurrency service Also interesting, don’t use Amazon SQS unless your needs fit this very specific model where you can deal with the latency and make up for it by being highly concurrent. I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is ... Read More Source July 21st, 2016 No Comments # For Apache Storm, use Kestrel There is a lot in this thread that is interesting. This is coming from Nathan Marz, the guy who invented Storm. He says he uses Kestrel, so that is a big endorsement. You want to make sure that your spout source can support the out of order acking that Storm requires for guaranteed message processing. We use Kestrel because it has this property and is the simplest. RabbitMQ is another good one to consider. Source July 20th, 2016 No Comments # The problem with RabbitMQ Seems like RabbitMQ is good but not perfect: Much like you’d chuck memcached on each of your web servers and access them in a ring, Darner can occupy a small niche on each box in your fleet’s resources. Tens of MB of RAM and negligible CPU opens up hundreds of gigabytes of queue spool per node. As queue size grows, memory usage remains constant. Contrast this with Redis, which is speedy but limited in queue size to what will fit ... Read More Source July 20th, 2016 No Comments # ZeroMQ is the opposite of a queue Reading this again, several years after the first time I read it, and now it occurs to me: I would use ZeroMQ only in exactly those situations where I would not use a queue. If I only need a queue, I can use Kestrel. But if I need some messaging pattern that is not supported by any queue, then I would use ZeroMQ. That is, if I had a truly unique situation that needed a unique pattern, then ZeroMQ would ... Read More Source July 20th, 2016 No Comments # Everything you need to know about queues This looks like an amazing resource and I’m sure I’ll go back and read more soon. Source July 20th, 2016 No Comments # The New York Times uses RabbitMQ RabbitMQ is written in Erlang, so I was inclined to think well of it, though I heard criticism of it. And then the New York Times used it. A very surprising vote of confidence in RabbitMQ: This architecture – Fabrik – has dozens of RabbitMQ instances spread across 6 AWS zones in Oregon and Dublin. The instances are organized into “wholesale” and “retail” layers. Connection to clients is via websockets/sockjs. Upon launch today, the system autoscaled to ~500,000 users. Connection times ... Read More Source July 20th, 2016 No Comments # Gauges uses Kestrel which is based on work from Blaine Cook Why does Kafka get all the attention if Kestrel is so reliable? I assume this is because Kafka can do so much more, though of course the devops work of Kafka can be frightening. I didn’t know about the Blaine Cook connection (of Twitter fame): Before I get too far a long with this fairy tail, let’s talk about Kestrel — what is it and why did I pick it? Kestrel is a simple, distributed message queue, based on Blaine Cook’s ... Read More Source July 20th, 2016 No Comments # Kestrel versus Resque Interesting note about these queues, which certainly convinces me to use Kestrel: There are a lot of things that could be done with either Kestrel or Resque. Because Resque is backed by Redis, you have to remember that all of the messages waiting to be processed have to be able to fit in the RAM of the Redis server, with Kestrel you could queue millions or billions of messages and then start to pull them off. The biggest difference between the ... Read More Source July 20th, 2016 No Comments # How bad can MongoDB be with Dirty Reads? Very worrisome, though no sane person would use MongoDB to track something involving money. That’s simply not what MongoDB is for. How bad are dirty reads? Read uncommitted allows all kinds of terrible anomalies we probably don’t want as MongoDB users. For instance, suppose we have a user registration service keyed by a unique username. Now imagine a partition occurs, and two users–Alice and Bob–try to claim the same username–one on each side of the partition. Alice’s request is routed to the ... Read More Source July 20th, 2016 No Comments # How to return a disallowed field in GraphQL Best Practices for GraphQL are still in a very immature stage, but these two ideas both have some merit: First, return null for the requested field. This seems to work well in cases where there is no real harm in asking for a particular set of data and no real harm in denying it. A good example would be asking for the email of a user where the backend only provides the user’s email to that user themselves. If I request my ... Read More Source July 20th, 2016 No Comments # Why has Google released gRPC This is interesting: Google has been using a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across our data centers for over a decade. Our internal systems have long embraced the microservice architecture gaining popularity today. Having a uniform, cross-platform RPC infrastructure has allowed for the rollout of fleet-wide improvements in efficiency, security, reliability and behavioral analysis critical to supporting the incredible growth seen in that period. Stubby has many great features – ... Read More Source July 13th, 2016 No Comments # Why I hate Ruby On Rails I’m trying to help a friend with a very small Ruby On Rails project, where she needs a few elements of the interface tweaked. But this is Ruby On Rails, so of course I’m losing 3 hours trying to get setup. I’ve been drawn into the endless swamp of upgrading various libraries, which seems to be an automatic part of dealing with Ruby On Rails. If this was Clojure, or NodeJS, setup would have taken 5 minutes and I would ... Read More Source July 13th, 2016 No Comments # Mixins are harmful — the Javascript edition Over the years there have been arguments against mixins, but the dominance of Object Oriented Programming meant that mixins still survived and had theoretical justification. Now, the spread of the Functional Paradigm has put renewed pressure on mixins. Dan Abramov at Facebook just came out with an essay against mixins: “How do I share the code between several components?” is one of the first questions that people ask when they learn React. Our answer has always been to use ... Read More Source July 10th, 2016 No Comments # Edges is just a wrapper around the real results that adds extra data used for slicing the result cursor I think the “edges” and “nodes” terminology is one of the most confusing things about React / Relay / GraphQL, at least at first: Relay connection defines a relationship an an object that has two root fields: edges and pageInfo. Edges is just a wrapper around the real results that adds extra data used for slicing the results (cursors), PageInfo has metadata related to the current page. This is how it looks like: connectionName { edges { ... Read More Source July 10th, 2016 No Comments # Make the Web compatible? It must be 1999 again because someone is writing about browser incompatibilities: Users won’t switch browsers, they’ll switch sites You might think that users will switch browsers to use your site. But many won’t or can’t. Users have no patience for things that don’t work, and they’ll just go to a competitor’s site instead. Failing at a critical point could turn a potential user away forever. According to Akamai, 32% of users who encounter a problem on your site are less likely to make ... Read More Source June 30th, 2016 No Comments # The Agile process of software development is often perverted by sick politics For those of you who don’t want to read this whole essay, here is the summary: The word “agile” has a plain meaning in standard English, and that meaning was considered something positive by software developers, so much so that the most successful new development process of the last 30 years calls itself “Agile”. However, at many of the companies that I have worked, actual agility is suppressed because of various political factors. Fear wins out over trust. Instead of actual ... Read More Source June 29th, 2016 No Comments # How much time and energy should a computer programmer spend learning a proprietary server system? By itself, this is an anecodote about Google’s particular system: (but see my point at bottom) Is there a lesson in this? Well, if I were in the business of programming Google App Engine, a few days’ effort up front to get it going might seem not to be a big deal. However, when I visit a team who have just added a new person, I invariably find that person struggling to set up their workstation. There’s usually someone in the room ... Read More Source June 29th, 2016 No Comments # Is Phoenix going to take over the tech industry? If not, why? This has been one of the big puzzles of the last 30 years: if some technologies are clearly superior, then why don’t they take over? Apparently because they lack an element that appeals to management in big corporations? That’s always been the argument against Lisp: it is great for the individual master craftsman, but it doesn’t work in a big corporation full of badly paid, mediocre programmers. Erlang has been the most safe, resilient technology out there ... Read More Source June 23rd, 2016 No Comments # Pagination with React / Relay / GraphQL This is an awesome article: Relay proposes a standard to define a has-many relationship for a GraphQL field. This standard defines a common structure that allows Relay to paginate and filter the results in an efficient way by using cursors, which I’ll explain in a bit. This is the definition of a Relay connection (from the Relay connection specs): Relay’s support for pagination relies on the GraphQL server exposing connections in a standardized way. In the query, the connection model provides a standard ... Read More Source June 15th, 2016 No Comments # Demanding ssh key login to a server is safer than allowing passwords This is good and true: Require ssh key authentication We tend to avoid passwords for logging into servers. There was a lot of discussion around this after Bryan’s original guide came out, but I tend to fall into this camp as well. Here are a few notes on this: ssh keys are better than passwords only because they contain and require more information. Passwords can be brute forced. Guessing a public key is so essentially impossible that they can be considered perfectly secure What ... Read More Source June 14th, 2016 No Comments # Object/Relational Mapping is rarely worth it I think this can be stated simply: for your basic operations (CREATE, UPDATE, READ, DELETE) then an ORM offers some convenience. But ORMs break down as soon as you have a complex query. And yet, if you start using an ORM because it helps you with the simple queries, it will seem natural to you to extend it and try to use it for the complex queries. This is a strained analogy to the Vietnam war: Object systems are typically characterized ... Read More Source June 11th, 2016 1 Comment # AWS is inappropriate for small startups because its complexity demands a specialist Sean Hull references a conversation that he and I had in Slack. I would like to expand on the argument that I made then. You might want to read his essay first, where he makes these points about AWS: 1. Over 70 services offered 2. Still complex to build high availability 3. Need a dedicated devops 4. Orchestration involves many moving parts 5. Troubleshooting failed deployments is difficult At the time of our conversation I had crashed an AWS instance and we were having trouble fixing ... Read More Source June 11th, 2016 No Comments # GraphQL is the best thing about React / Relay / GraphQL React / Relay / GraphQL: The big breakthrough here is obviously GraphQL. Not React or Relay, since there are many competing implementations that do similar things, but GraphQL. Even though there are graph query languages out there (Gremlin, etc) they were not suited to querying JSON over the wire. GraphQL is ideal. For my next project, I hope to do a Clojure implementation of most of GraphQL, because I think it can be married to Om.Next in a very powerful way. Actually, ... Read More Source June 10th, 2016 No Comments # Krubner, you saved my day Aw man, after all the mistakes I make, it is nice to know I help people sometimes: Source June 10th, 2016 No Comments # Sometimes I make a bad decision It’s very, very, very rare, almost unheard of, but sometimes I make a mistake. I was helping at Open Road this spring. Sean Hull was also helping. Open Road runs a bunch of WordPress sites, such as The Line Up. All of the images for these sites were stored on a central NFS server. All of the machines were starving for hard disk space. The previous Director Of Technology had made many, many bad decisions, including the decision to save ... Read More Source June 7th, 2016 No Comments # The Belief that Tests are Smarter than Code Telegraphs Latent Fear or a Bad Process This part is really good: Programmers have a tacit belief that they can think more clearly (or guess better) when writing tests when writing code, or that somehow there is more information in a test than in code. That is just formal nonsense. The psychological perspective is instructive here, and it’s important because that — rather than any computational property — most drives developer behaviour. If your coders have more lines of unit tests than of code, it probably means one of several things. They may be ... Read More Source June 7th, 2016 No Comments # The problem with unit tests Interesting: Unit testing is of course not just an issue in object-oriented programming, but the combination of object-orientation, agile software development, and a rise in tools and computing power has made it de rigueur. As a consultant I often get questions about unit testing, including this real one from a recent client of mine, Richard Jacobs at Sogeti (Sogeti Nederland B.V.): My second question is about unit tests. If I remember correctly you said that unit tests are waste. First, I was surprised by that. Today however, my ... Read More Source June 6th, 2016 No Comments # Ruby On Rails is obsolete This is a strange conversation to be having in the year 2016. Ruby On Rails changed the technology industry in 2004, but it can not claim to be winning in 2016. It still lacks a story for concurrency. It is being undermined by 2 forces: 1.) the need for concurrency, which is offered by other eco-systems, such as those that run the JVM. A language like Clojure offers the high level of meta-programming that Rubyists love, but with the vast arsenal ... Read More Source June 2nd, 2016 No Comments # Sometimes peer review delays progress for many decades This is what real leadership looks like: Morris intentionally consumes H. pylori. Like Marshall, he becomes ill, but unlike Marshall, he is not completely cured by antibiotics. The infection will remain with him for three years. I wish he had been given a medal for his self-sacrifice. He was attempting to save millions from suffering. Interesting, though very disappointing: 1881 Klebs notes the presence of bacteria-like organisms in the lumen of the gastric glands. 1889 Walery Jaworski describes spiral organisms in sediment washings ... Read More Source May 22nd, 2016 No Comments # Why does Rails become awful? Interesting: Let me tell you a story about what can happen in a convoluted Rails codebase. Once, I joined an existing project. It was a huuuuge app which was running an on-line shopping community website. Complicated sales model, complicated promotions, complicated product setups, coupons, user groups, messages – it had it all. I joined them to help ship a few new features. One of my early tasks was to…add a link to something on some page. It took me few days ... Read More Source May 15th, 2016 No Comments # The dumbest name for a good architecture is “serverless” I strongly agree that the idea is good, but the name “serverless” is really stupid: So… this is backend infrastructure (i.e. it runs on a server) to let your team deploy “severless” apps… So they’re serverless. Except the server you’re running it on. And the lamba/style code they wrote and uploaded to it. But serverless, because no ops staff required. Except the ones who installed and maintain this. This is like a snake eating its own tail and wondering what hurts. Edit: despite my sarcasm, ... Read More Source May 10th, 2016 No Comments # The sad, slow way a system of cron scripts becomes ugly Do you have a chore that needs to run in the background, maybe once a day, or once an hour? Cron scripts will save you! They are the most amazingly amazing thing God has invented since Adam and Eve! For sheer wonderfulality they have no equal among the products of mortal fallen flesh! At least at first. The simple cron script is wonderfully direct and efficient. But the first can lead to a second. The second can lead to a third. ... Read More Source April 10th, 2016 No Comments # What do AWS services actually do? I love this: Data Pipeline Should have been called Amazon ETL Glacier Should have been called Really slow Amazon S3 Kinesis Should have been called Amazon High Throughput It’s like Kafka I have used this but I did not know the name: Snowball Should have been called AWS Big Old Portable Storage Use this to Get a bunch of hard drives you can attach to your network to make getting large amounts (Terabytes of Data) into and out of AWS It’s like Shipping a Network Attached Storage device to AWS These are useful names that actually describe what ... Read More Source March 27th, 2016 No Comments # The fanaticism of “test driven development” is slowly fading This was one of those trends that went too far and I’m glad to see people talking about reasonable limits: If you’re part of the Ruby On Rails community for a long time, you’ve probably read tons of articles about testing Rails application (less these days, though). Although there always have been diverging opinions on the matter, it seems the common wisdom was to say that you had to test everything: models, controllers, views and full-stack tests. Oh, and you had ... Read More Source March 26th, 2016 No Comments # RESTful APIs are dead, long live GraphQL We can argue whether RESTful APIs ever existed, since very few ever implemented HATEOAS. Ruby On Rails claimed it offered RESTful APIs, but it never offered HATEOAS. After 10 years of failure, the world is looking for something new. Interesting: Imagine we have a simple application that fetches a list of stories, and some details about each one. Here’s how that might look in resource-oriented REST: // Fetch the list of story IDs but not their details: rest.get('/stories').then(stories => // This ... Read More Source March 26th, 2016 No Comments # Picking React isn’t a technology decision, it’s a business decision Interesting: The clear winner right now, is React. Components all the way down makes your application much easier to reason about. The learning curve is very flat. The important APIs would fit on one page. JSX is awesome. You get all the power of JavaScript and its tooling when writing your markup. It is the natural match for Flux and Redux (more on that later). The React community is amazing, and produced many best of breed tools such as Redux (also more on that later). Writing high ... Read More Source March 26th, 2016 No Comments # Pay discrimination in tech This sounds bad: Women who write the software that runs on mainframe computers earn on average 72 cents per dollar earned by their male counterparts, according to research conducted by Glassdoor Inc., the online job information firm. That pay gap exists even after controlling for age, education, experience, job title, employer and location. This suggests that the gap widens with age: However, not all tech jobs pay men and women so differently. Among mobile developers, there is just a 2.9% gap between ... Read More Source March 26th, 2016 No Comments # Left-Pad as a service Everyone is laughing over this. An 11 line function as its own NPM module? I am unsure how I feel. The trend has been to smaller and smaller pieces of functionality. I don’t find this outrageous. Still, the parody is funny: c4n4rd is game: This is really exciting!!! I was a bit disappointed that the right-pad will be out only in 2017. I am looking forward to that release because there is a high demand for it now. What kind of ... Read More Source March 26th, 2016 No Comments # null Testing WordPress to see if a title of “Null” works. Source March 18th, 2016 No Comments # The end of the test-first mania It worries me that the tech industry is driven by such intense manias, which have cost me several jobs. For a long time you were not allowed to question Object Oriented Programming. I still go to job interviews and I’m asked the basics of Object Oriented Programming, and it is clear I have to say nice things about Object Oriented Programming or I won’t get the job. Thankfully none of these people ever check Wikipedia, where I am cited as ... Read More Source March 17th, 2016 No Comments # Onyx tested by Jespen is genius tested by genius Good lord, this makes me happy. I think Michael Drogalis is a very smart guy and everyone knows that Kyle Kingsbury is a very, very smart guy. Kingsbury’s work on Jespen is the finest work that anyone has ever done on the problems of distributed data. Onyx is exciting as an a Clojure answer to Apache Storm. Sad to say, Storm is written in Scala. The idea of testing Onxy with Jespen is one of the finest ideas ... Read More Source March 14th, 2016 No Comments # Unix processes have gotten bigger and slower over the years Interesting: The obvious question thus is: Why state machines? Why not processes or threads? And the obvious answer is: Performance. When UNIX was still young, scheduling was supposed to be done by the OS on per-process basis. When implementing a network server, for example, you were supposed to fork a new instance of the process for each TCP connection and rely on the OS scheduler to switch between the processes. I guess it made sense from performance point of view back then. All ... Read More Source March 10th, 2016 No Comments # Jakub Holý on copying :pre and :post conditions Obviously I have read this article before (I offered response and was quoted before) but I’ve gone back to read it again and this jumped out at me: Do you repeat the same checks again and again? Then you could either copy them using with-meta (they end-up in metadata anyway) or reuse the explicitly: (defn with-valid-car [f] (fn [car] {:pre [:make :model :year]} (f car))) (def count-price (with-valid-car (fn [car] (do-something car)))) ;; or make & use a macro to make it nicer That ... Read More Source February 15th, 2016 No Comments # Sick politics is the driving force of useless ceremony This is part 7 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the driving force of useless ceremony 8.) Functional programming is not the same as static data-type checking Interlude 9.) Inheritance has nothing to do with Objects 10.) Is there a syntax for immutability? 11.) Immutability enables concurrency 12.) Quincy’s ... Read More Source February 13th, 2016 No Comments # Immutability changes everything (Acknowledgements: I offer a huge “Thank you” to Natalie Sidner for the tremendous editing she did on the rough draft of this post. To the extent that this article is readable, it is thanks to her. Any mistakes are entirely my fault, and I probably added them after she was done editing. If you need to hire a good editor, contact Natalie Sidner at “nataliesidner at gmail dot com”. Also, I thank Blanche Krubner for reviewing this work. As Mrs Krubner studied ... Read More Source February 13th, 2016 No Comments # Functional programming is not the same as static data-type checking (Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!) This is part 8 of a ... Read More Source February 10th, 2016 No Comments # Interlude I am mostly done writing the second half of this series, however, it needs to be edited, some code needs to be written, all of the code needs to be checked, and I need to work with Leah McCloskey to develop further cartoons for illustrating the points made in this series. Also, I have a full time job. So it will likely be 2 or 3 months before I can publish the second half of this series. The series is ... Read More Source February 10th, 2016 No Comments # Get rid of all Dependency Injection This is part 6 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the driving force of useless ceremony 8.) Functional programming is not the same as static data-type checking Interlude 9.) Inheritance has nothing to do with objects 10.) Is there a syntax for immutability? 11.) Immutability enables concurrency 12.) Quincy’s ... Read More Source February 10th, 2016 No Comments # Mutable iterators are the work of the Devil This is part 5 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the driving force of useless ceremony 8.) Functional programming is not the same as static data-type checking Interlude 9.) Inheritance has nothing to do with objects 10.) Is there a syntax for immutability? 11.) Immutability enables concurrency 12.) Quincy’s ... Read More Source February 10th, 2016 No Comments # Clojure has mutable state Warning: This post is an intro to mutable state in Clojure. If you already know Clojure, you can skip this. If you have no interest in learning Clojure, you can skip this. This is part 3 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate all articles about design patterns 3.) Clojure has mutable state 4.) Immutability changes everything 5.) Mutable iterators are the work of the Devil 6.) Get rid of all Dependency Injection 7.) Sick politics is the ... Read More Source February 10th, 2016 No Comments # Why I hate all articles about design patterns Warning: This article merely describes what I hope to achieve with this series. There is no software code in this article. Those who wish to stay focused on those articles that are focused on code should skip this article. Since I am here describing my personal motivations for writing, this article is necessarily the most self-indulgent of the articles in this series. This is part 2 of a 12 part series: 1.) Quincy’s Restaurant, a parable about concurrency 2.) Why I hate ... Read More Source February 10th, 2016 No Comments # A parable about concurrency (demonstrated with comical cartoons) (Note: Leah McCloskey is a brilliant illustrator who brings warmth and humor to every graphic she creates. She is also head of design at Haystack.im. Save for those rights which she specifically granted to me, she reserves all rights on these images, so if you wish to re-use them, then you must contact her directly at leah @ dendritecorp.com. I am grateful that she found the time to work on this project. View her portfolio!) This is part 1 of a ... Read More Source February 9th, 2016 No Comments # Why does Erlang allow so many processes to crash and how does promote reliability? Why does Erlang encourage crashes? Back-burning and controlled burns are a real world example of fighting fire with fire. In Saguenay–Lac-Saint-Jean, the region I come from, blueberry fields are routinely burnt down in a controlled manner to help encourage and renew their growth. To prevent forest fires, it is fairly frequent to see unhealthy parts of a forest cleaned up with fire, so that it can be done under proper supervision and control. The main objective there is to remove ... Read More Source February 6th, 2016 No Comments # Hindley-Milner and the lambda calculus A nice attempt to explain the crazy syntax of the math that formalizes the idea that the type of an expression can be deduced from the expression itself: Okay, so we want to talk about expressions. Arbitrary expressions. In an arbitrary language. And we want to talk about inferring types of these expressions. And we want to figure out rules for how we can infer types. And then we’re going to want to make an algorithm that uses these rules to ... Read More Source February 3rd, 2016 No Comments # Beware of Ruby libraries that generate way too many objects This is several years old and Ruby garbage collection has gotten better, but still, the point about certain libraries being excessive remains valid. Be aware that you are allocating objects, for instance something as simple as 100.times{ ‘foo’ } allocates 100 string objects (strings are mutable and therefore each version requires its own memory allocation). Make sure to evaluate the libraries you use, for instance switching a Sinatra XML rendering action from Builder to Nokogiri XML Builder saved us about ... Read More Source February 3rd, 2016 No Comments # Pat Shaughnessy dissects how much work Ruby has to do to give you a string Easy is difficult, and this is a great look at how much work Ruby has to do so that you, the software developer, can change your mind about what kind of string you want: The standard and most common way for Ruby to save string data is in the “heap.” The heap is a core concept of the C language: it’s a large pool of memory that C programmers can allocate from and use via a call to the malloc ... Read More Source February 3rd, 2016 No Comments # Software development is a complex system of multiple poorly understood feedback loops and interactions I wouldn’t use exactly the same words that this article uses, but I agree with the gist of this part, especially where small, fast-moving startups are involved : Finely grained management of software developers is compelling to a business. Any organization craves control. We want to know what we are getting in return for those expensive developer salaries. We want to be able to accurately estimate the time taken to deliver a system in order to do an effective cost-benefit analysis ... Read More Source February 2nd, 2016 No Comments # Leonardo Borges’s Imminent for futures and promises Very interesting: For the impatient, I’ve included a couple of examples below. I’ve chosen to translate the examples presented by Ben Christensen – of RxJava – in this gist. Albeit them being in Java, they highlight perfectly the problem with blocking futures. Here’s their Clojure equivalent: ;; ;; Example 1, 2 & 3 are handled by the approach below ;; Original examples: https://gist.github.com/benjchristensen/4671081#file-futuresb-java-L13 ;; (defn example-1 [] (let [f1 (remote-service-a) f2 ... Read More Source February 2nd, 2016 No Comments # Why the tech community rejected XML Interesting: Java was a very limited language, and it was extremely verbose. There were type declarations and coersions everywhere. Almost all looping was managed with for loops (there were no higher order functions, and recursion wasn’t tail cail optimized… also there was no loop-recur macro…). So XML wasn’t just there to be data, it was also code. Everyone wrote little mini-languages into XML, because some things were so damn painful to express in Java that it was easier to just implement ... Read More Source February 2nd, 2016 No Comments # Clojure community problems Interesting: What an amazing language. Relatively frequent, consistently stable releases. A pleasure to use. A friendly, smart community. I feel very lucky to be a Clojure user! Thank you for all of your hard work. The Clojure contrib process frustrates me more than any technical or community aspect of the language. Clojure gets a lot right, but as has been repeatedly discussed the pace of evolution and the maintainership’s dim view of 3rd party non-bugfix work flatly leads to worthy but minor ... Read More Source January 31st, 2016 No Comments # We want loose-coupling and high cohesion This seems like a great rule of thumb for microservices: “We want to avoid dumb, anemic services that are little more than CRUD wrappers” But it doesn’t cover the old territory which, if you were using Ruby or PHP, you would cover with a cron script. I suppose all the cron scripts must become functions that live inside the “service” which deals with a given part of the datastore. But that is not how my friends talk about “microservices”. Page 58, ... Read More Source January 31st, 2016 No Comments # A message hospital (or dead letter queue), where messages got sent if they failed I like this book very much. I also like the idea of a “message hospital”. Page 57, Building Microservices, Sam Newman: Time for a cautionary tale. Back in 2006, I was working on building a pricing system for a bank. We would look at market events, and work out which items in a portfolio needed to be repriced. Once we determined the list of things to work through, we put these all onto a message queue. We were making use of a ... Read More Source January 31st, 2016 No Comments # Jon Williams, Fractional CTO: Include business teams in the Agile process Jon Williams offers a smart idea about getting the business people and the tech team to share as much information as possible, and even have the business people join the scum sessions. The best implementation of the Agile process that I ever participated in was at ShermansTravel.com, back in 2011/2012. We had fairly good communication between the tech team and the folks running the actual parts of the business: editorial, advertising, marketing and more. A lot of the success of ... Read More Source January 31st, 2016 No Comments # Joe Armstrong figured out the right way to do everything, and nobody cared I am puzzled why good ideas so rarely win out in the tech world. I do know the old saying “Most industries have a top player with the best marketing and the second best technology, with a second place player that has the best technology and mediocre marketing.” Back in the 1990s it was common to apply that slogan to Microsoft and Apple (when Apple had mediocre marketing! Such long forgotten days!). Still, the lack of innovation in this industry ... Read More Source January 30th, 2016 No Comments # When technology goes on strike Interesting: Amazing but true When I’d finished this article, I wanted to spell check the content. emacs-ispell mode decided to go on strike. It could not find aspell, the program that I use for spelling checking. My emacs spell checker has worked faithfully on this machine for several years. And just when I complain that I spend half my life fixing things that shouldn’t be broken the emacs spell checker decides to break. I don’t believe in malicious Gods, nor that the laws ... Read More Source January 24th, 2016 No Comments # Groping in the dark as a method to discover module boundaries in microservices I’m not sure that we will ever have a better way (but then, we can never pretend to be like other types of engineers, can we?): Having clients talk to the application service instead of an ORM or the data backends directly lets you forget that there is a MASSIVE PROBLEM with ORM frameworks (for anything more interesting than, say, a blog website framework), because you will still have to write a translation between your application representation and the database representation. ... Read More Source January 24th, 2016 No Comments # Why I love Erlang And why I dislike Scala. I would bet that Scala was designed in the opposite fashion: Everything was very problem oriented and we did not have as goals that Erlang should be a functional language or that we should implement the actor model. We knew nothing of the actor model until later when we heard that Erlang implements it. :-) … Again our goal was to solve the problem, not design a language with a predefined set of primitives. Source January 22nd, 2016 No Comments # HTML is the failed GUI for TCP/IP I posted this to Hacker News. I was surprised that someone did not immediately understand what I meant when I referred to HTML as the GUI of TCP/IP. Someone had surveyed 80 frontend designers and found they lacked basic knowledge of HTML and CSS. I responded: The contrarian argument is that this signifies an important truth, that HTML never worked the way it was suppose to. In the same way that we might argue that a misunderstood product is the fault ... Read More Source January 20th, 2016 No Comments # An Amazon DSL in Clojure Good lord. This is an impressive list of supported Amazon APIs: Api Gateway Autoscaling CloudFormation CloudFront CloudSearch CloudSearchV2 CloudWatch CodeCommit CodeDeploy CodePipeline Config DataPipeline DeviceFarm DirectConnect Directory DynamoDBV2 EC2 EC2 Container Registry ECS ElastiCache ElasticBeanstalk ElasticFileSystem ElasticLoadBalancing ElasticMapReduce ElasticTranscoder Glacier IdentityManagement Kinesis KinesisFirehose KMS Logs Lambda MachineLearning OpsWorks RDS Redshift Route53 S3 SimpleDB SimpleEmail SimpleWorkflow SNS SQS StorageGateway Source January 20th, 2016 No Comments # A 3D DSL in Clojure Very interesting, and a nice example of how much Clojure allows the creation of new languages: My first step in the HOLO design process was the creation of a sufficiently flexible GP playground for my later experiments and to evolve path-finding agents to create a (typographic) form as an initial design idea. Related to this, though a year prior, I created the Mophogen DSL, partially done as a component of my commission for the Barbican / Google DevArt exhibition. Morphogen ... Read More Source January 16th, 2016 No Comments # Microservices first Stefan Tilkov adds clarity to an argument that I’ve been making since 2013: If you are actually able to build a well-structured monolith, you probably don’t need microservices in the first place. Which is OK! I definitely agree with Martin: You shouldn’t introduce the complexity of additional distribution into your system if you don’t have a very good reason for doing so. (So what would be a good reason? There are many, but to me the most important one is to ... Read More Source January 16th, 2016 No Comments # I give good tech advice to startups, and different good advice to Enterprises I had a job interview recently that went well. But then, the CTO read something that I had written a year and a half ago. Then the CTO said to me “We can’t hire you. We are a tiny startup. We are facing some serious deadlines. We need to push a product out the door.” I said, “I want to help you push a product out the door. I can work hard and help you guys move fast.” He said, ... Read More Source January 16th, 2016 No Comments # The cost of using patterns is that I have to give up the illusion that I am infinitely creative This is true of all creative fields: The cost of using patterns is that I have to give up the illusion that I am infinitely creative. I don’t invent a new programming language with optimizing compiler and novel operating system and complete programming environment for every line of code. To make best use of my three billion seconds, I should ignore most of my options and focus on a few degrees of freedom that really matter right now. Source January 16th, 2016 No Comments # Paralyzed by too much thinking I like this: When I started programming again, I vowed not to type a single character unless I knew what pattern I was applying as I did so. The result was incredibly frustrating. I want a class called “Stack”, but why “Stack” and not something else. Then I would go and write the patterns for naming classes and then I could type “Stack”. Then I would want to make its first method public, but why public? At first, it was like ... Read More Source January 14th, 2016 No Comments # The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise Interesting What is the black king in chess? This is a strange question, and the most satisfactory way to deal with it seems to be to sidestep it slightly. What more can one do than point to a chessboard and explain the rules of the game, perhaps paying particular attention to the black king as one does so? What matters about the black king is not its existence, or its intrinsic nature, but the role that it plays in the game. The ... Read More Source January 13th, 2016 No Comments # Invoxia will allow Alexa to figure out who is speaking I will be pleased if this becomes more than vaporware: Invoxia, one of the recipients of an Alexa Fund investment from Amazon last September, announced this week at CES that it’s the first third-party hardware maker to incorporate all the power of Alexa into a product other than the Echo. The company’s Triby, as it’s called, is a colorful, magnet-backed Bluetooth speaker resembling an old-school radio. It’s designed to let family members, including young children, make internet-based phone calls with ... Read More Source January 7th, 2016 No Comments # CQRS Read Models Interesting: We’ve been using this mechanism with great success during development. It let us freely modify the schema by only changing the Java classes and never worrying about table definitions. Thanks to combination with CQRS, we could even maintain long-running demo or pilot customer instances. Data has always been safe in the event store. We could develop the read model schema incrementally and have the changes automatically deployed to a running instance, without data loss or manually writing SQL migration scripts. Obviously ... Read More Source January 6th, 2016 No Comments # No one realizes that there is an app store for the Amazon Echo? More from Jose Jaquinta: I ran into some people during various New Year’s parties who had Echos, but had no clue there were skills for them. I laboriously explained how to launch the companion app, find the skills tab, and enable the skill. It would a lot simpler if a user could enable/disable skills without having to go through the companion app. For example: Alexa, enable Demotivate Me I realize this is a request for The Echo, and not the Alexa Skills Kit. However, ... Read More Source January 6th, 2016 No Comments # The Amazon Alexa certification process has gone out of control This is sadly accurate: The certification process has gone out of control. This is one example of the… I’m struggling to keep to professional language… questionable judgement… show by the certification team. I’ve had two skills come back with a similar objection. I want to ask them if they have even used Alexa regularly? It gets REALLY tedious to have it endlessly repeat things. It’s like, “Yes, I know what I can do next. Please let me get on with ... Read More Source January 1st, 2016 No Comments # Mark Lee Smith fights functional programming in 2011 I find it interesting to go back and look at the sites where was learning about Functional Programming back in 2011. It’s interesting to read those who were on the losing side of history, fighting against Functional Programming: If you’re writing 2000-line classes and 500-line methods then you’re in no way qualified to be speaking about object-oriented programming, let alone comparing it with the latest FAD ideology. Christian Toivola gave what has become the standard answer: @Mark Lee Smith: With the ... Read More Source January 1st, 2016 No Comments # When did the Internet start? Interesting: TCP version 1 was designed in 1973. This was documented through RFC 675. TCP version 2 was documented in March 1977. In August 1977, Jon Postel realized they were going the wrong direction with the protocol. “We are screwing up in our design of internet protocols by violating the principle of layering. Specifically we are trying to use TCP to do two things: serve as a host level end to end protocol, and to serve as an internet packaging and ... Read More Source December 31st, 2015 No Comments # We are contemplating allowing sender and receiver to specify different byte sizes Wow. Can you imagine how that would have turned out? On the bright side, Unicode would not be necessary, since a company in China could specify a byte size large enough to represent any Chinese characters, and likewise for Iran, Korea, Hindi, etc. . Source December 31st, 2015 No Comments # NLP versus Conversational AI Obviously, I have written a lot about this subject during the year. See my post “What happens when the Board Of Directors begins to panic“. But this is getting big: To date, commercial NLP research has been focused on understanding the many ways people say commands like, “Set my alarm for 7 am tomorrow” or “Find a good pizza place nearby.” To unlock the power of conversational computing, new technology is required that extracts semantics across multi-turn natural language ... Read More Source December 31st, 2015 No Comments # Allowing an object to exist in an invalid state is an Anti-Pattern This post uses the language of Object Oriented Programming, but its point would also be true to all forms of programming. In Clojure, if you have HashMaps that you then convert to JSON and save to MongoDB, then why would you ever allow the HashMap to exist in an invalid state? Real immutability might imply that the HashMap is created with everything that it will eventually need. This is an interesting idea. Why do we validate data? Typically, to make ... Read More Source December 31st, 2015 No Comments # Websockets are an all or nothing proposition Interesting: I’m also a heavy websocket user, and agree with most points. I have previously used websockets on top of a traditional web app, and have been disappointed with the results. My opinion now is that websockets are an all-or-nothing proposition. And I have gone all-in. My latest project has: - websockets-only api (except a some image uploads and oauth/login) - https-only - single page app, central store with observable data, using a single connection servicing the whole application - using SignalR (supports fallbacks for IE9 ... Read More Source December 31st, 2015 No Comments # Any growing Rails app runs into the problem of fat models This is a good comment: A lot of things in Rails are are anti-patterns in large code bases, but pragmatic in small ones. ActiveRecord itself is a prime example: when you start an app, putting your business logic directly in ActiveRecord objects works pretty well in most cases, but later on as the models proliferate and grow, you realize that some of them contain business logic which is far too complex to warrant being conflated with persistence concerns. The result ... Read More Source December 18th, 2015 No Comments # How to find a major security breach at Facebook Facebook offers a bounty for bugs, but Facebook did not play fair on this one. Instead, Facebook immediately reached out to the hacker’s employer and accused the hacker of unethical behavior. The Facebook CSO later offered the cheap excuse that the hacker might have been working in an official capacity for the employer. The more likely story is that Facebook was embarrassed and wanted this story to be remain hidden. Interesting: With the newly obtained AWS key, I browsed several ... Read More Source December 17th, 2015 No Comments # How to test the Salesvoice skill for the Amazon Echo 1.) if you have access to a Salesforce account, use that. If there is an admin at your company who manages the Salesforce account, talk to them. Otherwise, you can sign up for a free 30 day trial on Salesforce. Be sure to go for the Enterprise Edition. 2.) You must enable the SalesVoice Connected App 3.) Relax IP restrictions, or whitelist the IP from which your Amazon Echo will be in use 4.) You will need to enter 4 pieces of ... Read More Source December 15th, 2015 No Comments # The importance of the Module class to NodeJs Interesting: The Module type found in module.js has two main roles inside of Node.js. First, it provides a foundation for all Node.js modules to build off of. Each file is given a new instance of this base module on load, which persists even after the file has run. This is why we are able attach properties to module.exports and return them later as needed. The module’s second big job is to handle Node’s module loading mechanism. The stand-alone require function that we ... Read More Source December 15th, 2015 No Comments # Require and Export in NodeJs Interesting: Finally, the last thing to consider is what happens when you directly export a function: var powerLevel = function(level) { return level > 9000 ? “it’s over 9000!!!” : level; }; module.exports = powerLevel; When you require the above file, the returned value is the actual function. This means that you can do: require(‘./powerlevel’)(9050); Which is really just a condensed version of: var powerLevel = require(‘./powerlevel’) powerLevel(9050); Source December 14th, 2015 No Comments # How to build conversations via the Amazon Echo If you read my post about the startup I was this year, then you know the toughest challenge we faced was the building the finite state machine that could handle conversations. So I read this about the Amazon Echo with great interest: My task would have ended here if creating an event would only require a date and time. But to be useful, I would like to include a duration, a topic, perhaps even a location. To be even more ... Read More Source December 13th, 2015 No Comments # Using a glottal stop to force the Amazon Echo to correctly pronounce “tw” This an impressive bit of phoneme hacking, by Joseph Jaquinta: It’s also occasionally off on some words. A particular problem for me was Alexa’s inability to correctly pronounce “tweets”. It can say “tweet” just fine, but “tweets” comes out as “wheets”. Since one of upcoming skills, Tweet Poll, centers around producing election statistics based on tweets, it uses the word “tweets” a lot. So I was pretty excited when I saw that the new update included features for giving more fine control ... Read More Source December 13th, 2015 No Comments # Dialogue designers replace graphic designers when creating voice interfaces Over the last 30 years we’ve gotten used to the Graphical User Interfaces in software that appears on a computer screen. And most of us (who work in the tech industry) have had the experience of working with a person who, depending on their skills, will be given a title such as “graphic designer” or “user experience designer” or sometimes “product designer”. Working with the Amazon Echo, myself and my partner have come to realize that we still need a ... Read More Source December 13th, 2015 No Comments # How can I connect the Amazon Echo to wifi in an Enterprise with secure wifi? I posted this question on the TalkEcho forum. Any suggestions are welcome. One big problem we face (developing Enterprise apps for the Amazon Echo) is when we do a demo at a large company. Then we are typically facing the WPA2 Enterprise version of WiFi, which the Echo does not seem to support. When I look here: https://www.amazon.com/gp/help/customer/display.html?nodeId=201549640 I see it says: “Amazon Echo connects to dual-band Wi-Fi (2.4Ghz / 5Ghz) networks that use the 802.11 a/b/g/n standard. It does ... Read More Source December 13th, 2015 No Comments # The old debates about Clojure This was written in 2009 and I linked to it in 2011, and I just stumbled upon it again. It is interesting to remember the old fights about Clojure, the struggle for respectability that Clojure faced in those old days. For the most part, that particular fight is no longer facing Clojure. Source December 12th, 2015 No Comments # The Amazon Echo Blogosphere I will be updating this blog post as time goes by. I hope to build a complete catalog of blogs that are primarily focused on the Amazon Echo. Here is what I found after 30 minutes of searching: LoveMyEcho is one of the few blogs that seems devoted to the Amazon Echo Joseph Jaquinta has a blog on LinkedIn devoted to the Amazon Echo Engadget has a category for Amazon that sometimes covers the Echo Gizmodo has a category specifically for the ... Read More Source December 12th, 2015 No Comments # Custom commands for the Amazon Echo Interesting: Telling Amazon’s Alexa to play some music or add an item to your shopping list is pretty cool, but there comes a point where you start to feel the limits of the Echo’s pre-programmed voice commands. It doesn’t have to be that way anymore: Amazon and the automation wizards at IFTTT just announced a new feature that allows you to create custom voice phrases to make Alexa do whatever you want. Unfortunately, there’s a catch: every command has to ... Read More Source December 12th, 2015 No Comments # Ellery Coffman demonstrates home automation via Alexa Ellery Coffman gets Alexa to start a movie on the TV, then stop it, and then play an album on the record player, and then stop it. This is impressive in all ways, except I was surprised at how slow it was. I’ve got an Alexa app running on a Rackspace server, which pulls data from Salesforce, and the response is much faster, despite the fact that there are 2 API calls involved, and despite the fact that Salesforce ... Read More Source December 4th, 2015 No Comments # The advantages of a Natural Language Processing interface “admin” of Daily Hacker News is critical of my ideas regarding a Natural Language Processing interface for software. Some of their points conflate Natural Language Processing with the text version of Natural Language Processing, and right now I’m mostly working on Natural Language Processing via voice interfaces, so I’m going to reply with voice examples, so as to remind everyone that Natural Language Processing is a broad topic. “admin” starts with this point: Let’s go back to Lawrence’s example. Posit, ... Read More Source December 2nd, 2015 No Comments # People want to be able to talk to their computer When GUI interfaces were new in the 80s we were told they were better than command line interfaces because they offered “discoverability”. But it turned out they didn’t offer nearly enough discoverability, so in the 90s “Wizards” became big things, especially in the Microsoft world. A Wizard provided the context that a GUI by itself did not. A Wizard would tell a user “Do this, now do this, now do this, and then do this.” But Wizards were visually complex. ... Read More Source December 2nd, 2015 No Comments # Clojure is easier than Scala Interesting: After about a week of experimentation, we hit a sticking point. While we had a large number of “normal” CRUD operations, all of our queries on behalf of the end user, were generated dynamically. Most of the Scala database ecosystem is geared towards extending Scala’s type system to include database query type checking. Overall, our enthusiasm for Scala didn’t survive the first attempt to port our query language system. Clojure for the win It was at this juncture where we re-evaluated Clojure, ... Read More Source November 28th, 2015 No Comments # Christine Dodrill on the Universal Design She has her own terminology: State – What is true now? What was true? What happened in the past? What is the persistent view of the world? Events – What is being changed? How will it be routed? Policy – Can a given event be promoted into a series of actions? Actions – What is the outcome of the policy? Mechanism – How should an event be taken in and an action put out? Policy seems a bit confusing, but sometimes there is a reward if ... Read More Source November 28th, 2015 No Comments # Separate AWS accounts for development, staging, and production Interesting: Instead of using regions or tags to separate different staging and prod instances, we switched over totally separate AWS accounts. We need to ensure that our provisioning scripts wouldn’t affect our currently running services, and using fresh accounts meant that we had a blank slate to start with. The ops account serves as the jump point and centralized login. Everyone in the organization can have a IAM account for it. The other environments have a set of IAM roles to switch ... Read More Source November 28th, 2015 No Comments # I am one voice of many, who question Object Oriented Programming I am just one of many voices questioning Object Oriented Programming and promoting the Functional paradigm as an alternative. I pleased to think my big essay, which surveyed quite a bit of what had been written before, is still able to occasionally encourage people to re-think their support for Object Oriented Programming. So I see Matt’s Code Cave write: Why functional programming and why Clojure? I’ve been interested in the idea behind functional programming for a while now – being used ... Read More Source November 21st, 2015 No Comments # The States and Fates of Javascript Interesting resource regarding Javascript Promises: States Promises have three possible mutually exclusive states: fulfilled, rejected, and pending. A promise is fulfilled if promise.then(f) will call f “as soon as possible.” A promise is rejected if promise.then(undefined, r) will call r “as soon as possible.” A promise is pending if it is neither fulfilled nor rejected. We say that a promise is settled if it is not pending, i.e. if it is either fulfilled or rejected. Being settled is not a state, just a linguistic convenience. Source November 21st, 2015 No Comments # Why modern Javascript development sucks The package management tools are broken: Source November 21st, 2015 No Comments # How do you get latitude with Javascript Here is something I did not know about:$cordovaGeolocation.getCurrentPosition always returning “Position retrieval timed out.” in ios

That’s an error, but it made me aware that you can try to get location via Javascript. I think that is amazing. Why even bother with a native app? And I say that as someone who just spent 6 months at a starup that was committed to doing a native app, in part for native features, including the ability to get latitude and longitude.

Source

November 20th, 2015

# Trailblazer has good ideas hidden by Object Oriented Programming

Trailblazer for Ruby On Rails has some very good ideas in it though they are somewhat buried by the Object Oriented programming garbage:

To give an example of how operation works, let’s say you click on a delete button. What happens when you do that is you go to the control that knows a little bit of this action because it does authentication—if it’s valid it does one thing and if it’s not it does another thing. Instead of delegating ...

November 19th, 2015

# Nested relations are the death of Object Oriented Programming

Last winter made fun of a Python team for wasting time trying to figure out how to handle deeply nested relations. They were slowly groping their way toward a graph database solution, which seems to be the route that most end up taking, given enough years and enough pain. This seems to be an area where Object Oriented Programming clearly fails.

And yet, many smart folk continue to try. Ruby’s Reform:

Syncing Back

After validation, you have two choices: either ...

November 19th, 2015

# Rails derails

A super interesting debate about the move away from Rails.

We seem to be on the same side when it comes to quality metrics.

Source

October 10th, 2015

# A Java Hashmap is not a Clojure collection

I did not know this till just now. I have a co-worker who wrote a library in Java. From my Clojure code, I call this Java function:

public HashMap<String,HashMap<String,Integer>> init(String debrief,String companyName, ArrayList<String> contacts, ArrayList<String> accounts, ArrayList<String> requiredFields) { try { return transformer.transform(debrief, companyName, tecClassifier, rollioClassifier, caseClassifier, caselessClassifier, customClassifier, pipeline, parser, props, firstNames, lastNames, accounts, contacts, requiredFields); ...

October 10th, 2015

# The TCP checksum is weak, and the Ethernet checksum will accept corrupt TCP that passed the TCP checksum

Interesting:

At Twitter, a team had a unusual failure where corrupt data ended up in memcache. The root cause appears to have been a switch that was corrupting packets. Most packets were being dropped and the throughput was much lower than normal, but some were still making it through. The hypothesis is that occasionally the corrupt packets had valid TCP and Ethernet checksums. One “lucky” packet stored corrupt data in memcache. Even after the switch was replaced, the errors continued until ...

October 10th, 2015

# When is computer programming easy?

Interesting:

While it is often thought that “real” programmers like real programming environments (my husband, for instance, is perfectly happy in MF Assembler, which is pretty grim), most programmers seem to become pretty happy moving up to graphical user interfaces and visual environments where they choose from a list of “correct” choices. I don’t think it is only users that can be more productive when they are better supported.

I believe there is a direct relationship between the ease of use of ...

October 6th, 2015

# How easy is it to write immutable Javascript?

If you reinvent the language as a new language, then you can have immutable Javascript. On the frontend you have no choice, but on the backend? Why not use a language that offers what you need upfront, rather than forcing you to work for it? Interesting:

The main issue I’ve had using immutablejs with Redux is debugging. Whereas previously I could simply mouse-over a data structure when I hit a breakpoint (or crash), I now have to do a REPL ...

October 4th, 2015

# Erlang – OTP – Cowboy are the cutting edge

At the current time, the only thing out there, in the tech industry, that might cause me to move a way from Clojure, is something build around the Erlang VM . In those situations where performance and massive concurrency are needed, something like Erlang/Cowboy or Elixir/Phoenix need to be looked at.

I load tested (using wrk) nginx serving a two line, static HTML file, against a basic Cowboy service that parsed parameters, did an ETS lookup, and rendered several hundred ...

October 4th, 2015

# Clojurescript as the frontend to Erlang

Radically different, yet becoming more common:

We use vanilla SmartOS, so that there is no dependency on FiFo for your running VM’s. You could just switch FiFo off and all your VM’s would continue to just work. It also comes with a number of great advantages:

In our opinion, ZFS is simply the only file system that should ever be used – period.

Compression, ARC and ZIL work incredibly well, especially for DalmatinerDB which achieves amazing throughput partially thanks to being purpose ...

October 4th, 2015

# Hacking environmental protection features

Interesting:

Well, it turns out that all this environmental friendliness is starting to trip over itself, because most devices now start up in standby mode. So you can’t just power them on to power them on, you have to power them on and hit a button on the remote. Some devices, by pure dumb luck I assume, will accept the switch already being pressed in when they start. If that’s the case, you can hack the behaviour you want if you ...

September 26th, 2015

# The Recursive Function Pattern Matching Pattern

Sean Johnson has a great video up. Common in Erlang, useful in Clojure, 3 functions, one to start, one to do the work, one to end, all defined by arity, using Pattern Matching.

Also called the “Start, Work, End” pattern.

This is often done in Clojure with zipmap, but this does not make things as clear as the Erlang version:

This is much more clear:

Source

September 26th, 2015

# Best practice pattern matching in Clojure

If your function starts with a conditional, replace that with pattern matching:

Source

September 16th, 2015

# Clean up your act with lein-checkall

I am a bit late but I’ve only recently become consistent about using a linter to check my Clojure style. And I’ve only today discovered the Leiningen plugin lein-checkall, which combines lein check && lein kibit && lein eastwood && lein bikeshed.

Venanti has written about the importance of these:

Eastwood

Eastwood is a Clojure linter, invoked with lein eastwood. As a general request, please use a linter. Some of my favorite moments in the last year have come from people trying to ...

September 13th, 2015

# Calculating and Visualizing Voronoi Diagrams using the Quad-Edge Structure – Alan Shaw

Allan Shaw talks about a data structure called a “quad edge”. There were a lot of ideas here that were new to me. The data structure keeps rotation as one value, and the orientation of the edge as one value, but anything like coordinates are kept in a field called “data”. This separates the topology from the geometry. There is also a “next” function to find the next edge. This, apparently, gives great advantage when calculating a whole network of ...

September 12th, 2015

# Microservies mean freedom of future action

Interesting:

In short, maximize future freedom of action. This heuristic also answers the when and why questions for microservices.

To prove it, let’s start with Sprott & Wilkes on SOA. Wait, Service Oriented Architecture? SOA is microservices loving parent. The philosophy trickles down:

When the service is abstracted from the implementation it is possible to consider various alternative options for delivery and collaboration models. […] It is entirely realistic to assume that certain services will be acquired from external sources because it ...

September 11th, 2015

# Why are microservices happening now?

The term “microservices” only goes back to 2013. I would have trouble saying why I love the style so much, except that I hated dealing with Ruby On Rails and the PHP framework known as Symfony. I was willing to deal with a lot to get away from the pain of those systems. The growth of great package managers also seemed to weaken the need for monoliths.

But Martin Fowler says the most important change was the move to Continuous ...

September 11th, 2015

# The difference between Clojure and Common Lisp

Fascinating bit by Giles Bowkett. This uses recursion and is how you would do it in Common Lisp:

(defn build [list-1 list-2] (if (nil? list-1) () (concat (map (fn [bubble] (list (first list-1) bubble)) list-2) ...

September 11th, 2015

# Microservices are different

It does feel like the world is moving closer to distributed IPC. We are not there, but the push toward microservices seems a step in that direction. The warnings of the past feel less relevant now.

When I wrote Patterns of Enterprise Application Architecture, I coined what I called the First Law of Distributed Object Design: “don’t distribute your objects”. In recent months there’s been a lot of interest in microservices, which has led a few people to ask whether microservices ...

September 11th, 2015

# Zach Tellman — the need for backpressure in queues, and the limits

This is very good. A lot of this interesting, but perhaps the biggest surprise is when he adds 16 consumers of tasks, but also 16 producers of tasks, the crisis point comes suddenly, compared to when there was only 1 consumer and 1 producer. With multiple consumers the consumption is averaged out, so each machine works fine right up to the moment of crisis.

His main theme is “Unbounded queues are fundamentally broken because it puts the stability of our ...

September 11th, 2015

# Dynamic scope in Clojure

The interesting thing here is that the dynamic vars are wrapped in functions, which makes them a bit safer than dynamic scope would imply.

Each of these sub-clauses is very similar to the parent structures, and can be arbitrarily nested. As such, it is most easily constructed with a recursive function. However, Clojure’s let bindings are lexical, and don’t extend into recursive calls. We could make the memoized gensym call a parameter, but since we also need a separate generator ...

September 11th, 2015

# This instance doesn’t do much, but it does have a nice property: it’s only equal to itself

This is a very clever hack:

Graphviz assumes explicit identities for nodes, so what about when we want to represent a tree? We can’t simply use the identity of the node, because the same value at different positions within the tree must be treated as separate nodes. One possibility is to represent each node as a tuple of its value and its position in the tree, but this forces us to consider how to represent positions, which is neither obvious nor ...

September 11th, 2015

# The 8 fallacies of distributed computing

This is great:

The IT group usually has different administrators, assigned according to expertise–databases, web servers, networks, Linux, Windows, Main Frame and the like. This is the easy situation. The problem is occurs when your company collaborates with external entities (for example, connecting with a business partner), or if your application is deployed for Internet consumption and hosted by some hosting service and the application consumes external services (think Mashups). In these situations, the other administrators are not even under ...

September 10th, 2015

# Is Gradle the best build tool for the JVM?

Gradle continues to innovate with new features.

Is Buildr dead? Buildr is now part of Apache and yet its last update was over a year ago. I tried to use Buildr this year but I ran into errors and when I searched on Google I found all of the advice was many years old. No one responded to my question on StackOverflow. I’m left thinking that Buildr is dying.

Gradle has an awesome bunch of plugins and the eco-system is growing. ...

September 10th, 2015

# No locks for performance under load with concurrency

Interesting:

It’s quite clear that the lock-free approach scales a lot better under contention. This follows our intuition because lock-freedom allows system-wide progress even when a thread is blocked. If one goroutine is blocked on an insert or lookup operation, other operations may proceed. With a mutex, this isn’t possible.

Matchbox performs well, particularly in multithreaded environments, but there are still more optimizations to be made. This includes improvements both in memory consumption and runtime performance. Applying the Ctrie techniques to ...

September 10th, 2015

# Use Specter to transform Clojure’s lists

Interesting:

To change the name of all the stations named “Barista” to “Coffee Master”, I can use a for comprehension:

(for [site sites station (:stations site)] (if (= (:name station) "Barista") (assoc station :name "Coffee Master") station))

But using Specter:

(->> sites (transform [ALL :stations ALL #(= (:name %) "Barista")] #(assoc % :name "Coffee Master")))

Now, this example is not actually doing justice to Specter. But in a previous (superseded) versions of the ...

September 10th, 2015

# Matthew Phillips makes the case of Clojure’s “for” comprehensions

I am lazy so I would probably handle this with “reduce” as I then have a free-form function in which I can do whatever I want. And yet, list comprehensions are more idiomatic. Their limits make it more obvious what the structure of data should be. And for deeply nested items, they can handle everything at once, rather than needing to do nested calls to “reduce”.

But you can also do it this way, using a list-comprehension in Clojure’s for ...

September 10th, 2015

# Prismatic’s Schema is better than Typed Clojure because it has coercions

Interesting:

One very useful feature of Prismatic/schema which core.typed does not have is called coercions. With this feature you can both validate your data structure and transform it to the desired state. This is particularly useful when validating input data from a database or ring request and doing all the string->int, kebab-casing etc in one step. The net effect being much less defensive code ‘on the other side’ of the coercer since you can rely on the exact shape of the ...

September 10th, 2015

# Why is microservices popular now?

This is a word that came out of nowhere and took over the industry.

I wrote a good essay about microservices a few months before the word was invented.

Martin Fowler came up with the word microservices a few months later. He said:

Microservice practitioners, usually have come from an evolutionary design background and see service decomposition as a further tool to enable application developers to control changes in their application without slowing down change. Change control doesn’t necessarily ...

September 10th, 2015

# Fortran still survives

Interesting:

When you actually take a look at the bulk of FORTRAN, it looks suspiciously like C — and it has C linkage. So, to me personally, I lump FORTRAN and C together in my head and then the question becomes “What real advantage does the switch to C++ from C have for you?”. You can find vast amounts of information on the web to answer that question :)

I have not known a single person in over a decade to ...

September 10th, 2015

# Checkpoint and Restore In User Space (CRIU)

Interesting:

Luckily for us as we were investigating this possibility we ran into an incredibly ambitious open source project called Checkpoint and Restore in User Space (or CRIU for short). The name says it all. CRIU aims to give you the same checkpointing capability for a process tree that virtual machines give you for an entire computer. This is no small task and actually required changes to the mainline Linux kernel to pull off. The original goal of this project ...

September 10th, 2015

# Mutable versus Immutable history

What really happened? What is the state of your system?

Explaining the ideal behavior of our notebooks is relatively simple: regardless of how you enter or edit cells, it should show the results of executing the file from top to bottom: the same way node does. The easiest way to accomplish this of course is to just re-run the entire document from the start after every change. This is in fact how the “rewind” feature in works in bpython. Of ...

September 9th, 2015

# The conceptual clarity of CQRS

Interesting:

I found the cognitive load to be less than with other service based architectures I have worked on. I could jump into any area of the code base and because of the naming conventions for commands and events I could see what was going on – this has never been my experience with a service oriented architecture.

The main change in developer thinking that is necessary is that you don’t record state you record state transitions. Once that is internalised ...

September 7th, 2015

# Things that make Clojure beautiful: namespace declarations

In Python, and Java, the import statements at the top of the file are big jumble of disorganized statements. In Clojure the namespace declaration is a data structure that can be cleaned up using all the tools Clojure has for working on a data structure. Apparently that is the idea behind Slamhound, which detects when imports are not being used. That is clever. I did not know about Slamhound. I will have to check that out.

Also this:

which was ...

September 5th, 2015

# Is frontend Clojure development the best frontend development eco-system

I have not done much frontend work these last 3 years, so I am out of touch, but wow, there is a lot going on:

The first step is to define the routes we want. One of the designing features of bidi is that routes are data structures, not function/macro calls:

The app-routes ...

September 5th, 2015

# Limit the number of threads in Jetty

I do something similar, but I like the use of env, which I do not do:

(defn -main [& args] (let [config {:port (Integer/parseInt (or (env :port) "3000")) :join? false :min-threads (when (env :min-threads) (Integer/parseInt (env ...

September 5th, 2015

# Use ZeroMQ instead of Kafka

Considering how radically different these are, this is a funny comparison:

Stability was the key aspect of Kafka we were unhappy with after a yearlong journey with it.

An HA deployment of Kafka requires an HA deployment of zookeeper, which Kafka uses to coordinate distributed state and configuration. As I explained before, we’ve experienced a number of stability issues with this stateful cluster maintaining consistency through outages such as VM recycle. Some serious engineering time was going into reacting to issues, ...

August 30th, 2015

# Matrices are representations of linear transformations

A good intro to an amazing topic:

Source

August 29th, 2015

# Rich Hickey: Would you want to be on a team that only did what was easy?

A dangerous attitude: “I like this because its easy for me right now.” Rich Hickey talking at the Rails conference. In the obvious case, this is a wonderful criticism of Rails.

He asks if anyone would want to join the Foo Fighters if they were the Kazoo Fighters. What if they felt guitars and drums were hard, so they only played kazoos, because kazoos are easy?

We should make decisions based on the work we need to do, not because ...

August 29th, 2015

# Hashmaps are not simple

Interesting:

I was at this talk and I disagree with his fundamental statement that simple + simple = simple. I program in Ruby one of the biggest problems beginners make is not creating complex data structures where they are needed. Instead they pass around hashes of hashes of hashes. Why? Hashes are simple, they’re easy to understand and work with. Unfortunately this initial simplicity introduces unexpected complexity, things like deep dup and deep merge are now needed. Every part of ...

August 26th, 2015

# Garajeando takes on the Gilded Rose Kata in Clojure

I dislike this line:

not-conjured-item-name (clojure.string/replace name #”Conjured ” “”)

Seems like calling “update” twice could be done with the opposite of this test:

(if (.contains name “Conjured”)

How to find the name of the unconjured item? I agree that is a tough issue.

For something this short, perhaps it doesn’t matter, but if this was non-trivial software, I would store a “:conjured” key in “items” and match against that. Seems less fragile than matching against a string. ...

August 26th, 2015

# Stuart Sierra’s anti-patterns for Clojure

This is a subtle difference:

If the operation requires a collection, then pass it a collection every time.

A “helper” like wrap-coll saves you a whopping two characters over just wrapping the argument in a literal vector, at the cost of lost clarity and specificity.

If you often forget to wrap the argument correctly, consider adding a type check:

(defn process-batch [items] {:pre [(coll? items)]} ;; ... )

If there actually are two distinct ...

August 26th, 2015

# Hauptsatz is the central result establishing the significance of the sequent calculus

An interesting preliminary for sequent theory. I’m interested because of my interest in Shen.

The cut-elimination theorem (or Gentzen’s Hauptsatz) is the central result establishing the significance of the sequent calculus. It was originally proved by Gerhard Gentzen 1934 in his landmark paper “Investigations in Logical Deduction” for the systems LJ and LK formalising intuitionistic and classical logic respectively. The cut-elimination theorem states that any judgement that possesses a proof in the sequent calculus that makes use of the cut ...

August 26th, 2015

# Natural language processing (NLP) is a messy and difficult affair

Interesting:

Similar words are nearby vectors in a vector space. This is a powerful convention since it lets us wipe away a lot of the noise and nuance in vocabulary. For example, let’s use gensim to find a list of words similar to vacation using the freebase skipgram data6:

from gensim.models import Word2Vec fn = “freebase-vectors-skipgram1000-en.bin.gz” model = Word2Vec.load_word2vec_format(fn) model.most_similar(‘vacation’)

# [('trip', 0.7234684228897095), # ('honeymoon', 0.6447688341140747), # ('beach', 0.6249285936355591), # ('vacations', 0.5868890285491943), # ('wedding', 0.5541957020759583), # ('resort', 0.5231006145477295), # ('traveling', 0.5194448232650757), # ('vacation.', 0.5068142414093018), # ('vacationing', ...

August 23rd, 2015

# Why use Clojure at your startup

I guess these are becoming the standard set of answers?

The specific design decisions and abstractions in clojure which I expect to minimize costs are the following:

Immutability. Engineering is a super giant rubics cube. To solve it, you need to know each moving part. When you get all the moving parts to finally be in the correct locations and move together, that’s when you’ve solved it. Immutable data structures don’t move. The less moving parts you have, the more brainpower ...

August 23rd, 2015

# E.W.Dijkstra on multi process systems

There was some moment, in the late 1960s and early 1970s, when the field of computer science hit a happy fluid moment. By that point, a large number of people had been working with computers for 20 years, so they knew what worked and what did not work, and yet nothing had been settled yet, no architectures dominated. And then, suddenly, computers became a monoculture with standard parts:

1.) a CPU

2.) random access memory

3.) a storage device such as a ...

August 23rd, 2015

# C. A. R. Hoare in 1973

I am surprised at how little has changed since 1973:

I would like in this paper to present a philosophy of the design and evaluation of programming languages which I have adopted and developed over a number of years, namely that the primary purpose of a programming language is to help the programmer in the practice of his art. I do not wish to deny that there are many other desirable properties of a programming language, — for example, machine independence, ...

August 22nd, 2015

# Paul deGrandis in 2012, the emergence of Clojure success stories

I like this talk by Paul deGrandis because he says he is not an Enterprise guy, he is a startup guy, and he says Clojure is more a startup technology than an Enterprise technology. I like that because I work with startups. Some people say that Clojure is best as Clojure In The Large, but I don’t work in the large. He says Enterprise CTO’s are driven mostly be fear.

Clojure is great for building a prototype that can become ...

August 22nd, 2015

# Camille Fournier (Rent the Runway) – Consensus Systems for the Skeptical Architect

ZooKeeper owns your availability. Worth watching. I thought it was amazing when she said that even 5 years ago, even at Google, most programmers were not thinking in terms of distributed computing. Instead, its been the boom in microservices that has driven the adoption of distributed computing. I wonder why microservices wasn’t more popular 10 years ago? Everything in software started off monolithic and has gotten smaller. I suppose that is because of the increasing ease of composition?

Source

August 20th, 2015

# Concurrency is easier with Clojure

I’m helping a co-worker write concurrent code in Java. I’m reminded, once again, that Clojure is easier. In Java, we must wrestle with the issue of synchronized blocks. We have an operation that reads a value from a hashmap and then deletes the key from the hashmap. We are using the hashmap almost like a queue: many threads might read from this hashmap, and yet we only want one thread to read from this hashmap. Therefore, both the read and ...

August 20th, 2015

# SSL for Jetty via Apache reverse proxy

I found this to be useful

<VirtualHost *:443> SSLEngine on (...) ProxyRequests Off ProxyVia Off ProxyPreserveHost On AllowEncodedSlashes NoDecode RequestHeader set X-Forwarded-Proto https RequestHeader set X-Forwarded-Port 443 <Proxy *> Order deny,allow Allow from all </Proxy> <Location /test> ProxyPass http://localhost:8666/test nocanon </Location> (...) </VirtualHost> Source

August 20th, 2015

# How to create a POJO in Clojure

The premise sounds interesting:

Those times you need to have Java APIs.. Some of these APIs need to return data. In Clojure it is usually a map:

{:q “What is..?” :a 42}

In Java it is not that simple for several reasons.. Java maps are mutable, there are no idiomatic tools to inspect, destructure them, Java (programmers) like different types for different POJOs, etc..

I would have appreciated an actual example. The great thing about Clojure is how it makes data access easy. ...

August 20th, 2015

# SalesForce is confusing

Reading through SalesForce documentation I constantly come upon 404 errors:

They don’t keep the documentation up to date.

SalesForce has so many products that it is difficult to figure out which bit of documentation applies to whatever it is that you are trying to do.

Consider the question, “When does it make sense to add a Connected App to a Managed Package?” This is something I’ve wondered. And no one can give me a reliable answer.

August 20th, 2015

Interesting:

In Camille Fournier’s excellent talk on consensus systems, she advises that “Zookeeper Owns Your Availability.” Consensus systems are a necessary and powerful tool, but they add complexity and new failure modes. Specifically, if the consensus system goes down, you can’t do work any more. In Chronos’s case, you’re not just running one consensus system, but three. If any one of them fails, you’re in for a bad time. An acquaintance notes that at their large production service, their DB ...

August 20th, 2015

# Bad habits: someone reports a bug so you insult them

This is a good example of how to do things wrong. Kyle Kingsbury did another one of his epic posts on Jespen. Every one of his posts so far has been a masterpiece of technical investigation. In the latest article he found a bug in Chronos, and he created an “issue” on Github. One of the programmers of Chronos then insulted him.

Kyle Kingsbury says:

I’m still not clear how exiting preserves correctness and prevents “split brain behavior”, but I’ve ...

August 11th, 2015

# Redis Out Of Memory error

Interesting:

I’m getting “OOM command not allowed” when trying to set a key, maxmemory is set to 500M with maxmemory-policy “volatile-lru”, I’m setting TTL for each key sent to redis.

info command returns : used_memory_human:809.22M

If maxmemory is set to 500M, how did I reached 809M ? Info command does not show any Keyspaces , how is it possible ? keys * returns “(empty list or set)” ,I’ve tried to change db number , still no keys found. Here is info command output:

redis-cli -p 6380 ...

August 11th, 2015

# The wrk command replaces Apache Bench

Apache has long had a strange gap in its testing tools. There was “bench” and there was “flood”. bench is easy but only handles HEAD requests. flood can do everything but requires a complicated XML config.

Now the team at Netty have spun off the wrk command, which looks very useful:

wrk - a HTTP benchmarking tool wrk is a modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU. It combines ...

August 5th, 2015

# The basics of working with Supervisord

I’ve come up with a system whereby:

1.) Jenkins pulls our code from the master branch on Github

2.) I wrote a build script that I store in /usr/local/bin

3.) Jenkins calls the build script as one of its build steps

4.) I wrote a script that finds the PIDS of existing instances of the app, and kills them:

#! /bin/sh ps aux | grep SSAM | awk '{print \$2}' | xargs kill

5.) Jenkin calls the kill script as its last build step

6.) ...

August 4th, 2015

# Supervisord is Python

A reminder to myself, liking Supervisord entails liking Python idioms:

A [program:x] section actually represents a “homogeneous process group” to supervisor (as of 3.0). The members of the group are defined by the combination of the numprocs and process_name parameters in the configuration. By default, if numprocs and process_name are left unchanged from their defaults, the group represented by [program:x] will be named x and will have a single process named x in it. This provides a modicum of backwards compatibility ...

August 1st, 2015

# The humility of the Clojure community

Very true:

I was bragging to Alan about a Drake release, which I was calling 0.2.0. Alan rolled his eyes. He was like, “Considering the conventions put forth by Semantic Versioning, and considering that Drake has been in production at this company as well as other companies for years now, why not call it version 1 already?” I pushed back a bit, by surveying two existing open source Clojure projects I know of:

Aleph, Zach Tellman’s notorious asynchronous library for ...

July 31st, 2015

# But you can customize the code?

Mike Knepper says Ruby gems are dangerous, but I am ambivalent:

The real problem with Devise commandeering all these components is that it makes the system extremely rigid. If the business logic changes (and yes, it will change) in a way that affects user sign-in or account creation, the developer is limited to modifying the app within the constraints of Devise. The developers behind Devise may not have anticipated a situation quite like yours, so getting Devise to work with the ...

July 31st, 2015

# Refactor async work in Ruby

Kevin Buchanan makes an interesting point:

But, that’s starting to seem like a lot of behavior, and maybe that behavior is crucial enough to our application that we want to have more control over it, or want one place to go to change how we retry asynchronous tasks in our application. If we consider this retry, backoff, failure logic a key feature of our app, we probably don’t want to be coupled to using Sidekiq for this. What if tomorrow ...

July 30th, 2015

# Innovation has slowed since the 1970s

Interesting:

“Real rocket science” took place almost 50 years ago, with the Apollo moon landing. The Apollo missions set the speed record for humans at roughly 40,000 km/hour. But after that, the rocket science advances started to slow down. From 1685 on, the number of scientific papers published doubled every fifteen years—he likened it to Moore’s Law—but that leveled off in the 1970s.

Who was doing this rocket science, he asked; who was programming these rockets and spacecraft to land on the ...

July 29th, 2015

# Could I ever explain monads to someone else?

I would like to get to the point where I feel comfortable explaining monads to others:

In functional programming, a monad is a structure that represents computations defined as sequences of steps: a type with a monad structure defines what it means to chain operations, or nest functions of that type together. This allows the programmer to build pipelines that process data in steps, in which each action is decorated with additional processing rules provided by the monad.[1] As such, ...

July 28th, 2015

# When to use repeating tasks inside of an app?

I would guess the best thing about cronj is its focus on thread management. And yet, I am in favor of very small apps, which may only have a single task that repeats. But this raises problems for the JVM. If I bundle all tasks into a single JVM process, then the highly optimized JVM engine can take full advantage of the careful thread management of cronj. If I have separate apps, then they compete for resources (though the newest ...

July 28th, 2015

# Why Docker failed

Interesting:

Building container images for large applications is still a challenge. If we are to rely on container images for testing, CI, and emergency deploys, we need to have an image ready in less than a minute. Dockerfiles make this almost impossible for large applications. While easy to use, they sit at an abstraction layer too high to enable complex use-cases:

Out-of-band caching for particularly heavy-weight and application-specific dependencies

Accessing secrets at build time without committing them to the image

Full control over ...

July 28th, 2015

# Trying to convince devs of the merits of Clojure

Interesting:

Leon Grapenthin:

I have tried various different approaches from convincing of Clojure advantages in the Java devs concrete domain, showing off incredibly awesome toy projects, larger projects, not tryng to sell, trying to sell, sending ClojureTV videos and what not approach you can think of. I have not managed to introduce one Java dev to Clojure in a way that he picked it up and had no interest before. I have spent many hours thinking about how I could ...

July 28th, 2015

# The limits of RESTful interfaces

Interesting:

The chart leaves most of the rows blank for the year 1995, but of course, all of the necessary technologies existed in 1995, and in 1989. The focus of the article is on HTTP, but the article might be stronger if it confronted why HTTP was successful, when it lacked important abilities such as Authentication. There were other technologies that offered Business Logic and Logging and Realtime, back in 1989, so why weren’t they more successful?

It’s also curious ...

July 28th, 2015

# Mutable arrays in Clojure

Chris Zheng has done some impressive work with mutable arrays:

Predicates as Datastructure I’m quite proud of the expressiveness of the ova syntax. We can see below the number of possible ways that we can select elements out of the array. Here is just a sample of what can be done using the select function.

(def ov (ova [{:val 1} {:val 2} {:val 3} {:val 4} {:val 5} {:val 6} ...

July 27th, 2015

# When should Clojure imitate Object Oriented Programming?

Chris Zheng writes:

Servers that are running on a particular port can be tracked and stopped. I have to say, this was the feature that I wanted the most, which motivated the framework’s design. The annoying thing about development in emacs is that I have to be careful of not losing the reference to the server. Since there was no way of stopping it unless the repl is restarted. I wanted to implement a registery for references to running servers ...

July 26th, 2015

# The maybe monad represents computations whose result is maybe a valid value, but maybe nil

Here is a sentence I have read a dozen times without understanding it:

The maybe monad represents computations whose result is maybe a valid value, but maybe nil.

This seems like a good way to think about the Any type, when dealing with things such as Typed Clojure.

Source

July 26th, 2015

# The sins of Adam Bard

This last year I find that everything Adam Bard writes is worth reading. I have not been checking his blog very often but I should. I just saw this post from a year ago, which is really fantastic:

Any time you put a let in a let or an if in an if, that’s a strong code smell. It means you’re still writing code in a way that requires you to short-circuit your functions with returns.

There is a fascinating debate ...

July 25th, 2015

# Scheduling libraries in Clojure

Chris Zheng has some interesting arguments for his particular take on these issues:

There are now many more scheduling libraries in the clojure world:

at-at chime clj-cronlike cron4j monotony quartzite schejulure

With so many options, and so many different ways to define task schedules, why choose cronj? I have listed a number of design decisions that make it beneficial. However, for those that are impatient, cut to the chase, by skipping to the simulations section.

…In reviewing other scheduling libraries, it was found that fully-featured thread management capabilities were ...

July 25th, 2015

# Will I regret Functional programming?

I’ve spent the last year writing negative things about Object Oriented Programming, and I’ve been among the many people arguing that Functional Programming is the future. And I’ve talked about the political forces that drove the adoption of Object Oriented Programming. But that does raise the issue, what political forces are driving the adoption of Functional Programming? One possible answer is “None” and that would explain why Functional Programming remains a small niche compared to Object Oriented Programming. Another is ...

July 24th, 2015

# Why Python 3.x went off course

A fascinating post from 2005, which is when Python began to veer off course. Python 2x had some beautiful features that could have been further developed, but instead, with 3.0, Python went down the classic Object Oriented road. In this post, Guido van van Rossum explicitly rejects much of the Functional paradigm that Python had picked up from Lisp.

So now reduce(). This is actually the one I’ve always hated most, because, apart from a few examples involving + or *, ...

July 24th, 2015

# They cost of using Hystrix

Ouch!

We cannot use Clojure’s concurrency primitives (futures/promises/agents).

That is fascinating to think that at some point Clojure’s concurrency primitives are not enough, and so we need to give up on them and move to a Java library. I am aware that Netflix is dealing with unusual scale, but what is the point of Clojure if it doesn’t automate exactly these issues?

Marc: When using Aleph, the choice for lamina would seem a sure thing. What made you turn ...

July 24th, 2015

# The simplest step toward circuit breakers in Clojure

I think I missed the importance of this the first time I read it:

Consider you have this line within a service response:

{:body @(future (client/get “http://soundcloud.com/blah/wah”)) :status 200}

Now http://soundcloud.com/blah/wah goes down and those client requests start getting blocked on the request. In Clojure all future calls acquire a thread from the same thread pool. In our example the service is blocked up, is pilling new requests onto the blocked pool and we are in trouble.

My first solution to this problem ...

July 24th, 2015

# Chris Zheng is consolidating his libraries into Hara

I have some questions for Chris Zheng. I admire Ribol very much, though I was not aware of some of the other repos. I am curious why you are moving forward with an integration of these libraries? The Clojure community has so far avoided frameworks and even large libraries have met with rejection. What do you hope to gain by consolidating these projects?

The projects:

iroh has been moved to hara.reflect

ova has been moved to hara.concurrent.ova

cronj has been superceded by ...

July 24th, 2015

# Property based testing in the Functional Paradigm

Interesting:

Which brings me to another killer feature of many functional languages: property-based testing (PBT). While PBT is by no means an exclusive domain of functional languages, it does have deep roots in Haskell in the form of QuickCheck. What sets PBT apart from its traditional unit and integration brethren is its emphasis on defining a set of properties that describe a function and ability to generate a lot of random test cases to attempt to falsify these specifications. This ...

July 24th, 2015

# Functional programming is not the same as static typing

I like this article, but it does confuse Functional Programming with static data types. Consider this example of evil Ruby code:

Take for instance the following Ruby code that uses Virtus.

class Events < Array def Source

July 24th, 2015

# Dealing with mutation is hard

Obviously, I agree with this:

Dealing with mutation is hard. When I constantly grappled with these issues early on in my career, I thought I had serious issues. Perhaps I wasn’t cut out to be a developer. Maybe I should seriously consider a different career. It didn’t help that other developers breezed through similar assignments as if mutation and them were one (don’t get any ideas). But then I started reading about functional programming and how everyone was endlessly whining about ...

July 16th, 2015

# The worsening web

Let’s dive deeper into the CNN article. Among the 200+ HTTP requests the page makes are calls to 25 different domains.

Yes you read that correctly. TWENTY…FIVE. Among them are a few that are clearly ad related (ex. ad.doubleclick.net, pixel.moatads.com), a few that serve some analytics function, and many whose names are intentionally obfuscated to confuse us.

Jeff Edsell:

The part of web cruft I hate most:

“Why does our page load so slow? Should you compress the images ...

July 16th, 2015

# Greg Hendershott looks at Racket Macros

I wanted to ask Greg Hendershott for permission to copy his code in Clojure, but I could not find an email address for him. My own email is lawrence@krubner.com. I love this epilogue from Greg Hendershott:

"Before I had studied Chan (Zen) for thirty years, I saw mountains as mountains, and rivers as rivers. When I arrived at a more intimate knowledge, I came to the point where I saw that mountains are not mountains, and rivers are not rivers. But ...

July 15th, 2015

# Vim has a function with 400 lines of code

When I worked at Category4, they had a homegrown HTML library. The method that generated HTML tables was over 1,100 lines of code (not a typo). So I’ve seen some fairly large functions. Dealing with them is why I now prefer 5 line functions written in a clean language like Clojure.

This is interesting:

This function is over 400 lines and contains over 40 #ifdefs. Its job? To wait for keyboard input. Several factors caused this code to be so ...

July 15th, 2015

# What does the word “Cancel” mean?

Some of the worst User Interface Design I’ve seen in awhile. The Amtrak website uses “Cancel” as the text on the button that cancels your reservation. Almost everywhere in the world, if you see a button that says “Cancel,” it means you do not want to do something. But Amtrak uses it here, so if you do want to cancel your reservation, you have to hit the Cancel button.

Source

June 21st, 2015

# Image recognition: we can not get there from here with what we’ve got

Interesting:

Leopards (or jaguars) are complex 3-dimensional shapes with quite a lot of degrees of freedom (considering all the body parts that can move independently). These shapes can produce a lot of different 2d contours projected on the camera sensor: sometimes you can see a distinct silhouette featuring a face and full set of paws, and sometimes it’s just a back and a curled tail. Such complex objects can be handled by a CNN very efficiently by using a simple rule: ...

June 21st, 2015

# Where does image recognition fail?

Photo shows image recognition has a long way to go.

Source

June 20th, 2015

# How NodeJs fails to deal with backpressure

Interesting:

The Node concurrency model is kind of like a credit card for computing work. Credit cards free you from the hassle and risks of carrying cash, and they are completely great in almost every way, unless you spend more than you make. It’s hard to even know you have spent more than you make until the next bill comes. Similarly, node lets you do more work, and it’ll call you back when it’s done, whenever that is. You might not ...

June 19th, 2015

# Git history should show you reality, not a cleaned up version of reality

Well said:

The obsession of git users with rewriting history has always puzzled me. I like that the feature exists, because it is very occasionally useful, but it’s one of those things you should almost never use.

The whole point of history is to have a record of what happened. If you’re going around and changing it, then you no longer have a record of what happened, but a record of what you kind of wish had actually happened.

How are you ...

June 15th, 2015

# The New Netflix

I have my doubts about the heavy use of Javascript that’s become normal over the last 5 years.

I got my current MacBook Pro in 2011 and for awhile it seemed like a fast computer. Nowadays, however, if I launch Chrome, and I have 6 or 7 pages open, the computer is under stress. If I open the terminal and run “top” I see that the load varies between 1 and 3. It never goes below ...

June 10th, 2015

# The new RESTful style

Apparently this style is now popular with some developers:

Source

June 9th, 2015

# Strange problems with Rackspace cloud machine

What to make of this?

root@cloud-server-01:~# emacs The program ‘emacs’ can be found in the following packages: * emacs24 * emacs24-nox * e3 * emacs24-lucid * jove Try: apt-get install root@cloud-server-01:~# apt-get install emacs24 Reading package lists… Done Building dependency tree Reading state information… Done emacs24 is already the newest version. emacs24 set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded. 134 not fully installed or removed. After this operation, 0 B of additional disk space will be used. Do you want to continue? [Y/n] ...

June 9th, 2015

# Avoid RejectedExecutionException in lein :(

I am curious what Dave Ray wrote here in his Seesaw repo.

(defn -main [& args] (when-not (first args) (println "Usage: gaidica ") (System/exit 1)) (reset! api-key (first args)) (invoke-later (-> (make-frame) add-behaviors show!)) ; Avoid RejectedExecutionException in lein :( @(promise))

Does anyone know what this about? What causes RejectedExecutionException in lein?

Source

June 2nd, 2015

# When to use Patch

Interesting:

When should we use the PATCH HTTP method?

The HTTP methods PATCH can be used to update partial resources. For instance, when you only need to update one field of the resource, PUTting a complete resource representation might be cumbersome and utilizes more bandwidth

Also, the PUT method is idempotent. PUTting the same data multiple times to the same resource, should not result in different resources, while POSTing to the same resource can result in the creation of multiple resources.

June 2nd, 2015

# Working with monoids

Interesting:

Simply put, Monoids describe types containing a binary function and an identity value.

When applied to the identity value and a random value x, said function leaves its argument x untouched, returning it as a result. This short description should be enough to get the conversation started.

Here’s how Haskell defines a Monoid:

class Monoid m where mempty :: m mappend :: m -> m -> m mconcat :: [m] -> m ...

June 2nd, 2015

# A pipeline of agents creates a fully asynchronous programming model

I recall reading this code in 2010 or 2011, when I was first learning Clojure, and at the time there was a great deal about it that I did not understand, so I missed the point:

(def logger (agent (list))) (defn log [msg] (send logger #(cons %2 %1) msg)) (defn create-relay [n] (letfn [(next-agent [previous _] (agent previous))] (reduce next-agent nil (range 0 n)))) (defn relay [relay msg] (letfn [(relay-msg [next-actor hop msg] ...

June 1st, 2015

# Using reducers on the bytes in a memory mapped file

This could be a powerful technique, especially when speed and parallelization are important:

First, we’ll have a couple of parameters: the character set for the input and a hint for the size of chunk to break the file into. (def ^:dynamic *charset* (Charset/forName "UTF-8")) (def ^:dynamic *chunk-size* (* 10 1024 1024)) With those, we’ll break the file into chunks by skipping through it and reading ahead until we get to the end of a line. Later, when we actually read the file, this ...

June 1st, 2015

# Higher order functions in Clojure support chaining

Interesting:

One easy-to-miss strength of this approach is that it supports chaining. In a few places in our application, we wanted to send a computation to an agent and observe the state of the computation (e.g. to show and hide a spinner). We didn’t want to have to wire observation hooks or callbacks directly into the process, so we created a contextual function (defn with-process-callbacks [fire fun & args]). with-process-callbacks returns a function which calls fun with args, but calls fire ...

June 1st, 2015

# Stuart Sierra says avoid Dynamic Scope

Dynamic scope will ruin your life. In a different presentation he says that dynamic scope is okay so long as the var is really private, and marked as private.

The problem with this pattern, especially in libraries, is the constraints it imposes on any code that wants to use the library. The with-resource macro severely constrains what you can do in the body:

You can’t dispatch to another thread. Say goodbye to Agents, Futures, thread pools, non-blocking I/O, or any other ...

June 1st, 2015

# Mark Seemann feels that mutating state is intuitive

This is a case of someone trying to be open-minded, but still being biased because of their years spent getting comfortable with a very particular form of programming that allows a very particular form of mutable state:

In FP they’ve come up with this clever concept of monads to ‘work around’ the problem of mutating state. Yes, monads are very clever, but if they feel foreign in OOD it’s because they’re not required. Mutation is an inherent part of the ...

May 28th, 2015

# Simple Design

Interesting:

Yagni is a way to refer to the XP practice of Simple Design (from the first edition of The White Book, the second edition refers to the related notion of “incremental design”). [1] Like many elements of XP, it’s a sharp contrast to elements of the widely held principles of software engineering in the late 90s. At that time there was a big push for careful up-front planning of software development.

Let’s imagine I’m working with a startup in Minas ...

May 26th, 2015

# Why Appsflyer gave up on Python and switched to Clojure

Interesting:

At AppsFlyer we actually started our code base in Python. Two years later this wasn’t enough to handle the growing number of users and requests. We started to encounter issues like one of the critical Python processes taking too long to digest the incoming messages, caused mainly by string manipulations and Python’s own memory management system. Even partitioning the messages amongst several processes and servers could not overcome this. This eventually killed the process and caused data loss – ...

May 23rd, 2015

# Concurrency in Ruby

Celluloid is interesting:

Celluloid provides a simple and natural way to build fault-tolerant concurrent programs in Ruby. With Celluloid, you can build systems out of concurrent objects just as easily as you build sequential programs out of regular objects. Recommended for any developer, including novices, Celluloid should help ease your worries about building multithreaded Ruby programs.

Much of the difficulty with building concurrent programs in Ruby arises because the object-oriented mechanisms for structuring code, such as classes and inheritance, are separate ...

May 21st, 2015

# Sandi Metz tries to solve Object Oriented problems with composition

She says “We love dependency injection”.

First she acknowledges that the “proper” Object Oriented approach leads to more code for the same behavior:

but then she says the solution is to use composition, which offers all kinds of flexibility:

She says it is wonderful because in the end we get “pluggable behavior”.

But if it is a good idea to pull certain methods out of an object, and make them independent, in an object that can then be injected ...

May 21st, 2015

# Sandi Metz on hidden assumptions

Interesting:

Our code is full of hidden assumptions, things that seem like nothing, secrets that we did not name and thus cannot see. These secrets represent missing concepts and this talk shows you how to expose those concepts with code that is easy to understand, change and extend. Being explicit about hidden ideas makes your code simpler, your apps clearer and your life better. Even very small ideas matter. Everything, even nothing, is something.

Source

May 19th, 2015

# Predator-Prey modeling in Clojure

Interesting:

Introduction to Predator-Prey Systems In this edition of the blog, I’ll discuss how you can use Clojure to solve a system of nonlinear differential equations. The particular system I’ll be solving is the Predator-Prey or Lotka-Volterra Model, which you can read all about right here. The basic concept is that you are modeling a population of predators and prey (e.g. Foxes and Rabbits, which we’ll use here). How the two populations change over time can be modeled like so:

dR/dt=αR−βRF dF/dt=−γF+δRF

These are differential ...

May 10th, 2015

Interesting and important:

Problem:

A program may require to execute some pieces of instructions when application goes down. An application may go down because of several reasons:

Because all of its threads have completed execution

Because of call to System.exit()

Because user hit CNTRL-C

System level shutdown or User Log-Off

Concept at Abstract level: Write all the instructions(java code) in a thread’s run method and call java.lang.Runtime.addShutdownHook(Thread t). This method will then register this thread with JVM’s shutdown hook. At the time of shutting down, JVM will run ...

May 10th, 2015

# Test all your code in production using “feature flags”

I was talking to someone at Viacom, and he told me they were doing something very clever: they use feature flags to limit who can see new features, and in fact, all new features are at first limited to the QA team. Because of this, they have no development servers — everything is tested in production. This allows the testing to be much more realistic.

Feature flag libraries in Ruby and Clojure:

Ruby: rollout

Clojure: shoutout

Source

May 6th, 2015

# Anyone can contribute to clojure-doc.org

Clojure-doc.org is the place for people to contribute. This is the important bit:

Contributor-friendly This material is not covered by the Clojure Contributor Agreement and is developed using pull-requests on GitHub.

This is discussed in an interesting thread on the Clojure Google Group:

Assertion: “There are only a handful of people in the world with authorization to edit pages on clojure.org. ”

Fact: 14

Assertion: “Far more have authorization to edit the wiki pages on dev.clojure.org — hundreds, I think. A subset of those who ...

May 6th, 2015

# The changelog for the clojure.org wiki

This is not linked anywhere, but it is good to know about:

Source

May 6th, 2015

# The contents were now a boilerplate AT&T copyright notice claiming intellectual ownership of the otherwise still empty file

If the law allows a company to copyright an empty file, then the law should be changed:

In a 1984 version of Unix, things started heating up, and true grew to 276 bytes. The contents were now a boilerplate AT&T copyright notice claiming intellectual ownership of the otherwise still empty file.

Source

May 5th, 2015

# Caribou as the ultimate Clojure web framework?

Kyle Dawkins writes:

Justin Smith speaks the truth about Caribou. I’m also one of the team members, and although we did lose our funding, we’re all still around and there are a number of active Caribou projects alive and well. It does have everything Justin says, and yes, it’s imperfect, but it’s also very easy to get a site up and running quickly. But you don’t lose the modularity that we all love in the Clojure world. ...

May 4th, 2015

# If a language does not guarantee order in a hashmap can the hashmap be referentially transparent?

It’s an interesting question. Clojure says it is referentially transparent, meaning you can remove any function and replace it with its return value, with no effect on the program, but can Clojure be referentially transparent if it does not guarantee the order of its hashmaps (especially since hashmaps are also functions in Clojure)?

Note that this issue is similar to ensuring that a hash function is consistent with equals in a language, i.e. if two values x and y are ...

May 3rd, 2015

# ExecutorService makes exceptions disappear in a bad way

I have been bitten by this many times, mostly in Clojure libraries that use ExecutorService under the hood:

What will be the result of the following snippet?

executorService.submit(() -> { System.out.println(1 / 0); }); I got bitten by that too many times: it won’t print anything. No sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool just swallows this exception, as if it never happened. If it was a good’ol java.lang.Thread created from scratch, UncaughtExceptionHandler could work. But with thread pools ...

May 3rd, 2015

Tomasz Nurkiewicz offers this bit of advice, which sounds like it would also be useful in Clojure:

I can’t emphasize this. When dumping threads of a running JVM or during debugging, default thread pool naming scheme is pool-N-thread-M, where N stands for pool sequence number (every time you create a new thread pool, global N counter is incremented) and M is a thread sequence number within a pool. For example pool-2-thread-3 means third thread in second pool ...

May 3rd, 2015

# Clojure does not need job queues, because it has Storm, Onyx, Quasar, Pulsar

I find it interesting to consider the question that maybe Clojure has so many great systems for distributed processing that it does not need the classic job queue.

It’s worth considering that if you don’t need something very robust (and depending on your situation), you could just use Redis + Carmine directly to pass messages around (possibly representing jobs), and have workers pull from the message queue. There is nothing else you really need for this; it’s quite straight forward. ...

May 2nd, 2015

# Aaron Bedra: web apps in Clojure have some of the worst security

Aaron Bedra – clojure.web/with-security

Breda says the Clojure community needs to have a talk, because of some bad things that happened recently. The rate of people getting hacked is going up. He says:

“Clojure web apps are some of the worst I have seen in terms of security. We are talking about PHP-without-a-framework levels of insecurity.”

“We have bricks with no mortar.”

“If I missed your library, it’s because it doesn’t exist. I could not find it on the first page of Google, ...

May 1st, 2015

1 Comment

# Embarrassing code I wrote under stress at a job interview

(Note: this post was, in part, inspired by John Lawrence Aspden’s post about FizzBuzz.)

I write terrible code when I go to a job interview. That’s mostly because, when they ask me to solve a coding question, I get nervous. I thought it might be entertaining if I wrote about one such encounter.

Yesterday I went to a job interview, at a company in New York that had once built their stack (for managing online advertising) in Ruby but who are now ...

April 28th, 2015

# Jim Starkey says “Well, duh”

This doesn’t rise to the level of “Zed Shaw rant” style of meltdown, but Jim Starkey has decided the play the role of “famous old codger who did great work once but now seems completely out of touch“:

Any discussion of the CAP “theorem” revolves around the A – exactly what does availability mean?

The narrow sense is that availability means that all surviving nodes in all partitions continue to process work. This reduces the CAP idea to nothing more than ...

April 28th, 2015

# Kyle Kingsbury’s Jespen series is CAP enlightenment

So, again, I am late to the party. Yesterday, by accident, I discovered Kyle Kingsbury “Jespen” series. I stayed up till 4 AM reading through it. Every article is completely amazing. I wish he could do an article on every NoSQL database, but I read somewhere that each article cost him months of effort, so it is impossible to expect more from him. I recommend every single one of these articles:

Source

April 27th, 2015

# Redis losing 56% of writes during a partition

Kyle Kingsbury (the same guy who does cool Clojure stuff like Reimann and teasers) has an epic writeup on Redis:

“In this post, we demonstrate Redis losing 56% of writes during a partition.”

I have trouble finding the pithy excerpt here, because it is all good. I guess I’ll just quote the opening where Kingsbury sets up the argument, and then you can read the rest for the way things fall apart:

Redis offers asynchronous primary->secondary replication. A single server is ...

April 27th, 2015

# Can MongoDB keep its promises?

Kyle Kingsbury starts the fight without meaning to:

In this test, MongoDB returns the value “0″ for the document, even though the only possible values for the document at that time were 1, 2, 3, or 4. The value 0 was the proper state at some time close to the partition’s beginning, but successful reads just after the partition was fully established indicated that at least one of the indeterminate (:info) CaS ...

April 27th, 2015

# Parallelism versus concurrency

Someone posted this on Hacker News, and it is very good. Parallelism sends equal work to every processor, concurrency is the messy chaos of sending work to whatever processor is free, or bundling up many threads on one processor.

Source

April 27th, 2015

# Disque is a distributed and fault tolerant message broker, so it works as middle layer among processes that want to exchange messages

The people who gave us Redis announced a new project today:

Disque is a distributed and fault tolerant message broker, so it works as middle layer among processes that want to exchange messages.

Producers add messages that are served to consumers. Since message queues are often used in order to process delayed jobs, Disque often uses the term “job” in the API and in the documentation, however jobs are actually just messages in the form of strings, so Disque can be ...

April 26th, 2015

# A very long Java method

A common rule in programming is to keep methods as small as possible, so I am surprised to see this method in a seemingly well-run project such as ZooKeeper:

private void processEvent(Object event) { try { if (event instanceof WatcherSetEventPair) { ...

April 26th, 2015

# Java performance

I like Tomasz Nurkiewicz’s summary of this book:

JIT (just-in-time) compiler turns out to be one of the most important tools bringing performance to the JVM. First “real” chapter goes into great details of how JIT works in different modes, how to monitor and tune it. I learnt a lot already, but the best is yet to come. Obviously garbage collection is a major concern, thus Oaks devotes two full chapters to explain all popular GC algorithms. Besides pure description, expect ...

April 26th, 2015

# Your data is the API

Interesting:

The first time I realized that “Data as an API” was in fact a very good idea was at Kevin Lynagh’s presetation at Öredev. However, if you want something more like a struct Clojure provides something called a record. If you know Scala this is very similar to case classes. A record can be defined like this:

(defrecord Person [firstName lastName])

This creates an actual normal Java class Person with two immutable fields and implements hashCode and equals. A record also ...

April 26th, 2015

# The farce known as “object oriented programming”

Jayway has a nice critique of how much real-life object oriented programming departs from the beautiful theory:

One thing I have realized is that in Java we use classes for many purposes. In a typical web application written in Java using something like Spring framework you will often find:

Data transfer objects (DTO:s)

Services (REST API, controllers, DAO:s)

Rich objects (if you’re lucky!)

A DTO is just a struct and contains no behavior. To minimize the boilerplate in Java I tend to implement DTO:s using ...

April 26th, 2015

# New data structures in Clojure 1.8

Apparently there are a lot of new data structures which may arrive in Clojure 1.8, thanks to Zach Tellman:

So, at the end of this exercise we have more than 5000 lines of Java, and we want to add them to the core implementation of Clojure. Ideally, we won’t introduce any bugs in the process. But the same unrolling that makes the code faster makes it significantly harder to simply read the code and verify it’s “correct”. The Clojure code which ...

April 26th, 2015

# Design patterns have social value

An interesting bit from Adam Petersen:

Patterns have social value too. The format arose to enable collaborative construction using a shared vocabulary. In Patterns in C I write on the groundbreaking work of architect Christopher Alexander:

The patterns found in Alexander’s books are simple and elegant formulations on how to solve recurring problems in different contexts. [...] His work is a praise of collaborative construction guided by a shared language – a pattern language. To Alexander, such a language is a generative, ...

April 26th, 2015

# Automat for easy finite-state-machines in Clojure

Another interesting library from Zach Tellman allows for finite state machines:

For a more real-world use case, consider tracking browsing behavior on an online store. We want to see when the customer visits the cart, begins to checkout, but then returns back to the cart without having completed the checkout. Seeing this indecisive behavior, we can make a special offer.

Source

April 26th, 2015

# Using abstract classes to hide boilerplate code

I personally never do the kind of work where I need to implement abstract classes. I was asked in a job interview why I would use an abstract class, and I came up with some nonsense answer about having a default implementation for situations where my code expected a given situation 90% of the time, and edge cases only 10% of the time. But here is an interesting example in Clojure, where it interoperates with Java, using an abstract class ...

April 26th, 2015

# Zach Tellman on specialized tuples for Clojure

Other than Rich Hickey, I think I learn the most from Tellman, regarding how to do things and why they work the way they do:

Most data structures are designed to hold arbitrary amounts of data. When we talk about their complexity in time and space, we use big O notation, which is only concerned with performance characteristics as n grows arbitrarily large. Understanding how to cast an O(n) problem as O(log n) or even O(1) is certainly valuable, and ...

April 26th, 2015

# When to use reify in Clojure

This is a nice explantation. If you have a protocol, then you can instantiate with a record, like this:

(defprotocol Foo (bar [this]) (baz [this st]) (quux [this x y])) (defrecord FooRecord Foo (bar [this] (println this)) (baz [this st] (str this st)) (quux [this x y] (str this (* x y))))

But if only need to instantiate the protocol one ...

April 25th, 2015

# ExecutorCompletionService

How very much easier it is to use core.async in Clojure! My one concern is that core.async has a thread pool that is set to the number of CPUs + 2, so the thread pool is small and rigid.

Each call to contentFuture.get() waits until downloading given web site (remember that each Future represent one site) is finished. This works, but has a major bottleneck. Imagine you have as many threads in a pool as tasks (20 sites in that ...

April 25th, 2015

# Limit new features to a subset of your users

Inspired by the Ruby gem “Rollout”, Shoutout offers user buckets on which you can test new features:

Storage

Unlike rollout, shoutout is completely storage agnostic. You’ll have to implement your own storage backend, which implements ShoutoutStorage. The storage protocol has two functions, read-from-storage, and write-to-storage, both of which should be simple enough to implement. Both deal purely with serialized strings, and string keys, shoutout does the serialization logic itself.

The library provides an in memory store (used for testing) that you could ...