Technology

November 20th, 2014

Why is declarative programming so limited?

Interesting idea, though strange that this has not caught on decades ago:

Today’s computers don’t have enough contextual knowledge to make me a sandwich but there are lots of domains where this approach excels. The classic example is SQL databases. Rather than specifying exact data-structures and operations, the user specifies a high-level schema and sends descriptions of queries. It is the responsibility of the database to choose storage types, manage concurrent changes and generate query plans. It makes these decisions based ...

November 20th, 2014

Python encourages tinkering

I already linked to this, but it really is interesting. I don’t think this is possible in Ruby or PHP or script languages in general, but you can do crazy things in Python:

Above we did an in-place modification of a value in a numpy array. This is easy, because a numpy array is simply a data buffer. But might we be able to do the same thing for a list? This gets a bit more tricky, because lists store ...

November 18th, 2014

Software transactional memory in Python

Interesting:

Python has a GIL, right? Not quite – PyPy STM is a python implementation without a GIL, so it can scale CPU-bound work to several cores. PyPy STM is developed by Armin Rigo and Remi Meier, and supported by community donations. You can read more about it in the docs.

Although PyPy STM is still a work in progress, in many cases it can already run CPU-bound code faster than regular PyPy, when using multiple cores. Here we will see how ...

November 18th, 2014

Rapid intake and output of data, with control over the scale of the failures

Two new and interesting libraries:

s3-logging:

At first, we used the Hadoop S3 client, which we had used in some lightweight use cases before. Unfortunately, it ended up coping poorly. Each server had a sustained throughput of about 10 megabytes of compressed data per second, and every so often S3 would hiccup and stop accepting data for a few minutes. When this happened, Hadoop’s client would simply buffer the data in-memory until all the memory was exhausted, at which point the process ...

November 18th, 2014

Why Python is slow

This is a great comparison of C code to Python code. (Actually, this is true of most script languages, not just Python.)

Assign 1 to a Assign 2 to b call binary_add(a, b) Assign the result to c

The equivalent code in Python looks like this:

# python code a = 1 b = 2 c = a + b

here the interpreter knows only that 1 and 2 are objects, but not what type of object they are. So the The interpreter must inspect PyObject_HEAD for ...

November 18th, 2014

Perl is faster than Python

Interesting:

A related question is: why is Perl so fast? Quite a few years ago I wrote a little Runge-Kutta solver in Perl for some simulation work. It seemed like a good idea at the time. The equations of motion had to be integrated over a very long time, and it could take hours for a single run (still much faster than the Monte Carlo it was being used to do a sanity-check on). I re-wrote everything in C++, and picked up ...

November 14th, 2014

Setting up Flask to talk to Nginx

The main thing I learned was how to configure the reverse proxy for Nginx:

server { location / { proxy_pass http://127.0.0.1:8000; } location /static { alias /home/www/flask_project/static/; } }

Easy.

The configuration of the static files was also interesting:

So, HTTP requests that hit the / endpoint will be ‘reverse proxied’ to port 8000 on 127.0.0.1 ...

November 14th, 2014

Do you need to learn how to program a computer?

No. You do not need to learn how to program a computer. The question is stupid. And yet the debate goes on, endlessly. I recall this debate was going strong when I was a child. Here is the New York Times in 1984:

”SHOULD I learn to program?” and ”Do I have to learn to program?” are two variants of the question probably most asked by people testing the waters of computer ownership. The answer usually boils down to an ...

November 14th, 2014

How difficult is it to work with TCP?

I am thinking of creating a new protocol on top of TCP. I’ve been thinking that, with all the good libraries out there, I would not have to do much work. But then I read this, and it gives me pause: maybe TCP is tougher than I thought:

Zero windows are another frequent source of grief, to such lengths that at multiple mobile operators the technical staff have quizzed us extensively on our use of zero windows. I don’t quite ...

November 14th, 2014

The Rubinius community engages in extraordinary outreach

Interesting:

Here we are, in no particular order:

Sophia Shao: As a recent graduate of Carnegie Mellon University’s Electrical & Computer Engineering department, Sophia is currently tackling a massive application migration from MRI to Rubinius. She’s also been improving Rubinius every day. Hit her up for tips about debugging machine code.

Jesse Cooke: As co-founder of Watsi, a venture to fund healthcare for people around the world, Jesse was part of YCombinator’s first ever non-profit. Jesse has been contributing in any way he ...

November 12th, 2014

Martin Odersky on the transition from OOP to Functional programming

I am not wild about Scala, but Martin Odersky is a very smart guy. In this video he offers a fascinating history of OOP, which leads into his point about the transition we now face:

I appreciate Odersky pragmatic approach, he is willing to say that sometimes the mutable approach is somewhat more readable:

Odersky is self-effacing and willing to accept pragmatic definitions of “mutable” and “immutable”. He is not a purist.

I dislike the fact that Scala is so multi-paradigm ...

November 12th, 2014

Lisp is not functional, but Clojure is

Back in 1958 Lisp invented the idea of an everything-is-mutable language. Therefore, Lisp is not necessarily functional, since a mutable language is typically not functional. (Although it is possible to adopt a functional style in a mutable language, such as Javascript.) A functional language should be one where, given a set of inputs, one always gets the same output, in a deterministic fashion.

Let Over Lambda insists that Lisp is not Functional:

It seems like we use defun to define new ...

November 12th, 2014

The naming of ducks in Python

This is great:

Source

November 12th, 2014

The importance of names to signal intent in Python

Interesting:

Source

November 12th, 2014

The 11 greatest Python blogs

An interesting overview of the Python blogosphere.

Source

November 12th, 2014

Structs in Python

Interesting:

A record (aka “struct” in C) is a pre-defined collection of values where each is accessed by a unique name. Depending on the nature of the data, records may be a superior alternative to dictionaries and instances of custom classes.

PyRecord allows you to use records in Python v2.7 to v3.x and PyPy v2, and can be thought of as an improved namedtuple.

In Python terms, a record is an instance of any class for which you define some attributes but no ...

November 12th, 2014

Great ideas for zsh config

I got a lot of great ideas for my zsh config by reading this one

Source

November 8th, 2014

How to design a Clojure/Redis system with Components

This is really interesting:

The idea was to have a single client for the connection to the Twitter Streaming API and the persistence of the received Tweets in ElasticSearch, plus multiple machines to serve WebSocket connections to the client. For the communication between the processes, I picked Redis Pub/Sub because its model of communication appears to suit the requirements really well. As cute as the drawing may be, I prefer code (plus drawing), so I took the previous monolith ...

November 8th, 2014

Backends for RESTful interfaces versus WebSockets

Interesting:

This next step may or may not live in the same JVM – I haven’t decided yet. So the idea is to aggregate data on the server side and only deliver the aggregated data structures back to the client side. For example, this could be a few hundred aggregates over increments of five minutes each. These increments can easily be made addressable (and cacheable): let’s say it is 1:17pm. Then, we have a last and incomplete increment from 1:15pm ...

November 8th, 2014

Where Apache Storm fails

Very interesting post:

What comes to mind immediately when regurgitating the requirements above is Storm and the Lambda Architecture. First I thought, great, such a search could be realized as a bolt in Storm. But then I realized, and please correct me if I’m wrong, that topologies are fixed once they are running. This limits the flexibility to add and tear down additional live searches. I am afraid that keeping a few stand-by bolts to assign to queries dynamically would not ...

November 8th, 2014

How many bindings exist in your Angular app?

This script looks useful:

function getScopes(root) { var scopes = []; function traverse(scope) { scopes.push(scope); if (scope.$$nextSibling) traverse(scope.$$nextSibling); if (scope.$$childHead) traverse(scope.$$childHead); } traverse(root); ...

November 8th, 2014

All that is terrible in AngularJS

Interesting:

The scope of a variable is the part of the program where the variable can be legally referenced. If your system has variables, it has some concept of scoping.

Angular has a DSL that is entangled with the HTML and used primarily to express the data-bindings between the UI and application code. This has variables, and thus a concept of scopes. Let’s take a look at it. Consider for example ng-model:

<input type=”text” ng-model=”obj.prop” />

This creates a two way binding on ...

November 7th, 2014

Using zsh as my shell

I’ve been using bash for about 14 years, first on Linux, and the later on Mac OSX. I have heard good things about zsh. The idea of seeing my git branch listed in my terminal, at all times, will save me from many mistakes (I have often committed code on the wrong branch). This article finally convinced me to switch to zsh on my Mac:

z No this is not a typo, the last thing I want to show you is z. ...

November 7th, 2014

Logcheck keeps track of your logs, emails you when things get suspicious

I only just discovered logcheck, but now I want to use it on all of my servers:

How Logcheck Protects Your Server It turns out that Logcheck is a pretty simple tool. All it does is periodically check the log files you specify. It filters out the uninteresting parts and emails you what is left. It is smart enough to avoid mailing you the same information multiple times by keeping a “bookmark” for each log file. This way it can only ...

November 3rd, 2014

Network versus CPU bandwidth in a Storm topology

Scroll half-way down through this conversation among Storm committers, and you learn interesting things about the trade-offs between CPU and network bandwidth:

@Gvain,

There are two parts in your question, In your test, why throughput drops when worker number increase after reaching a value(8 in your test case)?

For this one, it is because your CPU reach limit for worker# = 8 (CPU usage: 89%), In this case, adding more workers will just adding more threads and context switch, hurting performance. While for ...

November 2nd, 2014

Why not use asserts in Python?

I use asserts a great deal in Clojure. They partly take the place of unit tests. Apparently they are mostly unused in the world of Python, although some of the reasons listed here would be general to any language:

Several reasons come to mind…

It is not a primary function

Many programmers, lets not get bogged down by the rationale, disrespect anything which is not a direct participant in the program’s penultimate functionality. The assert statement is intended for debugging and testing, ...

November 1st, 2014

The surprisingly difficult task of data importing

The level of difficulty is a surprise to me:

Second, it was clear that reliable data loads would require deep support from the data pipeline. If we captured all the structure we needed, we could make Hadoop data loads fully automatic, so that no manual effort was expanded adding new data sources or handling schema changes—data would just magically appear in HDFS and Hive tables would automatically be generated for new data sources with the appropriate columns.

November 1st, 2014

What Apache Storm demands from you

I’ve been trying to figure out how much I need to do manually when working with Apache Storm. This part seems important:

What is Storm’s reliability API?

There’s two things you have to do as a user to benefit from Storm’s reliability capabilities. First, you need to tell Storm whenever you’re creating a new link in the tree of tuples. Second, you need to tell Storm when you have finished processing an individual tuple. By doing both these things, Storm can detect ...

October 30th, 2014

Why not use fun notation?

Luciano Ramalho gets some laughs when he introduces cartoon characters as a form of notation to describe what his code is doing:

But why not use this notation for real? He makes the point that his notation is more clear than UML.

When I was a kid I hated math. As I got older I realized I loved logic, and this made me a good computer programmer. When I look back, I realize that as a kid I hated math ...

October 30th, 2014

Single dispatch in Python

Andrew Montalenti describes the almost-multimethods that Python got in 3.4:

from functools import singledispatch

@singledispatch    def handle(obj):      pass

@handle.register(A)    def handle_A(obj):      pass

@handle.register(B)    def handle_B(obj):      pass

def main(obj):      # obj could be ...

October 28th, 2014

Sandboxing in Python

After many years programming in Ruby, PHP, Clojure, etc, I am only now getting into Python. One thing that strikes me is the extent to which each eco-system emphasizes different things. In theory, all of these languages are Turing-complete, and so you could do anything in any of these languages, but the reality is different: each community regards some issues as important, and other issues as not important, and so you end with tooling for different tasks, in each of ...

October 27th, 2014

How to track garbage collection on the JVM

I had a Clojure app with painfully long pauses. I thought maybe they were GC pauses, except they are too long. This is a good tip:

You can use

jstat -gcutil 2000

to print the GC statistics every 2 seconds, http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html

It the long pause is from GC, the columns FGCT/FGC values would be large.

If you think it’s a swap issue, you may want to use

vmstat 1 10000

watch out the si/so columns.

Source

October 27th, 2014

When is visionary-driven development good?

Interesting:

By controlling every detail of the project, I could ensure the project remained at a very high quality. Since I knew the project from top to bottom, I could anticipate all the ways any given change would affect the project as a whole. And since I had a vision of where the project should go, I could prevent any changes from going in that conflicted with that vision (or modify them to be consistent). I could ensure the project always ...

October 23rd, 2014

Does Python need strict data-type enforcement?

Andrew Montalenti has a long post voicing his view of data-type enforcement in Python. He seems to lean against strictness (I will quote him at length below). I am mostly ignorant about Python and I would not want to match my knowledge of Python against his, but I have seen this debate occur in other languages, so I’m going to write about my own experience adding types to dynamic languages.

For the last 2 years I’ve been working with Clojure, ...

October 23rd, 2014

Once upon a time, not that long ago, there was a clear division between “programmers” and “sysadmins”. Programmers wrote software and sysadmins made sure all the machines were running. But, as Marc Andreessen said, “software is eating the world.” Sysadmin work has become increasingly automated, and therefore sysadmins (at least the good ones) are increasingly forced to become programmers (or they become irrelevant). The job designation “devops” was invented to refer to the new kind of sysadmin, who has to ...

October 22nd, 2014

Best practices are sometimes the worst practices

A detailed look at Java memory management in Minecraft 1.8 blames the performance problems on so called “best practices”:

Minecraft 1.8 has so many performance problems that I just don’t know where to start with.

Maybe the biggest and the ugliest problem is the memory allocation. Currently the game allocates (and throws away immediately) 50 MB/sec when standing still and up to 200 MB/sec when moving. That is just crazy.

What happens when the game allocates 200 MB memory every second ...

October 22nd, 2014

The difference between brains and computers

Interesting:

Michael Jordan: I wouldn’t want to put labels on people and say that all computer scientists work one way, or all neuroscientists work another way. But it’s true that with neuroscience, it’s going to require decades or even hundreds of years to understand the deep principles. There is progress at the very lowest levels of neuroscience. But for issues of higher cognition—how we perceive, how we remember, how we act—we have no idea how neurons are storing information, how they ...

October 19th, 2014

Archis’s blog on the need for compromise in engineering designs

I like this very much:

Why do I equate it with religion? Because religion allows you to do anything, and always be right. Need to feed the hungry? Sure. Need to NOT feed the hungry? Sure. Support gays? Sure. Not support gays? Sure. Invade Jerusalem? Sure. Defend against invaders? Sure. Forgive people? Sure. Not forgive people? Sure.

This has always been my observation with “frameworks”, “patterns”, “best practices” and especially, “testing”. You can write a few hundred tests that do nothing, ...

October 19th, 2014

How much of a speed boost can we get from new programming languages?

“K” is an interesting experiment of minimalism and speed.

Kdb+ is a testament to the rewards available from finding the right abstractions. K programs routinely outperform hand-coded C. This is of course, impossible, as The Hitchhiker’s Guide to the Galaxy likes to say. K programs are interpreted into C. For every K program there is a C program with exactly the same performance. So how do K programs beat hand-coded C? As Whitney explained at the Royal Society in 2004, ...

October 19th, 2014

The need for empiricism in computer programming

Since computer programming is based on math, you would think that one could conceptualize a program just as one might conceptualize a math proof, working out everything in advance. But in fact, trial and error are necessary:

Any experienced engineer who tackles even a moderately complex coding problem and decides beforehand to think through that problem with a machine checked specification (say with Lamports TLA) will very likely discover that

1. His first 4, 5 attempts at the specification are not ...

October 19th, 2014

Maximum Sim City

Someone worked hard to figure out the best algorithm for Sim City

Source

October 18th, 2014

Docker is like MemCache, if you need it, then you need another programming language

Maybe this should suggest another route altogether?

Managing Cabal dependency hell. Most of our application development is in Haskell, and we’ve found we prefer specifying a Docker image than working with Cabal sandboxes. This is equally a gain on other programming platforms. You can replace virtualenv for Python and rvm for Ruby with Docker containers.

I agree that the use of Docker is being driven by the use of languages like Python and Ruby, but maybe the existence of Docker suggests ...

October 7th, 2014

Axilmar’s Blog responds to “Object Oriented Programing is an expensive disaster”

Axilmar’s Blog has a very strange post. I can not know if English is their native language, and I am sympathetic that some ideas get lost in translation. Still, they seem to have misread most of the article. On Hacker News they were asked how they extracted points from the article, and they wrote:

I just read it, skipping the non-essential parts, after quickly scanning them. I might have missed one or two points, but it doesn’t really matter.

But they frequently ...

October 7th, 2014

Phil Trelford shrinks 30,000 words to a useful 167

Phil Trelford did a very good job summarizing my 30,000 words with just 167 words.

Source

October 7th, 2014

Object Oriented Programming is an expensive disaster which must end

The No True Scotsman fallacy leads to arguments like this:

Person A: “No Scotsman ever steals.”

Person B: “I know of a Scotsman who stole.”

Person A: “No True Scotsman would ever steal.”

Person A is thus protected from the harmful effects of new information. New information is dangerous, as it might cause someone to change their mind. New information can be rendered safe simply by declaring it to be invalid. Person A believes ...

October 6th, 2014

Brilliant insights ruin your personal life

Sort of funny:

Once I figured out this algorithm, I knew I was onto something big. It massively simplified the design of the system by avoiding all the aforementioned problems, along with making things way more performant. (Amusingly, the day I figured out the algorithm I had a date with a girl I’d met recently. But I was so excited by what I’d just discovered that I was completely distracted the whole time. Needless to say, I did not do well ...

October 4th, 2014

Transducers in Javascript

Interesting:

function filterer(f) { return function(combine) { return function(result, x) { return f(x) ? combine(result, x) : result; } } }

arr.reduce( filterer(function(x) { return x > 2; })( mapper(function(x) { return x * 2; })(append) ), [] ); // -> [ 6, 8 ]

Now that we’ve decoupled the data that comes in, how it’s transformed, and what comes out, we have an insane amount of ...

October 1st, 2014

C++ wants Lisp macros

Apparently C++ templates are getting better.

Lots of languages have added enough meta-programming to imitate Lisp marcos. Paul Graham predicted this:

8. A notation for code using trees of symbols.

9. The whole language always available. There is no real distinction between read-time, compile-time, and runtime. You can compile or run code while reading, read or run code while compiling, and read or compile code at runtime.

Running code at read-time lets users reprogram Lisp’s syntax; running code at compile-time is the basis ...

October 1st, 2014

Every function in Haskell officially only takes one parameter

The purity of Haskell is, of course, very interesting:

Every function in Haskell officially only takes one parameter. So how is it possible that we defined and used several functions that take more than one parameter so far? Well, it’s a clever trick! All the functions that accepted several parameters so far have been curried functions. What does that mean? You’ll understand it best on an example. Let’s take our good friend, the max function. It looks like it takes ...

October 1st, 2014

Abstractions are slow

Interesting:

Trampolining/thunks/laziness is slow If you are unfamiliar, here is a detailed description of trampolines, and I’m going to steal his example.

Consider a corecursive pair of functions:

def odd1(n: Int): Boolean = { if (n == 0) false else even1(n – 1) } def even1(n: Int): Boolean = { if (n == 0) true else odd1(n – 1) }

If you try to call these for large n, you will rapidly get a stack overflow.

A trampoline is the following construct:

sealed trait Bounce[A] case class ...

September 28th, 2014

Queues are databases

Interesting:

We’ve been using RabbitMQ for messaging for a long time. Some messages are purely reactive: An object was changed, so some other subscribing application needs to react by updating its data structures to reflect the change. This is an event. Other messages represent jobs. Jobs are tasks that are queued up for processing. Processing photos, for example, is a classic job. Jobs don’t fall that well into the RabbitMQ/AMQP framework because it’s designed for messaging and is, despite support for things ...

September 23rd, 2014

We need a new protocol to replace HTTP and allow for software

” We do not control the environment executing our JavaScript code, interpreting our HTML, or applying our CSS. Our users control the device (and, thereby, its processor speed, RAM, etc.). Our users choose the operating system. Our users pick the browser and which version they use.”

The original idea of a “browser” was something that was as agnostic as possible about its environment, but somehow managed to deliver a nearly universal experience. The traditional idea of a “browser” does not ...

September 23rd, 2014

Treacherous unexpected pitfalls in Java

Very interesting, this is a mistake I could easily make:

After being quite puzzled for a while as to why this was happening finally I found the answer in Java 7 api docs for Process.

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.

They found this fix:

They remark:

This is so ...

September 23rd, 2014

A painful pause while my app is running

I used Clojure to build a RESTful API. I was successful in so far as that went, but now I face the issue that every once in a while, the program pauses, for a painfully long time — sometimes 30 seconds, which causes some requests to the API to timeout. We are still in testing, so there is no real load on the app, just the frontenders, writing Javascript and making Ajax calls to the service.

The service seems like ...

September 22nd, 2014

How to restart public JVM services

How do have zero-downtime restarts and upgrades? This is interesting:

We’re going to use Stuart Sierra’s excellent component project to manage the lifecycle of our service which, for the moment, will simply store a random number on initialisation and serve that back in response to any request. Getting Jetty to start on a random, operating system assigned, port is simply a matter of passing zero as the desired port number. If we then communicate this port number in some way ...

September 22nd, 2014

Async is state machines

I thought I linked to this before but now I can not find evidence of the earlier link:

The State Machines of core.async

It seems this idea has been around for decades but only during the last 2 years did the idea go mainstream. Python and Clojure and Javascript have all added libraries that allow async programming. All of these libraries use state machines to hide the reality of “callback hell”.

I’m going to use a pretty trivial function to examine ...

September 22nd, 2014

Transducers in Javascript

Transducers in Javascript.

This is an interesting sentence:

The reduce function is the base transformation; any other transformation can be expressed in terms of it (map, filter, etc).

All of the programming languages seem to be adding all of the same features. Over the last 2 years, suddenly people woke up and realized they could escape “callback hell” by using finite state machines to enable a pleasant API for async programming, core.async being a good Clojure example.

Now James Long adds transducers to ...

September 22nd, 2014

What is so hard about disequilibrium dynamics?

This is a throwaway line by Paul Krugman:

What are the alternatives? One — which took over much of macro — is to do intertemporal equilibrium all the way, with consumers making lifetime consumption plans, prices set with the future rationally expected, and so on. That’s DSGE — and I think Glasner and I agree that this hasn’t worked out too well. In fact, economists who never learned temporary-equiibrium-style modeling have had a strong tendency to reinvent pre-Keynesian fallacies (cough-Say’s ...

September 22nd, 2014

The problems with Object Oriented Programming are well known

This is a great summary of all of the arguments against Object Oriented Programming:

Equality

Let’s look at intensional equality as provided by object identity. Apparently this is not what we want at all times, otherwise we wouldn’t see this pattern called ValueObject 3. A type like Javas String is a well-known example of a successful ValueObject implementation. The price for this freedom of choice is a rule that the == operator is almost useless and we must use equals because ...

September 22nd, 2014

Why do people use inheritance in Javascript?

I like this:

Why do people use inheritance?

JavaScript makes inheritance a pain in the ass to implement, so why is it so popular among frameworks? Part of the problem is that JavaScript has always looked like a flimsy lightweight scripting language next to Java and its strongly typed kin; keen to prove that they are using a real language for big people, JavaScript developers have rushed to adopt an OO feature that was never very good in the first place. Strongly ...

September 22nd, 2014

Inheritance is evil

Interesting:

All of the pain caused by inheritance can be traced back to the fact that inheritance forces ‘is-a’ rather than ‘has-a’ relationships. If class R2Unit extends Droid, then a R2Unit is-a Droid. If class Jedi contains an instance variable of type Lightsabre, then a Jedi has-a Lightsabre.

The other kind of inheritance

By the way, my gripe is with concrete inheritance – one class deriving from another and inheriting behavior from the parent class. I have no problem with interface inheritance, where ...

September 20th, 2014

The Dark Age Of Emacs

I am curious why such a famous text editor is in such terrible shape, and so far behind industry best practices when it comes to stuff like package management. This seems like a good argument:

Code from the Dark Age of Emacs is kept in blog posts, hosted on EmacsWiki, stuck in some obscure directory on university domains, lost to the ether that is personal websites with expired hosting… Tracking down updated versions is nigh impossible, because they’re often ...

September 9th, 2014

How to avoid race conditions in Java

Interesting:

Giving a precise answer involves examining the complete Java program – depending on the context in which this snippet executes, the transformed snippet may or may not be equivalent to the original. For example, if there are no writes to volatileF (and so regV is always 0 and so is regNV) in the Java program then the transform is trivially safe. However, it is more interesting to be able to make local judgments when JIT compiling a single method. Specifically, ...

September 8th, 2014

The Shen language was shaped by illness and rejection

I have become interested in the Shen language. While reading the “history” page, I notice that illness and death play a large role, and also rejection of new ideas by audiences which misunderstand the speaker:

The appearance of Qi was swiftly followed by a serious illness that laid me up for 2006 and most of 2007. Following a partial recovery in 2008, a factorising version of the Qi compiler was introduced which made Qi competitive with the fastest hand-compiled Lisp ...

September 8th, 2014

I am thinking this might be a new business model for me to work toward:

As a reader, I seek an algorithm which weeds out some repetition. For instance I sometimes see a Vox.com article in my feed from three different sources — it would suffice to see it once, along with a color shading indicating that some other people in my feed were tweeting the same thing. I also would like blocks on tweets about the ...

September 4th, 2014

New discoveries with The Sims

I have never played the Sims game. I did play Sim City back in the 1990s. I was first introduced to the game in 1994 and lost a weekend discovering all I could about it. In my life I have probably played the game 100 times?

The Sims has a strong culture around it, which is something that only a handful of games have achieved. This whole bit was funny and interesting:

But for whatever reason, I’d just never played ...

September 4th, 2014

Tech stereotypes I did not know: devops more exciting than programming

Last night I was a table full of programmers and devops. One of the devops said “I’ve been thinking that I’d like to convert to programming.” Another devops said “You will be bored.” Everyone nodded in agreement. I was surprised by this. Is this some known conventional wisdom? I get the adrenaline rush of having to fix the servers when they have crashed and you are offline and the siren is going, but is that more interesting, over the long ...

September 3rd, 2014

How ignorant am I, and how do I formally specify that in my code?

For decades, computer programmers have argued with each other over the issue of strict data-type enforcement, versus dynamic data-type unenforcement. And after 16 years of computer programming, I have come down firmly in the middle: I like optional typing.

In computer programming, the static type advocates wear a perpetual sneer, claiming that many problems in programming would be solved if only we adopted their approach. And yet, many languages are statically typed, and software written with those languages continue to ...

September 2nd, 2014

Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo?

I really wish I understood this article. I need to commit to spending some serious time studying statistics, so I can catch up with the modern boom in data analysis. Because 90% of this article is over my head. But from what I can glean, it is very informative:

You’ll noticed that I glossed over something here: the prior, P(Ftrue). The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are ...

September 2nd, 2014

Transducers are coming to all the core functions of Clojure

Interesting, though I think this is only a convenience:

One of the parts of the announcement of transducers was that clojure’s core functions (map, filter, take, etc) that normally operate on sequences will gain a new arity that returns a transducer when called with a single argument that’s a function. The new code for the 1-arity map looks much like what we already wrote (ignore the other arities for now).

;; Source of the new arity of the ‘map’ function. (defn map ...

September 2nd, 2014

We can not estimate how long that software project will take

This is great:

Combining Independent Estimates Improves Estimation Accuracy

The average of effort estimates from different sources is likely to be more accurate than most individual effort estimates. A key assumption for accuracy improvement is that the estimates are independent, which is true if their sources differ in expertise, background, and estimation process. A Delphi-like estimation process, such as “Planning Poker,” where software developers show their independently derived estimates (their “poker” cards) at the same time, seems to be particularly useful in ...

August 30th, 2014

Parsers in Clojure

The title is “Parsing CSS file with monadic parser in Clojure” but there is a lot here that is educational regarding writing any kind of parser.

(defn parse [parser input] (parser input))

(defn parse-all [parser input] (->> input (parse parser) (filter #(= “” (second %))) ffirst))

But this will require some study:

;; build parser that always returns given element without ...

August 30th, 2014

Apache Commons-IO overwhelms me with options

I sure wish I understood what all this stuff does, because it looks like there are a lot of powerful options here: How do normal people discover the real world use cases of all this stuff?

Class Summary AutoCloseInputStream Proxy stream that closes and discards the underlying stream as soon as the end of input has been reached or when ...

August 29th, 2014

Ribol is a restart library for Clojure

I am impressed with the look of Ribol, though I have not yet used it:

ribol provides a conditional restart system. It can be thought of as an issue resolution system or try++/catch++. We use the term issues to differentiate them from exceptions. The difference is purely semantic: issues are managed whilst exceptions are caught. They all refer to abnormal program flow.

Restart systems are somewhat analogous to a management structure. A low level function will do work until it encounter an ...

August 29th, 2014

repl-friendly development workflow

First I saw this question in the Github issues for Sente:

Could you recommend a workflow that works across REPL re-evaluations? My current solution is wrapping the initialization in a defonce:

(defonce sente-init (do (let [{:keys [ch-recv send-fn ajax-post-fn ajax-get-or-ws-handshake-fn connected-uids]} (sente/make-channel-socket! {})] (def ...

August 29th, 2014

The 5 types of dependency injection in Clojure

Interesting:

Globally Shared

This is often the first way developers think to share data across an application: simply throw it in a def in a namespace and allow any function that needs it to reference it from there.

….It has the advantage of being simple to implement. The disadvantages are numerous and Dependency Injection was originally developed to overcome the shortcoming of globally shared data. Among other things, putting the context in a globally shared data structure will ...

August 29th, 2014

Thread binding in Clojure is tricky

Very interesting. Apparently you can not use pmap in this example:

user=> (def *foo* 5)

#’user/*foo*

user=> (defn adder [param] (+ *foo* param))

#’user/adder user=> (binding [*foo* 10] (doseq [v (pmap adder (repeat 3 5))] (println v))) 10 10 10 nil

*foo* is bound to 10 by “binding” but pmap spins up other threads, and *foo* ...

August 29th, 2014

Open Source still lives

Funny:

Source

August 27th, 2014

Is there a single answer to the problem of package management?

Interesting:

Why are there so many goddamn package managers? They sprawl across both operating systems (apt, yum, pacman, Homebrew) as well as for programming languages (Bundler, Cabal, Composer, CPAN, CRAN, CTAN, EasyInstall, Go Get, Maven, npm, NuGet, OPAM, PEAR, pip, RubyGems, etc etc etc). “It is a truth universally acknowledged that a programming language must be in want of a package manager.” What is the fatal attraction of package management that makes programming language after programming language jump off this cliff? ...

August 27th, 2014

The growing power and status of Computer Science departments

In academia, statistics is losing ground to computers:

“They [the statistical profession] lost the PR war because they never fought it.”

I assume this is a USA development. In Europe the computer departments have tended to be outgrowths of the math departments.

Recently a number of new terms have arisen, such as data science, Big Data, and analytics, and the popularity of the term machine learning has grown rapidly. To many of us, though, this is just “old wine in new ...

August 27th, 2014

Problems of package management are sapping productivity for tech workers

It is especially bad on the front end:

The situation with packages and dependency hell today is horrendous, particularly if you work in a highly dynamic environment like web development. I want to illustrate this with a detailed example of something I did just the other day, when I set up the structure for a new single page web application. Bear with me, this is leading up to the point at the end of this post. To build the front-end, I wanted ...

August 27th, 2014

A single div

This is an interesting use of CSS gradients to draw images

Source

August 27th, 2014

NixOS as package management?

The end should be the headline:

The last point is huge: I can use Nix to manage my software on both my Linux and OS X machines! I’ll explain how I do that in a future post.

The whole thing sounds interesting:

The main differentiator of NixOS is its package manager, Nix, which stores packages in isolation on a read-only file system. It then makes them available to you by adjusting your environment variables (e.g. your PATH). This way, it can achieve ...

August 26th, 2014

Why are Emacs packages such a disaster?

Check out this screenshot. Do you see where it says “Build/Failing”? The red icon? This simply does not happen with other open source projects that I use. No other community thinks it is normal to push broken code to master. What is wrong with Emacs packages? Is their an attitude that we hackers should be such amazing hackers that we can fix the broken code in every project that we use?

Source

August 23rd, 2014

“Just works” versus “I understand it”

This does a lot to explain the difference between folks drawn to PHP and folks drawn to Lisp. Do you want a utility language that allows you to get basic work done, or do you want a language that you can understand? I wrote PHP for years, but in the end, I was frustrated by much of the magic in it, especially in the PHP object oriented stuff. I like Clojure because I can understand the underpinnings of the language ...

August 21st, 2014

Representational Value Transfer (REVAT)

Very interesting:

Representational Value Transfer (REVAT) What we need is a RESTful way to disambiguate state and value, so that we can reason about resource URLs. I propose that we use use existing REST semantics for indicating state and adopt a new conventional standard for indicating values:

GET http://api.example.com/values/5690ba7f-f308-4c32-b67c-56f654bbfd83

{ “id”: 12345, “title”: “Apple iPad Air”, “price_usd”: 599.99 } The salient points of a REVAT URL are:

REVAT values are immutable. Values are identified by random UUIDs No coordination is required to uniquely generate them A non ...

August 21st, 2014

Racket sounds awesome

Very interesting:

I said above that Lisp flat­tery is easy to find. The prob­lem with Lisp flat­tery is that it makes sense only to ex­pe­ri­enced Lisp pro­gram­mers. To oth­ers—es­pe­cially those who are try­ing to de­cide whether to learn and use a Lisp—it just comes across as un­sub­stan­ti­ated hoodoo.

For ex­am­ple, in his es­say How to Be­come a Hacker, Eric Ray­mond says “Lisp is worth learn­ing for … the pro­found en­light­en­ment ex­pe­ri­ence you will have when you fi­nally get it. That ex­pe­ri­ence will ...

August 20th, 2014

People strongly disagree with me

Interesting. On Hacker News I quoted one of my earlier blog posts:

“There is an important asymmetry between an architecture of small apps and an architecture of The Monolithic CMS. If you have small apps, and decide you want to move to a monolithic CMS, then you must do The Big Rewrite: the exhausting effort of reproducing all of your funtionality so that it is handled by your one, all-consuming CMS. But when you move from the monolithic CMS to an ...

August 20th, 2014

The S does not stand for Simple

Interesting:

REST is a vast improvement over complex things like SOAP and CORBA, but I think we still have a way to go before we’ve reached simple. REST is an acronym for REpresentational State Transfer, and I think the “state” part of that acronym gives rise to a lot of incidental complexity as systems grow.

You can think of state as a combination of value and time, and in the RESTful case, the time dimension is almost always “now”. The trouble ...

August 19th, 2014

I am intrigued:

A Motivating Example

ACME Co. buys, sells, and processes things. Unfortunately, their circa-2003 web interface is not a shining example of UI/UX design. Befuddled by all the modal screens, managers regularly put bad data into the system.

In fact, manager Steve just accidentally created an inventory record showing that ACME now has 999 Tribbles. This is ridiculous, since everyone knows that the CEO refuses to deal in Tribbles, citing “a bad experience”. In a rather excited voice, ...

August 17th, 2014

Security problems with OpenSIS

OpenSIS tells me that a password is “invalid” because another user is already using it. This is in the admin view, so I guess the assumption is that the admin can know if a password is being used more than once, but this still strikes me a security violation. See the bottom of the screenshot:

Source

August 7th, 2014

Event Tracing for Windows is a truly terrible API

I found this to be funny:

Blog Bio The Technician No Imperfections Noted The Jeff and Casey Show Jeff and Casey TimeCasey Muratori Seattle, WA The Worst API Ever Made A call-by-call look at context switch logging with the Event Tracing for Windows API.This article is part of a series where a new short-form tech article is posted every Wednesday. You can always check the contents page for the latest installment, or follow me on Twitter for updates. You can also RSS ...

August 6th, 2014

The explosion of sysadmin configuration complexity

Somewhat off-topic, but it is amazing to think about the complexity of the server setup that is implied by this:

First, let me start by explaining why we decided to port away from Puppet: We had a complex puppet code base that has around 10,000 lines of actual Puppet code. This code was originally spaghetti-code oriented and in the past year or so was being converted to a new pattern that used Hiera and Puppet modules split up into services ...

August 5th, 2014

Clojure core.async is just a bunch of callbacks

Interesting to note that the beautifully elegant core.async is secretly the same as the “callback hell” of Javascript — except that core.async automates it all for you, which is a huge difference.

core.async is a Clojure library implementing communication sequential processes, an approach that allows code to be structured as producers and consumers of messages passed through channels. CSPs are an approach to dealing with concurrent activity in a program, and exists as a strong alternative to the kind of ...

August 5th, 2014

Just write SQL

I’ve been on both sides of this debate, but mostly I write SQL, mostly because the limits of ORMs tend to be immediate and stifling.

ORMs map nicely when you are indeed modifying objects, but somethings don’t map well that way. So don’t map them that way! What we need is a low level abstraction layer alongside the ORM. The main problem with raw SQL is that what you really want is a genuine programming language. You almost want programmatic access to ...

August 5th, 2014

Visual Markov Chains

This is well done, the animation of the Markov chains makes the real-world impact of the probabilities more understandable.

Source

August 5th, 2014

Multimethods in Clojure

I’ve only recently discovered multimethods, but now I’ve come round to the idea that one should never use (cond) in Clojure. Rather, any time you have a complicated nexted (ifs) or a big (cond) it should all be replaced by multimethods:

A Clojure multimethods is a combination of a dispatch function and one or more methods, each defining its own dispatch value. The dispatching function is called first, and returns a dispatch value. This value is then matched to the correct ...

August 5th, 2014

PHP will never be a happily multi-threaded language

If you want concurrency, then use a language that was designed from the beginning to support concurrency. PHP will never be that language. I’ve said this before, and jerf on Hacker News also says it well:

Adding pervasive threading is a different story. Adding threading to a mutable-state dynamic scripting language has a long and sordid history… even when it is nominally successful (as in Python) it is still not very useful, and at times it has been simply a failure ...

August 5th, 2014

Web apps can be desktop apps if you bundle the web server

It’s an interesting idea, a sort of uberjar idea, in the sense that you bundle all dependencies as one app. You can bundle up all the pieces of a web app and make it a desktop app by including the webserver. Indeed, I do exactly that every time I run “lein uberjar” on one of my Clojure apps (when I embed the Jetty server with my app):

Node is a lightweight JavaScript runtime based on the Google Chrome V8 engine. ...

August 1st, 2014

This is a very good write-up about Liberator, the Clojure library for RESTful APIs. This example he gives was a revelation to me:

Here’s a longer example, a post on a blogging site. Anyone can read a blog post (with a GET), but only the user who wrote it can edit it (with a PUT) or DELETE it.

(defresource blog-post-resource [id] :allowed-methods [:get :put :delete]

;; Return 503 Service Unavailable if DB is down. :service-available? (fn [_] ...

August 1st, 2014

Relying on “standard” libraries which may or may not exist on a server

A strange post from ItsMe:

Programs (like not only browsers) are using shared libraries (e.g. glibc provided by the OS). So the dependant library(s) isn’t/ aren’t used by the browser alone.

most unix based OSes have python installed as a standard library (next to perl).

(My RHEL for example depends heavily on python because most of the standard programs that are used are written in python)

So why should be almost 8 MB for a program that reads text be nothing?

Particularly when ...

August 1st, 2014

The evolution of RESTful APIs

A good post on the importance of the PATCH verb:. This is from the Rails community, but I think all communities have adopted this. Lord knows Rails did a great deal to shape modern ideas about APIs.

What is PATCH? The HTTP method PUT means resource creation or replacement at some given URL.

Think files, for example. If you upload a file to S3 at some URL, you want either to create the file at that URL or replace an existing file if ...

July 28th, 2014

When should a program interpret words?

This is very good:

Consider the following “program” in English prose:

Assume that your favorite color is red. Now imagine a balloon that is your favorite color. Paint a canvas the same color as the balloon. As English goes, that’s a fairly clear program with a fairly well-defined result. When I follow those instructions, at least, I will always produce a red canvas (assuming that I have a canvas and some red paint, but a potential lack of art supplies is not the ...

July 28th, 2014

Rate limiting middleware for Clojure Ring

It is impressive how much the Clojure eco-system is catching up with Ruby On Rails. The middleware available for Ring is now very extensive. I just discovered Rate limiting middleware for Clojure Ring. Seeing as I just built an API with Liberator, this middleware could be very useful.

Source

July 28th, 2014

Organizing Emacs

This looks like a good article on simple and practical configs for Emacs. I hope to get a new Mac soon, and then re-install Emacs, and then re-configure it based on all I know now. My initial setup of Emacs was confusing since I was overwhelmed by the complexity of Emacs, and I was in a hurry to get stuff done.

Source

July 28th, 2014

The Cambrian explosion of Javascript innovation

I am not working with Javascript, so part of me wants to ignore it, but, to be honest, the things now happening in the land of Javascript are amazing. Consider Racer:

Racer is a realtime model synchronization engine for Node.js. By leveraging ShareJS, multiple users can interact with the same data in realtime via Operational Transformation, a sophisticated conflict resolution algorithm that works in realtime and with offline clients. ShareJS also supports PubSub across multiple servers for horizontal scaling. Clients can ...

July 27th, 2014

Understanding Ring Middleware

I am very, very stupid. Despite the great post by Darren Holloway, I was still wondering when you get the request and when do you get the response in a Ring Middleware.

Ryan Evans offers this simple middleware as an example:

(defn my-middleware [app] (fn [request] ;; This is where you’d do any processing on the request ;; Finally, keep the chain going by calling app (app request)))

The ...

July 26th, 2014

TJ Holowaychuk leaves NodeJS for Go

TJ Holowaychuk built out some of the most important nmp modules for NodeJS, but now he is leaving for Go.

Go versus Node

If you’re doing distributed work then you’ll find Go’s expressive concurrency primitives very helpful. We could achieve similar things in Node with generators, but in my opinion generators will only ever get us half way there. Without separate stacks error handling & reporting will be mediocre at best. I also don’t want to wait 3 years for the ...

July 26th, 2014

More than most projects, it seems like Node.js has seen a lot of churn in its leadership.

January 2012:

Citing a desire to work on research projects after three years of focused work, Node.js creator and project leader Ryan Dahl sent out a message today that he will be “ceding my position as gatekeeper to Isaac Schlueter”. He stated:

I am still an employee at Joyent and will advise from the sidelines but I won’t be involved in the day-to-day bug fixes. ...

July 26th, 2014

Adding real continuous loop behavior to Clojure apps

Interesting:

A while loop that is always true will continue to run until terminated, but it’s not really the cleanest way to obtain the result as it doesn’t allow for a clean shutdown. We can use a scheduled thread pool that will start and execute the desired command in a similar fashion as the while loop, but with a much greater level of control. Create a file in the src directory called scheduler.clj and enter the following code:

(ns pinger.scheduler ...

July 22nd, 2014

The problems of object oriented programming and strict typing

This is good, though too specific to Java:

The biggest problem I’ve encountered over the years looking at Java code is that it always seems to be the product of someone who fancies themselves as an architect. They must, because so often I find I’m reading code that looks more like a plan for something that solves a problem, rather than something that actually solves a problem. It’s not a subtle distinction. There are deep layers of abstraction and mountains of ...

July 22nd, 2014

How much can a developer possibly know?

The amount that programmers need to know is growing, so experienced programmers end up facing situations like the one described here by Tim Bray:

Where I’m stuck · I have a tab open to a page in the Gra­dle doc­s: Chap­ter 50. Depen­den­cy Man­age­ment. It has 63 header-delimited sec­tions or­ga­nized in­to 10 top-level sub­sec­tion­s, and it’s chap­ter 50 of 65 (plus five ap­pen­dices). ¶ Short sto­ry: I’m get­ting an in­com­pre­hen­si­ble Groovy er­ror try­ing to do some­thing that should be sim­ple, and fol­low­ing ...

July 21st, 2014

Pretty-print JSON from Clojure

Interesting:

What’s next? Oh, pretty-printing. Yeah, I pretty-print my JSON to go over the wire. It’s nice for debugging. I mean, who wants to curl one long, 1000-character line of JSON? Put some whitespace, please! How to do that?

(cheshire.core/generate-string mp {:pretty true}) That’s right, it’s basically built in, but you have to specify it. But, oh man, that’s long. I don’t want to type that, especially because my lazy fingers are going to not do it one time, then I’m going ...

July 21st, 2014

PrettyPrinting for test results

An interesting test style:

If the expression passed to is an S-expr, and the first element of the is recognized as a function. Then is prints that first symbol directly, then evaluates all the arguments to the function and prints the results. For instance:

expected: (function-name (arg1) (arg2)) actual: (not (function-name “1st arg value” “2nd arg value”))

However, if is does not recognize that first element as a function, the whole expression passed to is is evaluated for the actual, and you get:

expected: (something-that-evaluates-to-bool ...

July 21st, 2014

Why is uberjar such a rare option?

I find this very surprising. My first serious exposure to the JVM was via Clojure, which has the awesome Leinengen build tool, which has an uberjar option. Therefore I thought uberjar was common on the JVM. But no. Now that I am working with Java, I find that it is rare for anyone to put jars inside of jars:

You can add jars to the jar’s classpath, but they must be co-located, not contained in the main jar.

That was in 2008, ...

July 20th, 2014

Buildr as a continuous integration tool?

Interesting to read an old post from 2008 in which Liz Douglas stretch Buildr to the point that it becomes almost a continuous integration tool:

A few months ago the idea of myself writing such words (“Things I like about Buildr“) seemed very unlikely and I dare say that my project buddies may be surprised at the statement. Buildr, for those unfamiliar, is a build tool for Java applications that is written in Ruby. It’s key benefit is its concise ...

July 20th, 2014

Should project packaging/building be complicated?

Working with Java for the first time in 10 years. Using Buildr as my build tool. I have a certain admiration for Buildr and all that it allows:

Here’s another example:

jjtree = jjtree(_(‘src/main/jjtree’), :in_package=>’com.acme’) compile.from javacc(jjtree, :in_package=>’com.acme’), jjtree

This time, the variable jjtree is a file task that reads a JJTree source file from the src/main/jjtree directory, and generates additional source files in the target/generated/jjtree directory. The second line creates another file task that takes those source files, runs JavaCC on them, and ...

July 20th, 2014

Clojure encourages small functions

This is very true:

I have only two small Clojure projects, and other than writing tools on which nothing major depends, these projects will probably be the only ones in my current position. The rest will be done in Web languages and Perl. So, I enjoy a chance to enhance the Clojure projects.

I do not know why, and am not aware of any conscious prejudice, but writing Clojure code encourages me to create small functions, and external Clojure projects, like ...

July 19th, 2014

The difference between strict static typing and design-by-contract

When I write Clojure, I write pre and post assertions, following the pattern known as “design by contract”:

(defn fetch [ctx] {:pre [ (map? ctx) (string? (:database-query-to-call ctx)) (map? (:database-where-clause-map ctx)) ] :post [(future? %)]} “2014-07-01 – first we check the cache. If we get ...

July 18th, 2014

Classpath hell in Java

I rarely work in Java, and every time I recall what a pain it is to figure out the classpath. I finally set my classpath in the manifest to something hardcoded:

Main-Class: com/company/Main Class-Path: /Users/lkrubner/projects/launchopen/lofdg/target/

and at the terminal I compile my “.class” files in the same directory as the “.java” files:

javac src/com/company/*.java

Then I move them:

mv src/com/company/*.class target/com/company/

Then I create my jar file:

jar vcfm fakeDataGeneratror.jar manifest.txt target/com/company/*

which I call:

java -jar fakeDataGeneratror.jar

There are some important text files that ...

July 2nd, 2014

Testing RESTful APIs with cURL

This is great:

I believe it’s just as easy, for basic operations, to use CURL as you are likely to be in a terminal window anyway at such an early stage in development. For this reason, I thought I’d cover using CURL for the 4 basic RESTful methods (GET, POST, PUT and DELETE).

The following assumes you already have an application with a RESTful endpoint of ‘users’:

GET – This will get all users in our application.

1 curl http://www.mydemoapp.com/users POST – Here we are posting ...

July 2nd, 2014

Why would a sane programmer use PHP rather than Clojure for a RESTful API?

This question on StackOverflow seems a bit sad:

I did look at both Laravel, Sympfony2 and Codeigniter for this REST Api. They all had some elements I liked and some I disliked. My main concern was how to do the authentication because I had a rather complex algorithm where my users can log in with the apps’ access_token or access_tokens served by Google or Facebook. I also perfer being in complete control of my framework and the frameworks mentioned above had ...

June 30th, 2014

mongodump and mongorestore

This is a good overview for using mongodump and mongorestore:

Once you’ve taken the backup of a MongoDB database using mongodump, you can restore it using mongorestore command. In case of an disaster where you lost your mongoDB database, you can use this command to restore the database. Or, you can just use this command to restore the database on a different server for testing purpose.

1. Restore All Database without Mongod Instance

If you’ve taken a backup without mongod instance, use this ...

June 29th, 2014

Waste my screen like it’s 1996

A lot of the screen space here is fixed and does not scroll. Reminds me of frames circa 1996.

Source

June 28th, 2014

A simple regex rule

Don’t ever use this:

.*

.*?

Why? Because the first one is greedy and will almost always match too much.

This is a nice example:

Most people new to regular expressions will attempt to use . They will be surprised when they test it on a string like This is a first test. You might expect the regex to match and when continuing after that match, .

But it does not. The regex will match first. Obviously not what we ...

June 28th, 2014

Unix and C are the ultimate computer viruses

This is a very good take on “worse is better”. This also bears on the micro-services debate. Simple implementation allowed Unix to become the world’s favorite operating system. Is there redundant code in a Unix distro? Sure, you’ve got a lot of utilities that all have code for reading files. But if, instead of a bunch of small utilities, you tried to build the one ultimate tool that does everything (the monolithic framework) you would never get anything like the ...

June 27th, 2014

The cost of small apps

Micro services become popular:

Though they aren’t a particularly new idea, Microservices seem to have exploded in popularity this year, with articles, conference tracks, and Twitter streams waxing lyrical about the benefits of building software systems in this style.

I previously wrote about my preference for an architecture of small apps. But it is worth noting, this approach has its downside:

Where a monolithic application might have been deployed to a small application server cluster, you now have tens of separate services ...

June 27th, 2014

Maybe Unicorn works for Ruby

Since I said so many bad things about Unicorn, it is only fair that I also link to this piece that makes Unicorn sound good:

Unicorn was faster than Passenger or Thin with /borat and had the second highest transaction rate for the same. It had the shortest duration of both longest and shortest transaction with /borat as well. It was one of only two that actually finished the /pi test, and did so faster than Thin. Unicorn had the highest ...

June 26th, 2014

Regular expressions: greedy and lazy matching

This is a great tutorial on regular expressions:

As you’ve seen, a greedy quantifier will try to match as much as it possibly can and only give back matched characters as needed. Every time the engine greedily consumes one more character (or repeated token in general), it has to remember that it made that choice. It will therefore persist its current state and store it so it can come back to it later in a process we call backtracking. When ...

June 24th, 2014

Is the FizzBuzz test really hard?

I have been a fan of John Lawrence Aspden for several years now, but this post on FizzBuzz is especially good, both funny and illustrative of what I think is a common work process in Clojure:

;; I decided to use pull it out your ass driven development, where ;; you just pull the answer out of your ass.

;; First bit, print out the numbers from 1 to 100 (range 100) ;-> (0 1 2 3 4 5 6 7 8 9 10 ...

June 24th, 2014

Interesting:

Using ZFS as a replacement of Git for is probably not a good idea, but just to give you a sense of what ZFS supports at the file system level, let me go through a few typical git-like operations:

Creating a repository

Committing or tagging a version

Branching

Pushing and pulling changes from other storage pools, possibly on other machines

Notably missing is support for merging, which ZFS does not have direct support for as far as I’m aware.

Creating a repository

First, let’s create a filesystem ...

June 24th, 2014

Managing multiple Future objects in PHP

I have previously said negative things about Joe Watkin’s attempts to facillitate using objects and multiple threads in PHP.

However, here is an approach to Futures in PHP that does seem easy and interesting to me:

Managing Multiple Futures

Commonly, you may have many similar tasks you wish to parallelize: instead of compressing one file, you want to compress several files. You can use the FutureIterator class to manage multiple futures, via the convenience function Futures().

$futures = array(); foreach ($files as $file) ... Read More Source June 24th, 2014 No Comments Refactoring fat models I had a job interview at Bookspan.com. At first I talked to a guy named Tom. I assumed he was the leader of the tech team, so I talked about my wide experience. He seemed confused by my recent experience with Ruby. He asked if I was a serious PHP programmer? I said a few negative things about PHP, since most corporations are now pulling away from it. PHP is going out of fashion (as you can see in this ... Read More Source June 24th, 2014 No Comments Eventual consistency is probably the only consistency that one can hope for using Javascript and WebSockets On the one hand, I am very impressed with this article: Eventual Consistency in Real-time Web Apps. On the other hand, how can anyone keep up with what is best practice in the land of Javascript, when every week seems to bring a new framework or methodology? Having said that, I’ll point out that there is no way to ensure a 1-to-1 match between one’s backend model and one’s front-end model, so all one can do is pick one to ... Read More Source June 24th, 2014 No Comments PHP-FPM with Nginx I already linked to this, but I will link again because, seriously, this is one of the best setup tutorials I’ve seen: One of the greatest strengths of PHP-FPM is its ability to scale its worker processes up and down as load on the server increases. PHP-FPM can have several “pools” of PHP handlers: one for each different Web application, with different numbers of worker processes and different rules about when to add more processes or kill idle processes. Our needs ... Read More Source June 24th, 2014 No Comments Docker is the future Something like Docker is probably the future. Clearly, virtual machines is becoming a popular way to manage dependencies. However, Docker still has problems: Misconception: If I use Docker then I don’t need a configuration management (CM) tool! This is partially true. You may not need the configuration management as much for your servers with Docker, but you absolutely need an orchestration tool in order to provision, deploy, and manage your servers with Docker running on them. This is where a tool like ... Read More Source June 17th, 2014 No Comments Emacs struggles to get a good package manager Kind of pathetic that the world’s best text editor only got package-support in 2012: GNU Emacs 24 (released in June 2012) introduced official support for packages, that is, a way of installing extensions from a remote repository. This was a huge step forward for Emacs, as it not only allowed users to easily find and install extensions, but it also made it possible for extensions to build upon other extensions without having to tell the user “great you want to install ... Read More Source June 3rd, 2014 No Comments The strange way my memory works I was at a job interview today and I said “Do you remember that essay that James Garret wrote in 2004, in which he coined the acronym ‘AJAX’ “? Feeling uncertain about what I said, when I got home I decided to check my facts, and found that his name is actually “Jesse James Garrett“. I find it strange that my memory would hang onto his middle name rather than his first name. I am also surprised that the term “AJAX” ... Read More Source June 1st, 2014 No Comments Avoid overcrowding your web server with too many unneeded processes This is some very good advice (I’ve been thinking of using Arch Linux for future projects, as I understand it is an extremely minimalist Linux): Avoid overcrowding your web server with too many unneeded processes. For example, if your server is purely for web serving, avoid running (or even installing) X-Windows on the machine. On Windows, avoid running Microsoft Find Fast (part of Office) and 3-dimensional screen savers that result in 100% CPU utilization. Some of the programs that you can consider ... Read More Source June 1st, 2014 No Comments Using Apache Flood to test websites I only just learned about Flood. In the past I used ab, which is a severely limited tool, in that it only sends HEAD requests. What I have often wanted is a tool that was as simple as ab, but which could send parameters, make a GET request, and give me more feedback than CURL. The XML config for flood suggests it is nowhere as easy to use as ab, but it still looks fairly simple and it looks like ... Read More Source June 1st, 2014 No Comments How does PHP work I like this graph about PHP execution. The white boxes show how things worked as long ago as PHP4 and the gray boxes show all the new and optional stuff: Source June 1st, 2014 No Comments PHP and Squid This is interesting: Perhaps the most significant change to PHP performance I have experienced since I first wrote this article is my use of Squid, a web accelerator that is able to take over the management of all static http files from Apache. You may be surprised to find that the overhead of using Apache to serve both dynamic PHP and static images, javascript, css, html is extremely high. From my experience, 40-50% of our Apache CPU utilisation is in ... Read More Source June 1st, 2014 No Comments Threads in a Unix process While doing research for the post I just wrote about Joe Watkins and threads in PHP, I came across this graphic, which I thought did a nice job of showing threads in a Unix process: Source June 1st, 2014 No Comments Setup is hard, and it wastes an extraordinary amount of developer time I was just looking at this article, PHP with PHP-FPM, and it brought back memories of Timeout.com. All of the developers who worked at Timeout eventually had to set up the company CMS, and this was something of a hazing process, in that every developer later remembered the experience with dread. The CMS was a massive Symfony project — without question, the largest PHP code base I have encountered in 14 years of working with PHP. I’ll give you a ... Read More Source May 31st, 2014 No Comments Using Xdebug to profile PHP Interesting overview of using Xdebug: I’ve sorted the execution time of each call in order to determine which calls are the most expensive. The call to the Default_Model_Platform model’s hot() method ranks up towards the top, and because I know this data changes only every few hours, now I can safely cache it and thereby eliminate this expensive database query (which is indeed a fairly large JOIN operation). After implementing caching I again profile the page and indeed have eliminated that ... Read More Source May 31st, 2014 No Comments Do not ever use MemCache 6 tips about PHP performance: #1. Upgrade Your PHP Distribution #2. Use a Profiler #3. Tone Down Error Reporting #4. Take Advantage of PHP’s Native Extensions #5. Use a PHP Accelerator #6. Avoid Expensive Operations Through Memory Caching Missing from the list is “Ask yourself if PHP is the correct language for what you are trying to do.” I think of this especially in regard to the 6th tip, about memory caching. If you start using something like MemCache, then you ... Read More Source May 31st, 2014 No Comments Why Joe Watkins is wrong about pthreads in PHP Over at Reddit Joe Watkins wrote about pthreads in PHP. Someone asks: Is there a facility to use thread-local storage? and Joe Watkins replies: The static scope of a class entry can be considered thread local, in a way. Complex members (objects and resources) are nullified when creating new threads, but simple members (arrays/strings/numbers/mixture of any of the above) are copied, so in the static scope can be class::$config which contains connection info to whatever and class::$connection can be the connection itself, ... Read More Source May 31st, 2014 No Comments In PHP, the foreach() loop is a performance murderer I didn’t know this: Source May 31st, 2014 No Comments Building a recommendation engine for one’s customers This is a great overview of building a prediction engine. Some of the math is over my head. Of these approaches, the only one I have the slightest familiarity with is cosine similarity. Source May 4th, 2014 No Comments The further rise of polyglot programming Interesting: Almost every language shows a long-term downhill trend. With the exception of Java and (recently) CSS, all of these languages have been decreasing. This was a bit of a puzzler and made me wonder more about the fragmentation of languages over time, which I’ll explore later in this post as well as future posts. My initial guess is that users of languages below the top 12 are growing in share to counterbalance the decreases here. It’s also possible that ... Read More Source May 2nd, 2014 No Comments Design by contract in Clojure I posted a comment here: I have gotten in the habit of doing 2 things: 1.) I used :pre and :post conditions as you are doing here 2.) I also use dire so when the :pre or :post conditions fail and an Assert exception is thrown, I can capture the arguments and the return value and write a meaningful error message: https://github.com/MichaelDrogalis/dire I do a lot of this: :post [(:discount %)] I also test for value ranges: :post [ (> (:totals %) 100) (< (:totals %) 1000) ] I am thinking I might ... Read More Source April 21st, 2014 No Comments What is the correct way to handle errors when making HTTP calls? I like this: I wanted a simpler solution that: treated exceptions as exceptions was general enough to leverage clj-http exception gave informative error messages in the right place A quick side note on exceptions vs happy paths everywhere. Some will say a bad response isn’t an exception, but is something to be expected. I agree. This is something that should be handled at the app level though, and not the library level – if I get a 401 as an end user I expect to ... Read More Source April 21st, 2014 No Comments The problem with frameworks This is funny: Let’s pretend I’ve decided to build a spice rack. I’ve done small woodworking projects before, and I think I have a pretty good idea of what I need: some wood and a few basic tools: a tape measure, a saw, a level, and a hammer. If I were going to build a whole house, rather than just a spice rack, I’d still need a tape measure, a saw, a level, and a hammer (among other things). So I go to ... Read More Source April 21st, 2014 No Comments The system at Twitter This looks like a great talk about the system at Twitter: Late 2012 architecture Many open source components Memcache, redis, MySQL, etc. Necessarily heterogeneous Organized around services Distinct responsibilities Isolated from each other Distributed computation and data RPC between systems Multiplexing HTTP frontend Crucial for modularity, load balancing Programming the datacenter Concerns include Partial failures Deep memory hierarchies Split heaps Dynamic topologies Changes in variance, latency tails Heterogeneous components Operator errors Taming the resulting complexity is the central theme of our work. Source April 21st, 2014 No Comments Clojure versus Erlang WhatsApp was recently bought by Facebook, for$19 billion. This is the first time a startup had a big success, using Erlang as the basis for all of its technology.

The most stable commercial computing device in history is a telephone switch developed at Ericson and built with Erlang. It can handle millions of simultaneous connections, it has 1.7 million lines of code, and it averages 1 hour of downtime every 20 years.

The one language I would like to learn, other ...

April 21st, 2014

Fast setup with Vagrant and Ruby 2.0

This is a good overview:

How to do it:

download and install Vagrant – I use version 1.2.7 You can check that with:

vagrant -v

You should see:

Vagrant version 1.2.7

create a folder for your Rails application and go to it

mkdir rorapp cd rorapp

vagrant init precise32 http://files.vagrantup.com/precise32.box

This is your virtual machine (server) that will hold and run your Rails application. If you check now, Vagrant created a config file in your rorapp folder, called Vagrantfile. You’ll change that a little on next step. make ...

April 21st, 2014

Formal proofs for software: prove small theorems, not grand ones

Interesting:

For “you can’t prove anything big enough to be useful!”, consider the Quark project:

http://goto.ucsd.edu/quark/

showed you don’t need to prove a program of interesting size. You can defend millions of lines of buggy code with a “software firewall” made of formally verified code. Verify the right thousand lines of code that the rest needs to use to talk to anything else, and you have very strong security properties for the rest of the code. seL4 and CompCert are clearly also quite useful programs.

… I don’t think the technology ...

April 18th, 2014

Kill your process and restart it is still a popular hack in Ruby land

I find it surprising that this is still accepted as best practice in the Ruby community:

But it’s not all bad news. The Ruby core developers are aware of the problem, and there are some changes (3GenGC, oldgen space estimation) being tested which may bring relief. But right now, for users of Ruby 2.1, this is a very real problem that could easily affect you in production. What did we do? We used a combination of Unicorn::OobGC and unicorn-worker-killer to help tame ...

April 18th, 2014

Vagrant is easy

This page offers a simple explanation of how to start using Vagrant:

Getting Vagrant started

Before you can run vagrant, you’ll need to download and install a few things:

The Vagrant tool – This is the actual vagrant tool itself. It manages virtual machines Virtualbox – Virtualbox is the virtual machine where your code will run If you have an app that already has a Vagrantfile in it, it’s very easy to get started. At the command line change into the folder with the ...

April 18th, 2014

What is devops?

A devop is like a sysadmin, so why do we need a new word? Because of the new emphasis on automation:

A DevOps person isn’t someone who develops, and who does Ops. It’s someone who does only Ops, but through Development.

The last time I looked for a senior sysadmin — less than a year ago — I didn’t get anyone who was comfortable programming in Perl/Python/Ruby until I started using the term DevOps.

There are companies where the developers are ...

April 8th, 2014

Forked processes, concurrency, and memory problems

Last week I expressed my doubts about Unicorn (and the idea that it uses processes, therefore it uses Unix, therefore it must be good). Here is another article that looks at Unicorn, and in particular the memory consumption that goes along with forked processes:

Unicorn uses forked processes to achieve concurrency. Since forked processes are essentially copies of each other, this means that the Rails application need not be thread safe.

This is great because it is difficult to ensure that ...

April 7th, 2014

Yahoo has some very stupid programmers

Good lord, why is this developer at Yahoo so slow on the uptake?

Thank you for your submission to Yahoo! Unfortunately we are unable to reproduce the bug due to insufficient information. Please provide us with a proof of concept or any other additional evidence required to reproduce the issue.

** The attacker would have to know the invitation id correct?

One has the sense that the person reporting the bug is shocked by the lack of concern shown by Yahoo:

d4d1a179c0f3 changed ...

April 7th, 2014

The Clojure workflow still suffers and the REPL is not a cure all

Stuff like this happens to me:

Here is a scenario that you might recognize. You’ve done a pretty substantial refactor, including new dependencies in project.clj. You need to bounce the REPL. Knowing that this will take forever you immediately switch to Prismatic. 15 minutes later you look at your Emacs again where you notice that there is a syntax error so the REPL didn’t launch. You parse the impossibly long stack trace and fix the bug. cider-jack-in again and switch back ...

April 6th, 2014

Behavior driven development is broken

This is very good:

If it takes you ten lines to communicate the idea of adding subpages, then you’ve wasted my time. I’m not alone in thinking this. BDD expert Elizabeth Keogh tells us:

“If your scenario starts with ‘When the user enters ‘Smurf’ into ‘Search’ text box…’ then that’s far too low-level. However, even “When the user adds ‘Smurf’ to his basket, then goes to the checkout, then pays for the goods” is also too low-level. You’re looking for something ...

April 5th, 2014

Why has Linux not seen more forks?

Strong language, and strong opinions, as always, from Linus Torvalds. Now that I think about it, isn’t it amazing that Linux remains stable, even after all these years. I remember someone predicting, years ago, that Linux would split apart into a million useless forks, just like Unix did a long time ago. But that never happened. There are a lot of distros, but the kernel remains 100% under the control of Torvalds. That must mean people trust him. And ...

April 4th, 2014

Using Gloss to change bytes into Clojure data structures

Interesting:

I started creating a very simple protocol to allow clients to connect via telnet. So it is:

PUT LSA |*

We have two main commands, PUT and LSA. For PUT, author is the guy speaking, via is who noted it, and the fact is the statement itself. And for LSA command, you can pass the author’s name and the system will return all the facts spoken by the author. * means you want to read all the facts.

Any other command ...

April 4th, 2014

Hashmaps versus btrees

Interesting:

Unsuprisingly, a hash map performs far and above the rest. This is to be expected, mapping is exactly what hash maps are for and, in most situations, they should perform insertions and lookups with amortized O(1) time complexity. However, for situations where you made wish to preserve order, a tree may be a better choice. For that, you can see that a well-tuned btree was outperforming a red/black tree by more than 2 times.

As memory architectures begin to behave more ...

April 4th, 2014

Just Libraries – the composition of small apps

Clojure favors the composition of small apps. The Clojure community has shown a resistance to monolithic frameworks like Rails. Now Immutant is moving further down the small app road.

For its second major release, Immutant will simply be a collection of libraries, one for each of the commodity services currently available to applications using an Immutant 1.x container: web, scheduling, messaging, caching, and transactions. These services will be provided by the following underlying components: Undertow, Quartz, HornetQ, Infinispan, and Narayana, ...

March 31st, 2014

The worst web site ever: healthcare.gov

What an incredible disaster. I say this as a professional who develops websites. Several states, such as Kentucky, built their own web sites, which have worked great. But the Federal site, even 6 months after launch, remains a disaster.

This is the error message I got when I just now tried to sign up:

Today’s the last day to sign up for Obamacare if you’re planning on using the healthcare.gov website. Unfortunately for people who tried to log onto the ...

March 31st, 2014

Java 8 has an Optional to deal with NullPointerException

I don’t think I am impressed with this. The idea is borrowed from Scala. I have no love for Java or Scala, and I only follow Java because it impacts Clojure. If this enables Clojure to do something clever with NullPointerException, then maybe I will reevaluate this.

Source

March 31st, 2014

Content Security Policy and Ruby and Clojure

Although I love Clojure, I must admit that Ruby and Rails have an impressive depth of gems to help with every aspect of web development, including security. John P Hackworth recently wrote of the weakness of the Clojure eco-system, although his criticism is also an attack on the whole of idea of “small libraries that compose well” which amounts to an attack on the idea of “small pieces, loosely joined”. Clearly, good security can be achieved with small libraries that ...

March 31st, 2014

Using Clojure to build a microservices CMS

Many of us become cynical about technologies that promise big breakthroughs in productivity, so we become overly careful in our choices, but this is a good question for managers to always be asking:

“Why would a large organisation with a mix of technologies and legacy systems want to muddy the waters with a completely new language?”

If you want to make the conservative choice, and stick with what you already have, you should be able to articulate the reasons as clearly as ...

March 30th, 2014

eat food for food in foods when food isnt ‘chocolate’

Of the many attempts to re-invent Javascript, the mostly puzzling to me are those that do not fix any problems, and then invents some more. I realize there is a strong desire to borrow ideas from Ruby and bring them to Javascript, but where one can’t do that cleanly, one shouldn’t do it at all.

It’s a tool, that is all. Ambiguous code is a poorly thought out contrived example with a simple solution. To me, this: eat food for ...

March 30th, 2014

Chris Granger: more problems with object oriented programming

At this point the evidence against object oriented programming seems overwhelming. I’ve linked to many articles here on this blog. Chris Granger offers another take on this issue:

Programming is unobservable

We can’t see how our programs execute. We can’t see how our changes affect our programs. And we can’t see how our programs are connected together. That basically means we can’t observe anything. The state of the art in observability is a stepwise debugger, which forces us to stop the world ...

March 30th, 2014

NoSQL is a new of doing things, not a drop-in replacement for SQL

I like this:

Both NoSQL and Erlang had a burst of use and interest but because they were seen as silver bullets. Soon people realized you couldn’t simply translate your imperative code to Erlang and see improvements but instead regressions. Additionally, throwing your relational data at a NoSQL databases caused the same.

I feel the NoSQL culture and programmers haven’t retracted to the core yet as much as Erlang. Though Erlang may see another surge of misuse and misinterpretation now with the ...

March 27th, 2014

The advantages of Ruby on Unicorn

This is an interesting way to look at things. Since so much of Ruby code is not thread safe, the fact that Unicorn spins up processes that don’t talk to each other is the most safe way to get concurrency in Ruby. That is a good point, though it is equivalent to saying “Since the code is broken, the the application server to do something weird to compensate for the brokenness.” Clearly, some people have good results with this, though ...

March 27th, 2014

Rails lacks a story for concurrency. This is written by a Go programmer. Their criticisms are similar to mine, though for me the answer is “use Clojure” and so I end up doing JVM tuning, which is brought up as something scary to keep people away from jRuby. My impression is that the case against jRuby is weaker than the case against MRI Ruby (the C version).

Rails is fundamentally – and catastrophically – slow.

This well-known set of webapp ...

March 25th, 2014

If Unix is good for Unicorn, why can’t Unicorn handle slow connections?

In what I now think of as a famous essay, Ryan Tomayko said “I like Unicorn because it’s Unix“. There must be something to this because the essay has been widely quoted, and I remember it, and I have re-read 3 times in the last 4 years. It had an impact.

And yet, nothing in it convinced me to adopt that model. I rejected it and went ...

March 24th, 2014

Photon could save PHP

I have been extremely critical of PHP for the last 2 years. See “Why PHP is obsolete“. However, I just stumbled across Photon, which seems to address some of the core problems I see with PHP (especially the lack of any tools for dealing with concurrency):

Why targeting Mongrel2?

Mongrel2 is a very well designed, high performance server developed by pragmatic users who do not like bloated software. The use of ZeroMQ as the communication hub makes it extremely flexible while keeping ...

March 24th, 2014

The tremendous innovation in Javascript

There is no question that tremendous innovation is happening in the vast extended eco-system that touches upon Javascript. Sadly, I am not much interested in it. Maybe that is because I am not focused on the frontend right now. But also because I’m interested in solving these issues in other ways. All the same, Sam Ruby’s walkthrough of Angular.js is interesting:

We have a model, view, and controller on the client, seemlessly interacting with the model, view, and controller on ...

March 24th, 2014

Radical workarounds for the limits of MongoDB

Whoa. This gives me interesting ideas:

To reduce lock contention, we decided to run multiple MongoDB instances on one machine and create more granular databases in each instance. Basically data is stored in different instances based on its usage and in every MongoDB instance one database is created for each partner.

Some people hate the fact that MongoDB forces you to do more in your own app, but I prefer designing with those constraints in mind.

This has similarities to ...

March 24th, 2014

I feel like I’ve been away from HTML for awhile. 10 years ago I thought of myself as having some design skill, and I did a lot of front-end work, but in 2009, I moved to New York City and worked in some big companies with strict divisions of labor. I was a backender, and backenders are never frontenders. So I’ve been away from the frontend for awhile. It is slowly becoming foreign territory to me. I was surprised to ...

March 24th, 2014

Why is the technology for blogs so difficult?

The web is broken and can never be fixed. The only way forward is to start over again, to get rid of the Internet Protocol, the Transmission Control Protocol and the Hypertext protocol, and to invent a new protocol that fixes the problems with the old protocols. I am doubtful this can be achieved.

Back in 2005, David Heinemeier Hansson offered a Rails tutorial showing how you could create a blog in 15 minutes

:

This was a world changing moment. ...

March 24th, 2014

Once again, the shift to “smart services, dumb pipes”

Yesterday I linked to the article over at Martin Fowler’s website where he wrote about the shift away from complex routing frameworks, towards a system of “smart services, dumb pipes”. Here is one more data point:

At Digg our SOA consisted of many Python backend services communicating with each other as well as being used by our PHP frontend servers and Tornado API servers. They used Apache Thrift for defining the interfaces, clients and as the underlying protocol.

…Coming off the Digg ...

March 24th, 2014

Often businesses handle a degree of inconsistency in order to respond quickly to demand

Perfect consistency is too rigid for most businesses, and it is painful when technical teams try to enforce this on a company, out of some ideological commitment to doing things the “correct” computer science way. “Eventual consistency” has been the standard that businesses have striven after since the Arab-Hindu cultures first invented dual-entry accounting, more than 500 years ago, and this is the standard that tech teams should enable for the businesses they serve.

Choosing to manage inconsistencies in this ...

March 24th, 2014

What kind of standards are useful to your team?

I love this:

Its a bit of a diochotomy that microservice teams tend to eschew the kind of rigid enforced standards laid down by enterprise architecture groups but will happily use and even evangelise the use of open standards such as HTTP, ATOM and other microformats.

The key difference is how the standards are developed and how they are enforced. Standards managed by groups such as the IETF only become standards when there are several live implementations of them in the wider ...

March 24th, 2014

The conceptual model of the world will differ between systems

Decentralization of data management presents in a number of different ways. At the most abstract level, it means that the conceptual model of the world will differ between systems. This is a common issue when integrating across a large enterprise, the sales view of a customer will differ from the support view. Some things that are called customers in the sales view may not appear at all in ...

March 23rd, 2014

A complexity that is frankly breathtaking

How can anyone possibly think this is a good idea?

To quote James Lewis and Martin Fowler:

Certainly, many of the techniques in use in the microservice community have grown from the experiences of developers integrating services in large organisations. The Tolerant Reader pattern is an example of this. Efforts to use the web have contributed, using simple protocols is another approach derived from these experiences – a reaction away from central standards that have reached a complexity that is, frankly, ...

March 23rd, 2014

The pushback against the monolithic framework

I am pleased to think that others are as ready as I am to abandon the concept of the monolithic framework:

Monolithic applications can be successful, but increasingly people are feeling frustrations with them – especially as more applications are being deployed to the cloud . Change cycles are tied together – a change made to a small part of the application, requires the entire monolith to be rebuilt and deployed. Over time it’s often hard to keep a good modular ...

March 23rd, 2014

Leave the error checking in your code

I leave the asserts in my Clojure code. I see a similarity of spirit expressed in the sentiment of James Hague (I especially like the use of the word “reckless”):

That error checking is great during development was not controversial, but opinions after that were divided. One side believed it wasteful to keep all that byte and cycle eating around when you knew it wasn’t needed. The other group claimed you could never guarantee an absence of bugs, and wouldn’t ...

March 23rd, 2014

A defense of MongoDB

I posted this on Hacker News and now re-post it here.

MongoDB offers the greatest benefit to those who have an evolving concept of their schema, and that tends to be startups, though I have worked in large firms that entirely re-invented their schemas. I worry that I would seem tedious if I listed the places that I have worked, and yet, on Hacker News, when I speak in abstract terms, I tend to get downvoted, so I will name a ...

March 21st, 2014

Announcing Humorus-MG

I just released Humorus-MG. This is an admin CMS for managing a collection in MongoDB. The app is written in Clojure.

The README contains an unintentional mini-manifesto of what I believe about creating web software. This part in particular comes close to summarizing the kind of software that I would like to create this year:

—————-

1.) This app will never have more than 2,000 lines of Clojure code. None of my apps will ...

March 20th, 2014

Why I use MongoDB

I posted this on HackerNews. I am in agreement with what Jun Xu wrote.

I think this is true:

“For a technology startup with limited resources, broadly adopting a new DBMS means betting its own future on the DBMS. ”

It has become popular to attack MongoDB, but I think it is difficult to get an objective view of what people are doing with it. If you want to read a really scathing attack on MongoDB, consider this post:

http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/

But I recall reading that ...

March 9th, 2014

Criticker gives away all of its users passwords

Criticker gives away all of its users passwords in plain text. Of course the site is written in PHP. While you can make mistakes in any language, this kind of laziness is what you expect in PHP.

Every request contains the secret key in the url. So all I need to do is capture a single request sent by the app and I have the key. Easy.

My theory was that I’d get the list of users that the app had ...

March 8th, 2014

Working with images using Clojure

I am intrigued by Mike Anderson’s “imagez” library:

Source

March 8th, 2014

How to monitor Clojure apps?

Interesting:

Powerful stream primitives

(where (or (service #”^api”) (service #”^app”)) (where (tagged “exception”) (rollup 5 3600 (email “dev@foo.com”)) (else (changed-state (email “ops@foo.com”)))))

Riemann streams are just functions which accept an event. Events are just structs with some common fields like :host and :service You can use dozens of ...

March 8th, 2014

How to bankrupt a successful software company

Interesting:

Quark 5 and OS9 was what we were used to, but it was pretty miserable. The things that stick out:

Restarting your computer and losing your unsaved work over software freezes was a regular part of your day. Like, many times a day. We had all these crazy workarounds to achieve certain effects like drop shadows or change-and-repeat. It was all pretty rudimentary and hard to standardize across many designers in a department. Shapes were pretty much a non-issue, so we had to ...

March 7th, 2014

The downside of Unit Testing

Interesting:

I’m back in Java-land these days, which is culturally very pro-unit testing. After getting exposed to it again for a few months again I’ve come to side with the author here. I’ve never really been comfortable with the amount of time certain people dedicate to unit testing, especially the TDD crowd, but in my hiatus something has arisen in popularity which has made it all the worse: mockito. Prior to mockito, unit testing was (more or less) limited to testing that ...

March 4th, 2014

Darren Holloway walks through the philosophy of Ring/Clojure

Darren Holloway has written a post that should be added to the wiki on Github where Ring is hosted. He covers all the stuff that had me the most confused when I started doing web development with Clojure. He offers easy examples in pseudo-code to get the basic ideas across. I wish every project on Github had an introductory tutorial written in this style.

An excerpt:

Ring Conceptually

Technically, Ring isn’t a framework or an application, but rather a specification ...

March 4th, 2014