They cost of using Hystrix

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com

Ouch!

We cannot use Clojure’s concurrency primitives (futures/promises/agents).

That is fascinating to think that at some point Clojure’s concurrency primitives are not enough, and so we need to give up on them and move to a Java library. I am aware that Netflix is dealing with unusual scale, but what is the point of Clojure if it doesn’t automate exactly these issues?

The comments are interesting:

Marc:
When using Aleph, the choice for lamina would seem a sure thing. What made you turn to RxJava?

The answer:

My main reasons were:

1. RxJava works very well with Hystrix.
2. We have good experience at SoundCloud of using RxJava in our Android apps.
3. The documentation of RxJava is of a very high standard.
4. RxJava has been proven in high load environments at Netflix.

I’m not confident enough yet about Lamina in production at scale but its definitely a project I’m watching closely.

Of course, Lamina is now Manifold, which I use.

Harry:
Can you explain why it is a problem that Clojure `future` calls all use same pool? My understanding was that futures use the same pool as the agent send-off function, which is unbounded. If that is indeed the case then how could that thread pool block, and why would it be advantageous to use multiple thread pools?

The answer:

While the future thread pool is unbounded in reality there is a breaking point where you are consuming all your machines resources creating new threads and managing existing ones.

In some situations this could be fine (no sudden spikes or bursts which all spawn new threads). However in this case I wanted to ensure that a failure or slow latency does not result in the threadpool filling up and hence impacting the function of the system. In scheduling over a number of thread pools of a limited size we rarely encounter a situation where a single problem takes down the entire ship (the Bulkhead pattern). And we can fine tune the settings to adapt and grow as usage evolves.

Source