April 9th, 2012
(written by lawrence, however indented passages are often quotes)
When you’re forced into parallelism at that level of detail, you quickly realize how difficult it is to have a generic way of doing it. The fact that languages like scala and clojure will allow you to just say “apply this logic over the collection in parallel” is very cool, and a great way for developers to dip their toes in parallel processing of their collection data. But it breaks down quickly when you are talking about vastly different types of data running in vastly different types of computation. If you need to eke out every ounce of performance, you want every computation to be tuned to the right batch size and the right number of threads, and the size of the thread pool for task X may depend on whether it tends to run at the same time as task Y or task Z. Don’t forget that task Z, if requested, must be prioritized above all else. The size of the batches themselves may depend on the type of object inside them, and how that affects the cache lines of the processor you’re running on. Oh, and of course, you’re not just tuning this for prod, it has to at least run to completion on your integration boxes and they are of course slightly different processors. And don’t forget, your developers aren’t writing code on Azul boxes, they’re writing code on desktops with 2 cores each; threading even the test data to a pool of 50 doesn’t make any sense, but they still need to run things in parallel or they’ll never see the stupid threading bugs they added when they forgot to use concurrent data structures.
I’ve always thought that the next truly big leap in VMs and languages will be those that can do smart things with threading without making developers sweat too much. If we’re really entering a world of more and more data over more but slower cores, we need a language and a VM that revolutionize our management of threads the way Java and the JVM have revolutionized our notion of memory management. The functional languages out there help, but you can’t forget the underlying VM magic that will really make this possible. Look to the example of memory management: it’s not Java itself that enables you to write code without worrying about deleting references only when they are truly not being used, but rather the VM that watches all objects and knows how to find the live ones and delete those that are no longer referenced, all without the developer needing to indicate a thing. For us to conquer the thread problem, we need a VM to observe the behavior of the system beyond just the number of cores and help us to appropriately occupy CPUs over collections of varying size and composition. Languages can force developers to think about applying logic over their data (and thus help with parallelism as a mindset), but they will never solve the problem of automated parallelism without the VM to tune it in the background.Source