Transducers are coming to all the core functions of Clojure

Interesting, though I think this is only a convenience:

One of the parts of the announcement of transducers was that clojure’s core functions (map, filter, take, etc) that normally operate on sequences will gain a new arity that returns a transducer when called with a single argument that’s a function. The new code for the 1-arity map looks much like what we already wrote (ignore the other arities for now).

;; Source of the new arity of the ‘map’ function.
(defn map
([f] ;; When called with a single argument,
(fn [f1] ;; map returns a transducer that accepts a reducing function.
(fn ;; Like our crude transducer, this transducer returns
;; a reducing function,
([] (f1))
([result] (f1 result))
([result input] ;; and this arity does the same as what ours did.
(f1 result (f input)))
([result input & inputs]
(f1 result (apply f input inputs))))))

I am a little startled that this one arity returns a function that returns a function with 4 different arities. On the one hand, I am impressed with the code that I could write with code this flexible. On the other hand, I doubt that I am intelligent enough to take advantage of these possibilities. I am unlikely to understand what such code does, and I can imagine myself lost in a fog of debugging, misunderstanding what my own code does.

And, in the end, all this really does is wrap a reduce function inside another reduce function, which I can do myself, in a plainer style, and the plainer style might be more readable for me.

All the same, I am grateful to the Clojure team for re-writing all the core functions to support a single-arity-function signature for all of the core functions. Perhaps with enough time I’ll learn how to take advantage of all of this flexibility.

There is a plainer style of coding, available in any programming language, where you take some object or array (in which you are accumulating results) and you walk it through loop after loop, until you have the final results that you want. But if you go that route, then you pay a high price in terms of performance, since you are going through multiple loops. Composing reduce functions is certainly a wiser way to go, since it means you only need to walk the sequence once. As the article says:

When using the thrush, or ->>, form above, map accepts an input sequence one by one, and runs inc over each of the elements, generating a new sequence, which is then reduced by reduce. This creation of logic via composition of sequence-iterating functions complects the created logic with the machinery of sequences. This also means that unnecessary intermediate sequences are created just to execute your logic.

If we could separate our logic from sequences, we could write more performant code by avoiding incidental sequences. We could also use the same logic across things that aren’t necessarily sequences (eg: clojure.core/reducers, channels).

Transducers let us do that by composing functions “inside” the traversal of the input, instead of composing sequence-traversing functions “outside” on the sequences themselves.

They are similar to traditional function composition (eg: comp) in this way, but a comped function iterated over a sequence can only know about a single element at a time, and doesn’t have state. Transducers can encapsulate state (eg: the take transducer uses an atom to count elements), and can control generation of output (eg: the filter transducer can decide whether or not to pass a token to the passed-in reducing function, whereas a composed function that’s mapped needs to generate one output per input).