(Warning contains graphic abuse of the type system in Scala code).In languages like Scala, and of course Erlang, a lot of the focus is on Actors and message passing as a way to achieve concurrency and parallelism without using share-memory threads.
This is great, works awesome, and I love it. But its not all of the story. Sometimes you have a need to "fan out" a whole lot of slow/blocking things, and then join it all back together later on - a classic example used is hitting up a lot of remote resources at once (and very easy to do). Or, perhaps you have quite a few processors available, and a few computationally heavy tasks to do etc...
Well despite the bashing multi-threaded programming gets, the reality is threads are here to stay. But they are hard and I can't really do then.
Thankfully there are those smarter than me that have built lovely frameworks that take away most of the management of threads from us mortals (such as Doug Lea and java.util.concurrent on the JVM, and Grand Central Dispatch on OS-X - (and soon to be in BSD !)).
Speaking to Jon Tirsen of Google, he was musing over the idea of a language based around 100% async dispatch where everything was returned as a Future - pretty much the opposite to to lazy languages:
So I looked into java.util.concurrent.Executors - you can create executors (based on thread pools that you don't manage) and pass it "callable" tasks to run asynchronously. The trick is getting results back - this is where Futures come in. So when you submit a task to an executor, it returns a Future immediately - the future is a promise of a value at some point. When you call "get()" on the Future, it then blocks until it has a result (or course, hopefully it already has one). In java, this is understandable cumbersome, but works:
ExecutorService service = ...
Future result = service.submit(new Callable() {
public String call() {
return ... expensive computation ...
}
});
So result is of type Future of T - where T is the type of the real result. You have to call result.get() and then it will block... ugh, crazy... but worth it.
Enter Scala, before creating a whole new language as Jon suggested, I thought I would give ordinary Scala a go:
Wouldn't it be nice to take a piece of code like:
val result = someExpensiveFunction(42)
and make it async, and not worry about Future and all that? Well by the power of greyskull, thunking and implicit conversions this is possible:
val result = future { someExpensiveFunction(42) }
Note that I didn't have to change anything other then wrap future {.. } around it. Now, as Scala has type inference, this looks cleaner, but future is of type Future[T], where T is the return type of someExpensiveFunction. However, due to implicit conversions, we can pretend it isn't - it will return immediately, and you can pass it around, and once it actually needs the value of result, then, and only then will it block (automagically) - hopefully by then it will have its result !
So, how does it work:
object FutureProcess {
val executor = Executors.newCachedThreadPool
implicit def fromFuture[T](future: Future[T]) : T = future.get
def future[T](lambda: => T) : Future[T] =
executor.submit(new Callable[T] { def call=lambda })
}
So the above is all of it.
- we create an executor (its in an object so we only have one instance of it, easy). In this case its just a thread pool (but you could size it based on CPU cores if you like).
- we have an implicit def, which when it sees a Future of T, but needs T, it will call "get" (ie block and wait for result).
- the Future function: take any "block" of code, which returns T, and submit it wrapped in a Callable (nice how compact it is). The only clever bit is the "thunk" - the "=> T" bit - which means it is a parameter which isn't evaluated when it is called, but when it is needed - this is important as it means it isn't evaluated until it is safely submitted to the executor (and thus has a separate thread doing it) and can return immediately.
So the happy result is that all you do is import FutureProcess, and then wrap future { ... } around the bits you want done async, but really, no other code changes. Classic scala, slightly mind bending at the library level, but a breeze to use.
Enjoy.
