CCR 101 - Part 1: Introduction
[This post is intended to provide an introduction to the CCR, complementing other introductory materials online (links at the end)].
So we agree that multi-threading programming is hard right? Or, at least that it gets hard quickly. Sure, if you only have a single task you want to execute in parallel with your main code, you can just spin up a thread, or maybe even use the BackgroundWorker class. But the thing about multi-threading is that things start to get tricky quite quickly:
- Can I easily cancel this task? This suggests some inter-thread communication. It also suggests that there are points in the long-running task where I can check for this pending cancellation.
- How can I detect if the task fails? More inter-thread communication is required. I also need to worry about the thread termination in the failure class.
- Actually, how do I detect if the task completes at all? Can it timeout?
- What if I'm writing a server application? I can't really spawn a thread for each request - this is not a model that scales well on Windows.
So even reasoning about this parallelism is complicated by the fact that I pretty much have to do a lot of the basic scaffolding myself.
What about the thread-pool?
Now the thread-pool helps a lot, and combined with C# 2.0 features like anonymous delegates, I can quite succinctly capture simple operations, but I still have the issues of propagating the success/failure of the result back to the original instigator. What's more, as the complexity of the operation increases, anonymous delegates break down as an implementation mechanism. And traditional 'named' delegates often tend to split the same logical operation across multiple physical ones which introduces different readability and maintainability issues.
And there's one more aspect of the thread-pool that's often overlooked. It is essentially just a single queue of work items. They might actually be dispatched concurrently across the available processors, but if you post a whole lot of items into the pool, they'll be processed approximately in turn. If you have logical requirement to support priority queues, then you'll need to implement these yourself with or without the thread-pool.
So is the CCR just a better thread-pool?
Well, it is a better thread-pool, but it's quite a bit more as well. The CCR provides some library primitives that allow you to more easily to exploit the available processing power of your machine, which is probably quite a lot (although the CCR runs on some fairly minimal implementations of Windows). It does this by providing a lightweight message-passing infrastructure that can easily be rolled on top of existing asynchronous programming models. In addition, it supports a pseudo-declarative style of co-ordinating messages, a mechanism for propagating cross-thread failure in a consistent fashion and support for priority (or at least multiple) work queues to avoid the starvation problem outlined above. The CCR will dispatch your work items across the available processors, either over the existing CLR thread-pool or via a custom pool of dedicated threads.
Back up a bit - what do you mean by message passing?
Basically, rather than explicitly co-ordinating shared data between threads (e.g. via events, semaphores etc), parallel dependent tasks pass messages between one another. These messages travel over queues and tasks are registered to run on the arrival of the message. Queues may be short-lived or long-lived and can be passed around between tasks. Any form of WIndows UI programming is ultimately an exercise in message passing, albeit a less dynamic one. Message passing as a technique for describing parallel computations is nothing new, but the CCR brings it within range of any competent .NET developer.
Sounds a bit slow to me...
The CCR is very lightweight and has clocked some impressive figures. It's quite possible that a custom solution using native OS primitives to schedule co-ordinating threads would be quicker, but it probably wouldn't be as simple or as consistent and more importantly, probably wouldn't scale as well as the number of available cores increases.
And what's the Asynchronous Programming Model?
Well, the actual nuts-and-bolts of the Asynchronous Programming Model (APM) have been described elsewhere; what I just want to mention is really the spirit of the thing, and this takes us back to the introduction of I/O completion ports in NT3.51. IOCPs really surfaced to the C/C++ application developer the underlying reality that I/O took place off the processor. The CPU would dispatch a disk I/O (or a network I/O) via a device driver and then go and do something else until an interrupt fired, signalling completion. Abstractly, IOCPs propagated this interrupt into the application code, allowing the developer to instigate an I/O, do something else, and have some other code fire when it completed. Coupled with an efficient task-dispatching mechanism that worked over a small number of threads (ideally one-per-processor to eliminate the overhead of a context-switch), IOCP-based programs could reach tremendous levels of scalability and responsiveness. Of course, they were extremely difficult to write correctly, especially in C/C++. They still are difficult to write, even with the APM, C# 2.0 and the CLR thread-pool, but the principle remains:
Always perform I/O asynchronously; never block a thread that could otherwise do useful work.
As I'll demonstrate in future posts, the CCR has facilities that allow the developer to remain true to this principle, whilst avoiding many of the implementation pitfalls of doing it all 'manually'.
