I'm sorry Dave, I'm afraid I can't do that.

I have finally bitten the bullet and got my own site. All my old posts have been copied there. Hope to see you there!

I'm moving this blog to live spaces. The feed can be found here. Despite the rather idiotic restriction of only one category-per-post, I've found it marginally easier to format in-line code there, and the ability to put code samples up on a publicly available folder was the clincher.

Proof perhaps that beggars can be choosers....

F# ships with library support for ‘mailboxes’, an asynchronous message-passing mechanism recently popularised by interest in Erlang. This is good news. Message-passing in concurrent systems is always a saner approach than explicitly spawning threads and taking locks, and F#’s support for immutable types and values provides additional safety. Combined with the asynchronous workflows that the language also supports, a powerful Actor-oriented programming model is available to F# developers.

If you’re playing in this space in .Net though, I still think you can’t go past the Concurrency and Coordination Runtime (CCR), especially on performance terms, and it turns out to be straightforward to integrate the CCR with F# asynchronous workflows.

Let’s start out by defining some basic CCR support – this is mostly just plumbing…

#light

namespace FSharp.Ccr.Adaptors

open System
open System.Threading
open Microsoft.Ccr.Core
open Microsoft.Ccr.Core.Arbiters

type Receive() = 

    static let taskQueue = new DispatcherQueue()
    
    static member internal TaskQueue with get() = taskQueue
    
    static member internal activate(task : ITask) = 
        Arbiter.Activate(taskQueue, [| task |])

In the declaration above, there is only one task queue over the standard CLR thread-pool. I’ll cover the support for specifiable task queues in a later post. Now let’s cover some adaptor code that enables the use of the basic arbiters from within F# asynchronous workflows...

    (* Receive. This allows us to write an asynchronous workflow that waits (but does not block) 
       on the port as follows: 
    let recv (port : Port<string>) =
        async {
            let! item = Receive.from port
            printfn "%s" item
        }
     *)    
    [<OverloadID("from")>]
    static member from (p : Port<'a>) = 
        Async.Primitive(fun (cont, econt) ->
            Arbiter.Receive(false, p, (fun result -> cont result))
            |> Receive.activate)
    
    (* JoinedReceive. This allows us to write an asynchronous workflow that waits (but does not block) 
       until both ports have an item for processing. It then returns both in a tuple: 
    let recv (p1 : Port<string>, p2 : Port<int>) =
        async {
            let! s1,i1 = Receive.from_both (p1,p2)
            printfn "%s %i" s1 i1
        }
     *)
    [<OverloadID("from_both")>]
    static member from_both (p1 : Port<'a>, p2 : Port<'b>) =
        Async.Primitive(fun (cont, econt) ->
            Arbiter.JoinedReceive(false, p1, p2, fun a b -> cont (a,b))
            |> Receive.activate)
    
    (* Choice. This allows us to write an asynchronous workflow that waits (but does not block) 
       until either port has an item for processing. It then returns the result in a
       Choice:
    let recv (p : PortSet<string,int>) =
        async {
            let! result = Receive.from_either p
            match result with 
            | Choice2_1 s -> printfn "%s" s
            | Choice2_2 i -> printfn "%i" i
        }
     *)    
     [<OverloadID("from_either")>]
    static member from_either (ps : PortSet<'a,'b>) =
        Async.Primitive(fun (cont, econt) ->
            Arbiter.Choice(ps, (fun s -> cont (Choice2_1 s)), (fun e -> cont (Choice2_2 e)))
            |> Receive.activate) 
    
    (* MultipleItemReceive. This allows us to write an asynchronous workflow that waits (but does not block) 
       until n items have queued to the port. It then returns the result in an array.
    let recv (p : Port<string>) =
        async {
            let! result = Receive.multiple_from (p,100)
            printfn "%i items received." result.Length
        }
     *) 
    [<OverloadID("multiple_from")>]
    static member multiple_from (p : Port<'a>, n : int) =
        Async.Primitive(fun (cont, econt) ->
            Arbiter.MultipleItemReceive(false, p, n, fun r -> cont r)      
            |> Receive.activate) 
    
    (* MultipleItemReceive. This allows us to write an asynchronous workflow that waits (but does not block) 
       until n items have queued to the portset. It then returns the result as a tuple of
       collections.
    let recv (p : PortSet<string,Exception>) =
        async {
            let! results,failures = Receive.multiple_from_both (p,100)
            printfn "%i results received and %i failures." results.Count failures.Count
        }
     *) 
    [<OverloadID("multiple_from_both")>]
    static member multiple_from_both (p : PortSet<'a,'b>, n : int) =
        Async.Primitive(fun (cont, econt) ->
            Arbiter.MultipleItemReceive(p, n, fun f s -> cont (f,s))      
            |> Receive.activate)
    
    (* MultiplePortReceive. This allows us to write an asynchronous workflow that waits (but does not block) 
       until each port in the specified array has a pending item. It then returns the 
       result as a array.
    let recv (p : Port<string> array) =
        async {
            let! results = Receive.from_multiple p
            printfn "%i results received." results.Length
        }
     *) 
    [<OverloadID("from_multiple")>]
    static member from_multiple (p : Port<'a> array) =
        Async.Primitive(fun (cont, econt) ->
            Arbiter.MultiplePortReceive(false, p, fun r -> cont r)      
            |> Receive.activate)

The Overloads are defined because each of these members has an alternate version that takes a timeout limit. I’ll cover those in the next post.

This has been a pretty content-free zone lately. Most of that has been due to work pressure. I've got a few systems going live around now and, as always, the last 20% of each project takes 80% of the time. Anyway, I have been looking at a few things here and there that might be of interest...

First off, I noticed that George Chrysanthakopoulos has announced on the MSRS CCR forum that work is underway to unify the CCR with the PFX libraries recently released to CTP, although no time-frames have been given. Additionally, George also states that they are working towards the integration of the CCR into the core .NET libraries (possibly via PFX?). This is excellent news all round. I read with interest the MSDN article on the Task Parallel library, which sits at the core of PFX. Actually, in my line of work, I believe the CCR would be more useful than PFX, as taming the asynchrony, whilst exploiting the latent concurrency tends to be the bigger (and harder) problem than parallelising the processing of the data itself. Nevertheless, it's as a good a vehicle as any in which to ship the CCR and means that in future, if you're considering the library and don't work in robotics you won't have to try and convince your boss why buying and deploying MSRS is a good idea.

I've also been following the herd and researching F#, following the announcement that Microsoft are to fold it into the set of officially supported .NET languages. I've noted before what I liked in general about the language, but readers of this blog might be interested in what it offers in the sort of space that CCR plays in.

• Immutability. In common with many functional languages the default syntactic declaration of a 'variable' is immutable. Immutability is a generally useful property in concurrent programming and message-passing systems in particular. F# makes it simple to both declare types (records) and instances of them in a way well-suited to message-passing.
• Composition. Functional languages tend to compose and decompose solutions very well (see this classic introduction). This ability helps factor out a lot of concurrent machinery, whilst still exploiting it. In fact, Joe Armstrong called this 'Abstracting out Concurrency' in his PhD thesis, and Erlang programs, whilst naturally concurrent, tend to make heavy use of these common patterns.
• The latest version of F# contains support for 'workflows', which are very similar in concept to CCR iterators, although an entirely different implementation under the covers. Robert Pickering's blog has covered these in a recent series of tutorials, as well the Erlang-style message-passing approach (something akin to one-time CCR receivers in a read-process loop) it supports.

Incidentally, I read Robert's book, Foundations of F# as my first foray into the language. I thought the book was pretty good, but felt (as an FP novice) that it might have benefited more by explaining a little more about FP style and accumulated good practice. Anyway, since then I have also read Practical OCaml, since F# aims to be compatible with a subset of OCaml. If you're considering either, my personal recommendation would be read the OCaml book first, and then read Foundations, since the former does a better job of introducing functional programming and the latter provides solid examples of integration with the existing BCL.

I've also been working on very lightweight COM-based C++ library for Erlang-style message-passing and task distribution over the native Win32 thread-pool. It's an ATL-style solution i.e. headers only and I'll blog a little more about this in future.

Joe Duffy points to the announcement that the Developer Division at Microsoft is folding F# into the 'official' set of supported .NET languages. This is great news.

I don't normally write about my job, so I'm going to write about a different one that could be yours...

My team has vacancies. We design, develop and support equity-derivatives trading systems for one of Australia's foremost financial institutions for markets at home and abroad. You'd like to work in a small, highly-skilled team of software developers and have some or all of the following:

• Strong C++ and optionally VB6 and/or .NET
• Significant experience of multi-threaded server application development
• SQL experience (preferably SQL Server)
• A solid understanding of IP-based communication protocols
• A working knowledge of equity-derivative products
• A working knowledge of client-side development

You will be a self-starter who can work with minimum supervision on projects from inception through delivery and onto support, across the various architectural tiers. You will be comfortable communicating with the key business stake-holders, primarily traders and business managers and be focused on delivering solutions that meet their needs.

Successful applicants would fill vacancies at our Sydney offices and should hold either Australian Citizenship or Permanent Residence. (There are opportunities abroad also).

If you're interested, please drop me a line via my gmail address (nagunn at ...) and we can take it from there.

My inexperience with F# has caught me out. Robert Pickering has an article on asynchronous workflows (similar concept to CCR iterators) in which he mentions a forthcoming article on 'mailboxes', the message-passing feature I said didn't exist. I look forward to it.