olivier deheurles

As you may know I’ve spend some time porting the disruptor to .NET and I’m now quite happy with the current version. If you want to have a look you can find the source code on GitHub or play with the assemblies available on NuGet.

Of course feel free to fork and send pull request, I will do my best to merge your new goodies to the main repo.

I’m now planning to implement other core components described in Martin’s/Mike’s presentation. The next one on the list is the journaler.

Here is a quick overview of this component:

• it is responsible of persisting to disk every single message (byte array) received from the network adapter
• it is consuming messages from a ring buffer (disruptor) and should store messages to disk as fast as possible (minimize latency and maximize throughput)
• it is a mono-threaded component, writing to disk serially
• since the disruptor can deliver messages in batch, the journaler should take advantage of this mechanism: if multiple messages are available in the ring buffer the journaler thread should pick as many messages as possible before flushing to disk (nice mechanism to absorb messages bursts)
• the journaler should be designed with mechanical sympathy in mind: data is send to disk by block, filling blocks as much as possible before flushing to disk sounds like a good idea..
• the journaler should minimize allocations (GC pressure) and ideally be alloc free
• of course messages should be persisted to disk in a binary format that allow messages to be read later on and the journaler API should offer both read and write functionalities

You can find a more in depth overview of the Journaler and other LMAX components in Martin Fowler’s review of the LMAX architecture.

I’m going to create a new GitHub project to host this component. My plan is to first define the journaler API, then build a performance test project and then I will play around to test different implementations. I would like this component to be stand-alone and have no dependency on third party components (disruptor included).

If you are interested and want to contribute let me know: mail at odeheurles dot com