Pure Julia MapReduce

Final Project for 6.338 Parallel Computing

This is still a work in progress, as I’m waiting on Julia mutlithreaded support to improve, but this is an effort to implement a full MapReduce framework in Julia. This includes a fault tolerant file system built on Paxos.

The basic architecture is that Excavator is a MapReduce system which reads files from Cairn, the distributed file system built on Rock, the Paxos implementation.