How much data is processed in a typical eDiscovery event? Hundreds of thousands of files? Millions? Billions? Many of the cases our clients are dealing with creep up into the millions very quickly. In fact one government agency is processing over one billion emails to support a specific case.
When we architected the Index Engines discovery platform we knew it had to scale to support large volumes of data. Therefore we made sure the index footprint was as small as possible, about 5 to 8% of the original data size. Others were not as cautious. Their index may require a cache copy of the file and as a result is bloated – resulting in an index size of 60 to 120% of the original data.
It became very clear at a recent trade show just how much a bloated index adds to the overall cost. At this event, Index Engines was located next to Google. Google has an indexing appliance, as do we. They make a cache copy of each file for indexing, so their index is at least 100% of the original data size. As a result, for a project that includes 30 million files they need 6 to 8 servers to process the job. Our technology, with a 5 to 8% index footprint, only requires one server (actually half a server) and 1/10th the cost.
It pays to be efficient!