Optimizing MongoDB-Hadoop Performance with Record Grouping

Abstract

Computational cloud computing is more important than ever. Since time is literally money on cloud platforms, performance is the primary focus of researchers and programmers alike. Although distributed computing platforms today do a fine job of optimizing most types of workflows, there are some types, specifically those which are not computation-oriented, that are left out. After introducing important players in the world of computational cloud computing, this paper explores a possible performance enhancement for these types of workflows by reducing the overhead that platform designers assumed was acceptable. The enhancement is tested in two environments: an actual distributed computing platform and an environment that simulates that platform. Along the way it becomes clear that computational cloud computing is far from perfect and its use can often deliver surprising results. Regardless, the presented solution remains viable and is capable of increasing the performance of particular types of jobs by up to twenty percent.