Type: Research Highlight
Title: Resource Bricolage for Parallel DBMSs on Heterogeneous Clusters
Jiexing Li, Jeffrey Naughton, Rimma V. Nehme
Available in: PDF
heterogeneous resources has become increasingly common, due to cluster evolution and increasing interest in moving applications into public clouds or shared infrastructures. For database systems running in a heterogeneous cluster, the default uniform data partitioning strategy may overload some of the slow machines while at the same time it may underutilize the more powerful machines. Since the processing time of a parallel query is determined by the slowest machine, such an allocation strategy may result in a significant query performance degradation. We take a first step to address this problem by introducing a technique we call resource bricolage that improves database performance in heterogeneous environments. Our approach quantifies the performance di↵erences among machines with various resources as they process workloads with diverse resource requirements. We formalize the problem of minimizing workload execution time and view it as an optimization problem, and then we employ linear programming to obtain a recommended data partitioning scheme. We verify the effectiveness of our technique with an extensive experimental study on a commercial database system.
Jiexing Li is a software engineer at Google. She received her Phd from University of Wisconsin-Madison under the supervision of Prof. Jeff Naughton. During her Phd study, she also worked as a research assistant in Microsoft Jim Gray Systems Lab headed by Prof. David DeWitt.
Rimma V. Nehme is a technical advisor in Microsoft Data Group and is responsible for designing, building and evaluating new technologies for Microsoft database products. Since 2009, she has made critical contributions to Microsoft database products and technologies, including SQL Server, SQL Server PDW, Azure SQLDB, Azure SQLDW, SQL Server Live Query Statistics, and Partial Results for Database Systems. She is one of the original founders of the PolyBase technology, integrating structured and unstructured worlds of data. Rimma graduated with a bachelor degree in Computational Mathematics from Hillsdale College, MS degree in Computer Science from Worcester Polytechnic Institute, PhD degree in Computer Science from Purdue University, and MBA degree from the University of Chicago.