SlowCoach : Mutating Code to Simulate Performance Bugs

Chen, Yiqun and Schwahn, Oliver and Natella, Roberto and Bradbury, Matthew and Suri, Neeraj (2022) SlowCoach : Mutating Code to Simulate Performance Bugs. In: 33rd IEEE International Symposium on Software Reliability Engineering :. 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE) . IEEE, New York, pp. 274-285. ISBN 9781665451338

Text (permute)
permute.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (397kB)

Abstract

Performance bugs are unnecessarily inefficient code chunks in software codebases that cause prolonged execution times and degraded computational resource utilization. For performance bug diagnostics, tools that aid in the identification of said bugs, such as benchmarks and profilers, are commonly employed. However, due to factors such as insufficient workloads or ineffective benchmarks, software defects related to code inefficiencies are inherently difficult to diagnose. Hence, the capabilities of performance bug diagnostic tools are limited and performance bug instances may be missed. Traditional mutation testing (MT) is a technique for quantifying a test suite's ability to find functional bugs by mutating the code of the test subject. Similarly, we adopt performance mutation testing (PMT) to evaluate performance bug diagnostic tools and identify where improvements need to be made to a performance testing methodology. We carefully investigate the different performance bug fault models and how synthesized performance bugs based on these models can evaluate benchmarks and workload selection to help improve performance diagnostics. In this paper, we present the design of our PMT framework, SLOWCOACH, and evaluate it with over 1600 mutants from 4 real-world software projects.