Auto-tuning MPI Collective Operations on Large-Scale Parallel Systems

Zheng, Wenxu and Fang, Jianbin and Chen, Juan and Pan, Xiaodong and Wang, Hao and Huang, Chun and Sun, Xiaole and Tang, Tao and Wang, Zheng (2019) Auto-tuning MPI Collective Operations on Large-Scale Parallel Systems. In: The 21st IEEE International Conference on High Performance Computing and Communications :. IEEE, pp. 670-677. ISBN 9781728120591

[thumbnail of 1570538287]
Text (1570538287)
1570538287.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (665kB)


MPI libraries are widely used in applications of high performance computing. Yet, effective tuning of MPI colletives on large parallel systems is an outstanding challenge. This process often follows a trial-and-error approach and requires expert insights into the subtle interactions between software and the underlying hardware. This paper presents an empirical approach to choose and switch MPI communication algorithms at runtime to optimize the application performance. We achieve this by first modeling offline, through microbenchmarks, to find how the runtime parameters with different message sizes affect the choice of MPI communication algorithms. We then apply the knowledge to automatically optimize new unseen MPI programs. We evaluate our approach by applying it to NPB and HPCC benchmarks on a 384-node computer cluster of the Tianhe-2 supercomputer. Experimental results show that our approach achieves, on average, 22.7% (up to 40.7%) improvement over the default setting.

Item Type:
Contribution in Book/Report/Proceedings
Additional Information:
©2019 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
ID Code:
Deposited By:
Deposited On:
20 May 2019 08:10
Last Modified:
18 Jun 2024 23:45