Straggler root-cause and impact analysis for massive-scale virtualized cloud datacenters

Garraghan, Peter and Ouyang, Xue and Yang, Renyu and McKee, David and Xu, Jie (2019) Straggler root-cause and impact analysis for massive-scale virtualized cloud datacenters. IEEE Transactions on Services Computing, 12 (1). pp. 91-104. ISSN 1939-1374

[thumbnail of tsc2016b]
PDF (tsc2016b)
tsc2016b.pdf - Accepted Version
Available under License Creative Commons Attribution.

Download (2MB)


Increased complexity and scale of virtualized distributed systems has resulted in the manifestation of emergent phenomena substantially affecting overall system performance. This phenomena is known as “Long Tail”, whereby a small proportion of task stragglers significantly impede job completion time. While work focuses on straggler detection and mitigation, there is limited work that empirically studies straggler root-cause and quantifies its impact upon system operation. Such analysis is critical to ascertain in-depth knowledge of straggler occurrence for focusing developmental and research efforts towards solving the Long Tail challenge. This paper provides an empirical analysis of straggler root-cause within virtualized Cloud datacenters; we analyze two large-scale production systems to quantify the frequency and impact stragglers impose, and propose a method for conducting root-cause analysis. Results demonstrate approximately 5% of task stragglers impact 50% of total jobs for batch processes, and 53% of stragglers occur due to high server resource utilization. We leverage these findings to propose a method for extreme straggler detection through a combination of offline execution patterns modeling and online analytic agents to monitor tasks at runtime. Experiments show the approach is capable of detecting stragglers less than 11% into their execution lifecycle with 95% accuracy for short duration jobs.

Item Type:
Journal Article
Journal or Publication Title:
IEEE Transactions on Services Computing
Additional Information:
© 2019 IEEE. This is an author produced version of a paper published in IEEE Transactions on Services Computing. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Uploaded in accordance with the publisher's self-archiving policy.
Uncontrolled Keywords:
?? cloud computingstragglerdistributed systemsroot-cause analysisdatacenter information systems and managementcomputer science applicationshardware and architecturecomputer networks and communications ??
ID Code:
Deposited By:
Deposited On:
21 Oct 2016 10:30
Last Modified:
07 May 2024 00:08