XRL-SHAP-Cache : an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks

Xu, X. and Wu, F. and Bilal, M. and Xia, X. and Dou, W. and Yao, L. and Zhong, W. (2024) XRL-SHAP-Cache : an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks. Science China Information Sciences, 67 (7): 170303. ISSN 1674-733X

Text (XRL_SHAP_Cache__An_Explainable_Reinforcement_Learning_Approach_for_Intelligent_Edge_Service_Caching_in_Content_Delivery_Networks)
XRL_SHAP_Cache_An_Explainable_Reinforcement_Learning_Approach_for_Intelligent_Edge_Service_Caching_in_Content_Delivery_Networks.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (2MB)

Abstract

Content delivery networks (CDNs) play a pivotal role in the modern internet infrastructure by enabling efficient content delivery across diverse geographical regions. As an essential component of CDNs, the edge caching scheme directly influences the user experience by determining the caching and eviction of content on edge servers. With the emergence of 5G technology, traditional caching schemes have faced challenges in adapting to increasingly complex and dynamic network environments. Consequently, deep reinforcement learning (DRL) offers a promising solution for intelligent zero-touch network governance. However, the black-box nature of DRL models poses challenges in understanding and making trusting decisions. In this paper, we propose an explainable reinforcement learning (XRL)-based intelligent edge service caching approach, namely XRL-SHAP-Cache, which combines DRL with an explainable artificial intelligence (XAI) technique for cache management in CDNs. Instead of focusing solely on achieving performance gains, this study introduces a novel paradigm for providing interpretable caching strategies, thereby establishing a foundation for future transparent and trustworthy edge caching solutions. Specifically, a multi-level cache scheduling framework for CDNs was formulated theoretically, with the D3QN-based caching scheme serving as the targeted interpretable model. Subsequently, by integrating Deep-SHAP into our framework, the contribution of each state input feature to the agent’s Q-value output was calculated, thereby providing valuable insights into the decision-making process. The proposed XRL-SHAP-Cache approach was evaluated through extensive experiments to demonstrate the behavior of the scheduling agent in the face of different environmental inputs. The results demonstrate its strong explainability under various real-life scenarios while maintaining superior performance compared to traditional caching schemes in terms of cache hit ratio, quality of service (QoS), and space utilization.