Dynamic Early-Exiting Networks for Efficient Deep Learning

Peng, Haoran and Soriano Marcolino, Leandro (2026) Dynamic Early-Exiting Networks for Efficient Deep Learning. PhD thesis, Lancaster University.

Text (2026HaoranPengPhD)
2026HaoranPengPhD.pdf - Published Version
Download (2MB)

Abstract

Dynamic neural networks are an effective approach for accelerating deep neural networks. Compared with traditional deep neural networks, dynamic neural networks can adjust their architectures based on different inputs. This helps reduce computational cost. Although they have been widely applied across various domains, several fundamental questions remain open. In this thesis, we investigate several fundamental questions in dynamic networks, including how to address gradient conflicts among classifiers during training, how to enable effective collaboration among classifiers during inference, and how to apply dynamic neural networks to deep reinforcement learning. As a representative form of dynamic neural networks, early-exiting networks are often used to investigate such foundational research questions. By adding extra classifiers in the middle of the network, early-exiting networks can exit early at these intermediate classifiers. This thesis addresses these research questions by examining early-exiting networks from three perspectives: training, inference, and the application in Monte Carlo Tree Search (MCTS).