A Runtime Analysis of Bias-invariant Neuroevolution and Dynamic Fitness Evaluation

Fischer, Paul and Warwicker, John Alasdair and Witt, Carsten (2024) A Runtime Analysis of Bias-invariant Neuroevolution and Dynamic Fitness Evaluation. In: GECCO '24 : Proceedings of the Genetic and Evolutionary Computation Conference. UNSPECIFIED, New York, pp. 1560-1568. ISBN 9798400704949

Abstract

In the field of neuroevolution (NE), evolutionary algorithms are used to update the weights, biases and topologies of artificial neural networks (ANNs). A recent theoretical work presented the first runtime analysis of NE in a simple setting, considering a single neuron and intuitive benchmark function classes. However, this work was limited by the unrealistic settings with regard to activation functions and fitness measurements. In this paper, we extend upon this first work by overcoming the two shortcomings. Firstly, we consider a more realistic setting in which the NE also evolves a third parameter, termed the bend, allowing the previous benchmark function classes to be solved efficiently even in the fixed bias case. This setting mimics rectified linear unit activation functions, which are common in real-world applications of ANNs. Secondly, we consider a dynamic fitness function evaluation paradigm where the weights and biases are updated after each new sample. Experimental results in both cases support the presented theoretical results.