Frimpong, Richlove and Csala, Denes and Soetanto, Danny (2019) Approximate Policy Iteration (API) with neural networks for the generalized single node energy storage problem. Masters thesis, Lancaster University.
2019frimpongmscbyresearch.pdf - Published Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.
Download (802kB)
Abstract
Energy storage problems are hard sequential decision-making problems often modelled as markov decision processes. Exact solution using dynamic programming quickly becomes implausible with large state spaces hence approximate dynamic programming using policy iteration (API) is often employed in such cases. API does not always work, one reason being that the approximation architectures used are often linear for computational tractability reasons. We propose a mathematical model which allows easier implementation of non-linear approximations with API. We use neural networks along with monte-carlo simulation to predict the future values for the generated states during the improvement step of the API algorithm. Our initial experiments suggest that the proposed method provides good results which can be further improved with more fine tuning of the neural network parameters.