Towards improving weather and climate models using reinforcement learning

Moss, Henry and Nath, Pritthijit and Shuckburgh, Emily and Webb, mark (2024) Towards improving weather and climate models using reinforcement learning. In: American Geophysical Union Fall Meeting, 2024-12-01.

Abstract

As artificial intelligence (AI) based techniques come to the forefront of climate modelling, this study presents one of the first research efforts in integrating reinforcement learning (RL) with idealised climate models in the field of AI-assisted modelling. Addressing some limitations of contemporary AI-driven approaches, the use of RL benefits from direct environment interaction, ability to work with sparse reward signals, online learning, long-term planning, and high adaptability - crucial for dynamic climate systems of today. In this study, 16 experiments using 8 widely used continuous action model-free RL algorithms (REINFORCE, DDPG, DPG, TD3, PPO, TRPO, SAC, TQC) in total were performed on two different idealised climate system RL environments: one for simple climate bias correction (SimpleClimateBiasCorrectionEnv) and another for radiative-convective equilibrium (RadiativeConvectiveModelEnv), scaled on a high-performance parallel computing infrastructure such as JASMIN and constrained under a fixed set of episodic steps. Experiments demonstrate off-policy exploration heavy algorithms such as DDPG, TD3, TQC (for SimpleClimateBiasCorrectionEnv) and on-policy exploitation focused algorithms such as DPG, PPO, TRPO (for RadiativeConvectiveModelEnv) outperform others along with reducing largest biases in magnitude by upto 90% for RadiativeConvectiveModelEnv. The study's findings, using a specially scaled experimentation setup to focus on specific key best performing algorithms, highlight the potential of RL-based parameterisation schemes for integration into global climate models, leading to more accurate and efficient climate modelling. Code accessible at https://github.com/p3jitnath/climate-rl.