Optimizing Training Architectures for High-Dimensional, Multi-Agent Systems in Missile Defence

Abstract

This study is focused on identifying an effective state representation, neural network architecture, and multi-agent training paradigm for high-dimensional missile defence scenarios. We follow a progressive approach, beginning with simpler environments and problem structures, then extending our findings to more complex settings. Initially, we analyze the impact of state representations on agent learning in the MountainCar environment. We demonstrate that higher resolution representations, such as the Radial Basis Function (RBF) transformation paired with convolutional deep learning architectures, enhance agent performance. Furthermore, we provide evidence that even lower resolution representations with higher noise can be trained effectively when a higher resolution representation is used as input to the critic network. We then broaden our exploration to a multi-agent particle environment, where we investigate the relative merits of centralized and decentralized execution paradigms, finding the decentralized paradigm to consistently outperform the centralized one. Lastly, we introduce the custom missile defence environment, where we apply lessons learned and perform ablation studies to validate the generalizability of our findings. We compare the performance of our trained agents with hard-coded baseline agents, effectively demonstrating the success of our progressively complex approach in the domain of Deep Reinforcement Learning (DRL) methods for missile defence scenarios.