18 Apr 2024

Artificial Intelligence (AI)–Enabled Wargaming Agent Training

Abstract: Fiscal Year 2021 (FY21) work from the Engineer Research and Development Center Institute for Systems Engineering Research lever-aged deep reinforcement learning to develop intelligent systems (red team agents) capable of exhibiting credible behavior within a military course of action wargaming maritime framework infrastructure. Building from the FY21 research, this research effort sought to explore options to improve upon the wargaming framework infrastructure and to investigate opportunities to improve artificial intelligence (AI) agent behavior. Wargaming framework infrastructure enhancements included updates related to supporting agent training, leveraging high-performance computing resources, and developing infrastructure to support AI versus AI agent training and gameplay. After evaluating agent training across different algorithm options, Deep Q-Network–trained agents performed better compared to those trained with Advantage Actor Critic or Proximal Policy Optimization algorithms. Experimentation in varying scenarios revealed acceptable performance from agents trained in the original baseline scenario. By training a blue agent against a previously trained red agent, researchers successfully demonstrated the AI versus AI training and gameplay capability. Observing results from agent gameplay revealed the emergence of behavior indicative of two principles of war, which were economy of force and mass.