18 Apr 2024

Artificial Intelligence (AI)–Enabled Wargaming Agent Training

Abstract: Fiscal Year 2021 (FY21) work from the Engineer Research and Development Center Institute for Systems Engineering Research lever-aged deep reinforcement learning to develop intelligent systems (red team agents) capable of exhibiting credible behavior within a military course of action wargaming maritime framework infrastructure. Building from the FY21 research, this research effort sought to explore options to improve upon the wargaming framework infrastructure and to investigate opportunities to improve artificial intelligence (AI) agent behavior. Wargaming framework infrastructure enhancements included updates related to supporting agent training, leveraging high-performance computing resources, and developing infrastructure to support AI versus AI agent training and gameplay. After evaluating agent training across different algorithm options, Deep Q-Network–trained agents performed better compared to those trained with Advantage Actor Critic or Proximal Policy Optimization algorithms. Experimentation in varying scenarios revealed acceptable performance from agents trained in the original baseline scenario. By training a blue agent against a previously trained red agent, researchers successfully demonstrated the AI versus AI training and gameplay capability. Observing results from agent gameplay revealed the emergence of behavior indicative of two principles of war, which were economy of force and mass.

17 Apr 2024

Enabling Understanding of Artificial Intelligence (AI) Agent Wargaming Decisions through Visualizations

Abstract: The process to develop options for military planning course of action (COA) development and analysis relies on human subject matter expertise. Analyzing COAs requires examining several factors and understanding complex interactions and dependencies associated with actions, reactions, proposed counteractions, and multiple reasonable outcomes. In Fiscal Year 2021, the Institute for Systems Engineering Research team completed efforts resulting in a wargaming maritime framework capable of training an artificial intelligence (AI) agent with deep reinforcement learning (DRL) techniques within a maritime scenario where the AI agent credibly competes against blue agents in gameplay. However, a limitation of using DRL for agent training relates to the transparency of how the AI agent makes decisions. If leaders were to rely on AI agents for COA development or analysis, they would want to understand those decisions. In or-der to support increased understanding, researchers engaged with stakeholders to determine visualization requirements and developed initial prototypes for stakeholder feedback in order to support increased understanding of AI-generated decisions and recommendations. This report describes the prototype visualizations developed to support the use case of a mission planner and an AI agent trainer. The prototypes include training results charts, heat map visualizations of agent paths, weight matrix visualizations, and ablation testing graphs.