Reinforcement Learning in Game AI

Abstract

Reinforcement learning has produced some of the most dramatic demonstrations of AI capability -- from Atari game mastery to superhuman Go and chess play, from StarCraft II grandmaster performance to Dota 2 world champion defeat. Beyond headline results, game AI research has driven fundamental advances in exploration strategies, sample efficiency, multi-agent coordination, and hierarchical planning that transfer directly to robotics, autonomous systems, and scientific discovery. This study provides a comprehensive evaluation of ten reinforcement learning algorithms across six game domains spanning increasing complexity: classic Atari (ALE), continuous control (MuJoCo), perfect-information board games (Chess, Go), imperfect-information card games (Poker), real-time strategy (StarCraft II micromanagement), and cooperative multi-agent (Hanabi). Algorithms evaluated include DQN, Rainbow, PPO, SAC, AlphaZero, MuZero, R2D2, Agent57, MCTS-RL, and QMIX (multi-agent). Evaluation covers sample efficiency (performance vs. environment steps), final asymptotic performance, computational requirements, and generalisation to unseen game variants. MuZero achieves the best mean performance across board game and Atari domains (Atari median human-normalised score: 731%). PPO remains the most sample-efficient algorithm for continuous control. QMIX achieves 84.2% win rate on StarCraft II cooperative micromanagement. Agent57 is the first algorithm to exceed human baseline on all 57 Atari games. A taxonomy of game complexity and algorithm-game matching framework is proposed, with analysis of transfer learning between game domains.