Multi-Agent Systems in Complex Environments

Abstract

Multi-agent systems (MAS) -- collections of autonomous agents that interact within shared environments to achieve individual or collective objectives -- are increasingly relevant as AI transitions from isolated models to networked systems of cooperating and competing agents. Complex environments introduce challenges that single-agent reinforcement learning cannot address: partial observability, non-stationarity (each agent's optimal policy depends on the changing policies of others), coordination under communication constraints, emergent collective behaviour, and scalability from small teams to large swarms. This study evaluates ten multi-agent system architectures across five complex environment classes: cooperative logistics planning, competitive market simulation, mixed-motive negotiation, decentralised power grid management, and large-scale social simulation. Architectures evaluated include independent Q-Learning (IQL), MAPPO (Multi-Agent PPO), QMIX, MADDPG, Mean Field MARL, communication-augmented MARL (CommNet, TarMAC), LLM-coordinated agents (AutoGPT-Multi, LangChain-Agents), and a proposed Adaptive Communication and Coordination Framework (ACCF). ACCF achieves the best mean performance across all five environment classes (composite score 8.24/10) by dynamically selecting between centralised and decentralised coordination based on environment complexity. MAPPO achieves the best cooperative logistics performance (task completion 84.2%). LLM-coordinated agents achieve the highest negotiation success rate (78.4%) at the cost of computational overhead. Mean Field MARL achieves the best scalability (1,000+ agents). A multi-agent system design taxonomy and deployment framework are proposed.