Abstract
The rapid urbanization of modern metropolitan areas has precipitated a critical need for intelligent traffic management systems capable of mitigating congestion and reducing environmental impact. Traditional traffic signal control (TSC) systems, which rely on fixed-timing plans or simple actuation logic, are increasingly insufficient for handling the stochastic and non-linear dynamics of contemporary urban traffic flows. This inadequacy is exacerbated during the transitional phase of intelligent transportation systems (ITS), characterized by mixed traffic conditions containing both Human-Driven Vehicles (HDVs) and Connected Automated Vehicles (CAVs). This paper presents a comprehensive Multi-Agent Reinforcement Learning (MARL) framework designed to optimize signal control in such heterogeneous environments. We propose a decentralized Actor-Critic architecture that leverages the connectivity of CAVs to estimate traffic states while inferring the hidden states of HDVs through a novel probabilistic inference mechanism. By treating each intersection as an autonomous agent that cooperates with neighbors to optimize network-wide throughput, our approach addresses the scalability issues inherent in centralized systems. Extensive simulations demonstrate that the proposed method significantly outperforms state-of-the-art baselines in reducing average travel time and queue lengths, particularly in scenarios with low-to-medium CAV penetration rates.

This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2025 Liping Feng (Author)