Topology-Aware Deep Reinforcement Learning for RIS Beamforming: A GNN-PPO and Risk-Sensitive Evaluation

Abstract

Reconfigurable intelligent surfaces (RIS) enable control of radio propagation via large arrays of passive reflecting elements. Optimizing RIS phase profiles for spectral efficiency is
challenging due to the high dimensionality of continuous actions and non-convex channel coupling. We cast RIS beamforming as a sequential decision problem and evaluate four reinforcement-learning (RL) agents—A2C, Graph-Neural-Network Proximal Policy Optimization (GNN-PPO), Soft Actor–Critic (SAC), and Quantile-Regression PPO (QR-PPO)—in a realistic simulator with mobility, dual-slope log-distance path loss, shadowing, and Rician fading. Using a common protocol and PCA/GNN feature extraction, we compare agents on rate (mean and variability), tail risk via CVaR at 5%, mean SNR, and wall-clock cost. GNN-PPO attains the best mean rate, the lowest variability, the highest CVaR at 5% (strong tail performance), and the highest mean SNR. A2C is the compute-efficiency winner with the shortest total time, SAC provides a balanced compromise, while QR-PPO is cost-inefficient and underperforms in the tails under our configuration. We discuss design insights and directions for scalable, risk-aware RIS control.